File compression code

[skywalker]

I was looking to study some file compression code. Looked thru the code examples and only saw some file splitters. I know Jeremy has some, but it's console code.

Thanks.

[Vortex]

File compression code doesn't depend on the user interface like console or GUI. Probably, the example was builded as a console application.

[P1]

skywalker,

It's "My way or the Highway".  My way, code your own, but expect others to code it for you.  Or the Highway, use the compression libraries available from open sources ( like http://www.7-zip.org/ ) or other coders.

Jeremy has moved to here:  http://www.bitsum.com/#jcalg1  And still has available his work with source.

Regards,  P1  :8)

[skywalker]

skywalker,

It's "My way or the Highway".  My way, code your own, but expect others to code it for you.  Or the Highway, use the compression libraries available from open sources ( like http://www.7-zip.org/ ) or other coders.

Jeremy has moved to here:  http://www.bitsum.com/#jcalg1  And still has available his work with source.

Regards,  P1  :8)

Thanks.

I was just there couple hours ago. It's 16 bit stuff.
What do you mean exactly by using the compression libraries?

[MichaelW]

I was just there couple hours ago. It's 16 bit stuff.

JCALG1

High performance compression library. Intended for extremely good compression ratio and very rapid decompression with low memory overhead. Used by PECompact (as pec2codec_jcalg1.dll, not default). Written in x86 32-bit assembly language, the full source code is included.

[donkey]

Zlib is a fairly easy one to use if you want ZIP type compression, I pretty much only use that one as it is easily portable and any utility can open the files it generates. But don't expect the greatest compression ratios, it is a general purpose compression algorithm and therefore it is fair to good in almost all circumstances. If you want great compression then choose an algorithm that best suits the data patterns that you expect to encounter, most authors will explain what data patterns a particular algorithm is best suited for.

If you would like to use ZLIB, you can download my WinExplorer test bed in the GoAsm forum, I have included the ZlibStat static library and include file as well as a small example routine showing how to read the contents of a zip file.

[BogdanOntanu]

I would recomend APlib by Jørgen Ibsen or JCALG by Jeremy Collake.

Both are very nice, effective an simple LZ77 implementations in modern 32bits ASM with source code (for the decompressor at least) and DLL's available.
MASM32 contans an sample file compressor/decompressor using ApLib AFAIK. (aPack)

ApLIb - http://www.ibsensoftware.com/products_aPLib.html
JCALG1 - http://www.bitsum.com/#jcalg1

Please note that Aplib site also contains an BriefLZ implementation.

Also try to read more carefully since JCALG1 is NOT 16 bit code ...

[Vortex]

Hi skywalker,

Using Jeremy Collake's JCALG code, I created a tiny SFX archive demo :

MASM version :

http://www.masmforum.com/simple/index.php?topic=1720.msg13326#msg13326

POASM version :

http://www.masmforum.com/simple/index.php?topic=3816.0

[skywalker]

I was just there couple hours ago. It's 16 bit stuff.

JCALG1

High performance compression library. Intended for extremely good compression ratio and very rapid decompression with low memory overhead. Used by PECompact (as pec2codec_jcalg1.dll, not default). Written in x86 32-bit assembly language, the full source code is included.

Sorry, big mistake on my part.
I just looked at the .asm files, the .model directives are in the .inc files.
I'll have to ask a lot of questions when I study it.

jccomp.inc

option casemap:none
.486p
.model flat,stdcall
............

[skywalker]

Zlib is a fairly easy one to use if you want ZIP type compression, I pretty much only use that one as it is easily portable and any utility can open the files it generates. But don't expect the greatest compression ratios, it is a general purpose compression algorithm and therefore it is fair to good in almost all circumstances. If you want great compression then choose an algorithm that best suits the data patterns that you expect to encounter, most authors will explain what data patterns a particular algorithm is best suited for.

If you would like to use ZLIB, you can download my WinExplorer test bed in the GoAsm forum, I have included the ZlibStat static library and include file as well as a small example routine showing how to read the contents of a zip file.

Thanks.
I will try it.
I just wanted to have the ability to zip up data files in my app instead of having to use a third party.

[P1]

I just wanted to have the ability to zip up data files in my app instead of having to use a third party.

Just for the record, when you don't make it, it's third party material.

Regards,  P1  :8)

[skywalker]

I just wanted to have the ability to zip up data files in my app instead of having to use a third party.

Just for the record, when you don't make it, it's third party material.

Regards,  P1  :8)

I think you know what I meant. You've seen your handle in my source code.

Take care.

[skywalker]

Zlib is a fairly easy one to use if you want ZIP type compression, I pretty much only use that one as it is easily portable and any utility can open the files it generates. But don't expect the greatest compression ratios, it is a general purpose compression algorithm and therefore it is fair to good in almost all circumstances. If you want great compression then choose an algorithm that best suits the data patterns that you expect to encounter, most authors will explain what data patterns a particular algorithm is best suited for.

If you would like to use ZLIB, you can download my WinExplorer test bed in the GoAsm forum, I have included the ZlibStat static library and include file as well as a small example routine showing how to read the contents of a zip file.

I looked over zlib and it looks like it's in C, maybe I'm wrong. Jeremy has the source for compressiing and decompressing already in asm. I would my app to be able to do both. The Jcalg2 example shows how to use his existing .exe to unzip a file. I would like to look at his code and get some help incorporating it.

[Mark Jones]

Hey guys, how can you use APlib to compress streams larger than the available memory? Say you wanted to compress a 1GB file using a 640kB buffer - how will the decompressor know where each "packet" begins and ends if the compressed packet length is variable? Perhaps the only way is to "double-buffer and packet-parse" (read each "AP32" signature block into a buffer and decompress)? Thanks for helping relieve a migraine headache. :bg

[BogdanOntanu]

The decompressor aways starts with a LITERAL byte and it will remain in syncro with the source stream ever since (state machine).

The end of stream is detected by a special TAG bits value and an null offset, if i recall corectly.

This way aPlib does not require any additional memory for decompression -- IMHO a very important feature. It will diectly decompress the source stream into the destination buffer.

Of course all will work corectly IF the source stream is not damaged.

For checking that the source stream is not damaged it make sense to add a small header to should contain the expected uncompressed size and a CRC/MD5 ... but this header is NOT required for the algorithm to work.