Compression

[Scalemail Ted]

YES!!!!!!!!!

I have compressed a file!!!!!

Now to build a decompressor and check it.   :-p


One issue I do have is that my read_file proc does not read large files.  I tried to read a file that was 8kb in size and it reads about half of it and then crashes.
I don't understand why it crashes.  It works fine for smaller files, why not larger? Im reading it one byte at a time, so whats the difference, is there a carry flag that I'm triggering at a certain point?   :-/


I solved it!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!



Again, thank you for any input you can provide!!!!!!!

[Scalemail Ted]

OH NOOOO!!!!!!!

After fixing the file reading problem... I realized the compressor wasn't reading all the file before... now come to find out  my compressor is expanding my data.

:-(

I need to find a good cut off for huffmans to stop "compressing" and just use ascii again... maybe the 16th element down the frequency list???

[FORTRANS]

Hi,

   That is unlikely to work very well.  The overhead of switching
between Huffman codes and plain binary will hurt.  What kind
of data are you trying to compress?  For debugging your routine
you should start with either hex data or simple text to limit the
number of symbols to encode.  I used hex data because I could
calculate the Huffman codes by hand and determine the
compression ratio and so forth.

Regards,

Steve N.

[Scalemail Ted]

I'm primarily building it for text.  I modified my seeding and now i am compressing.  Its not pretty, about 10-20%.  hahaha.

but not bad for a first...umm second attempt.  I'll strive to improve it.



EDIT:

AND I'm done... now i just have to figure out why it doesnt decompress.

[Scalemail Ted]

UPDATE:



I just finished building an operational  Huffman based compressor/decompressor.

It  works well on text files.

What would need to be done for it to also on executables

It allows me to compress exes, but masm crashes when I attempt to uncompress it.

[hutch--]

Ted,

Without looking at your code, if it crashes through the decompression stage it is almost exclusively a memory read or write error with one or both going out of the memory allocation range. Step through your code bit by bit and check that each step is not reading or writing outside of your allocated memory range.

[PBrennick]

Ted,

Are you realizing that exes contain NULLS?

Paul

[Scalemail Ted]

Ted,

Without looking at your code, if it crashes through the decompression stage it is almost exclusively a memory read or write error with one or both going out of the memory allocation range. Step through your code bit by bit and check that each step is not reading or writing outside of your allocated memory range.

It only crashes on the decompression stage with executables... text files work fine... ive tried up to 200k with text files with success whereas with the executable a tiny 23k crashes the program

edit: after you mentioned this i tried a larger text file at 1.5MB... sure enough when I try to decompress it... it fails after decompressing 667kb... its readable but cut short.

Ted,

Are you realizing that exes contain NULLS?

Paul

No, I didn't know this...

Someone else was telling me that executables can  use characters that are 0 bytes...  I might have to reorder some of my methods for determining end of files.

[Scalemail Ted]

UPDATE:

Handled the null characters... my compressor now can compress and decompress executables...

still need to resolve the other problem... it crashes after decompressing only 667kb of a 5mb file