Entropy calculator code not working

[Ghirai]

Hey, i've attached a zip with an app i wrote some time ago, to calulate the entropy of a file.
The algo is pretty simple, but i just can't get it to work.

Here's the algo.

Given the following stream (ascii bytes):

'abcdcdaad'

we count the number of repititions of each byte

a = 3
b = 1
c = 2
d = 3
------
9 total

so we can say that the frequency (provavility) of apparence of each char is
the number of times it appears divided by the total length
of the stream

a = 3/9 = 0.33...
b = 1/9 = 0.11...
c = 2/9 = 0.22...
d = 3/9 = 0.33...

Now we have that the entrophy for each symbol is defined by the formula:

-log base 2 (frequency or provability)

so we apply the formula to each symbol

for A and D

-log base 2(3/9) = 1,5849625007211561814537389439478

for B

-log base 2(1/9) = 3,1699250014423123629074778878956

for C

-log base 2(2/9) = 2,1699250014423123629074778878956

now we compute the sum of all the entropies

1,5849625007211561814537389439478 A
1,5849625007211561814537389439478 A
1,5849625007211561814537389439478 A
3,1699250014423123629074778878956 B
2,1699250014423123629074778878956 C
2,1699250014423123629074778878956 C
1,5849625007211561814537389439478 D
1,5849625007211561814537389439478 D
1,5849625007211561814537389439478 D
----------------------------------------------
17,019550008653874177444867327367 -> result.

What i do in the code is fill and array with how many times each byte appears (from 0 to FF).
Then i divide each member of the array by the filesize.
Then i follow the algo above.

But it doesn't seem to work, it returns 0 :(

Andy help appreciated.

[attachment deleted by admin]

[Mark Jones]

Hey Ghirai. I loaded it and tried to compile and got "EC.asm(196) : error A2070: invalid instruction operands" for "fst tbyte ptr[ecx+esi]". Still checking. Replaced with "fst dword ptr[ecx+esi]", no error but... :toothy

Tricky, can't interactively debug it using Olly, keeps crashing in module "RPCRT4."

[MichaelW]

An fstp tbyte ptr[ecx+esi] will assemble OK, but working out of QE the resulting exe will not run on my system.

[raymond]

As Mark mentioned, fst tbyte ptr[ecx+esi] is an invalid instruction which should have been detected by your assembler. That is probably a misprint on your part.

2 comments:
1) Dividing 0 by a non-zero number is perfectly legal and could simplify your code by not skipping it. It would also guarantee that whatever is stored in memory for the result would also be perfectly correct.
2) You are multiplying and adding only the first 255 items of the array instead of the entire 256. This would lead to an erroneous result.

The only reason I see for getting 0 as the final result would be that an invalid operation was performed at some point on the FPU and a NAN stored in your array. This would then be carried over for the last addition. The conversion of such a NAN by the FpuFLtoA function would return an empty string. I noticed that you don't check for the return of the ERROR code after calling the FpuFLtoA function.

You will need to include some error checking code in the source file if you want to locate the source of the error.

Raymond

[Ghirai]

Alright, thanks everyone.

I'll look at it, and get back to you guys if there are still problems.

[Ghirai]

I checked the return values of FpuFLtoA, and it's 1.
And yes, fst tbyte ptr[ecx+esi] should be fstp.

I'm rewriting the whole thing again, this time as a console app, to make things easier.

Hopefully i'll get it to work :P