Problem when using FPU

[Airam]

I get a problem with this algorithm:

.const
MAX_ELEMENTS   equ 50000

.Data?
OxAxis         Real10      MAX_ELEMENTS Dup(?)

.code

calculateOxAxis Proc Uses Esi Ecx
   ; Stack : | step | -
        ; subIntervals = 1000
        ; step = 0.01
;-------------------------------------------------------------------
   Xor Esi, Esi                  ; Esi = 0
   Xor Ecx, Ecx                  ; Ecx = 0
   Mov Cx, subIntervals
   Lea Esi, OxAxis
   Fldz                        ; 0 | step | -
   Fstp Real10 Ptr [Esi]
   Fld Real10 Ptr [Esi]
   Add Esi, 10
   .Repeat
      Fadd St(0), St(1)            ; amount + step | step | -
      Fstp Real10 Ptr [Esi]                   ; OxAxis = amount  <---> | step | -
      Fld Real10 Ptr [Esi]                 ; amount | step | -
      Add Esi, 10                  ; Esi = Esi + 10
   .UntilCxZ
   Ret
;-------------------------------------------------------------------
calculateOxAxis EndP


The output that I get is:

0.01
0.02
0.03
0.040000000000000000004336808689944017736029811203479766845703125000000000000000000000000000000000000000000
0.050000000000000000004336808689944017736029811203479766845703125000000000000000000000000000000000000000000
.
.
.


What I want is a 0.04 instead of 0.040000000000000000004336808689944017736029811203479766845703125000000000000000000000000000000000000000000

I don't know where is the error. Can somebody help me?
Thanks for your help.

[jj2007]

First, it is always a good idea to see yourself what happens in your code - OllyDebug is the right tool.

Second, you should post complete code, so that we can run it, too. It seems your code has an output problem, maybe because the buffer is too short; but it is impossible to know where the bug is if the output part (printf??) is missing.

[MichaelW]

Airam,

How are you producing the 100+ decimal digit strings? For a REAL10 any digits beyond ~19 are not meaningful.

I think the problem is that you are working with numbers that cannot be represented exactly in the IEEE real number format, and expecting exact results. Of all the possible real numbers, only a very small portion can be represented exactly. For those that cannot, the best that the FPU can manage is a more or less close approximation.

For example, using an older Borland C compiler for which the printf function supports the long double (REAL10) type, displaying the value 0.04 rounded to 19 decimal digits I get:

0.0400000000000000008

If I drop the precision back to 17 decimal digits, I get:

0.04000000000000000

As another example, here are the three representable values centered on the value 12.12, each rounded to 15 decimal digits:

12.119999999999997
12.119999999999999
12.120000000000001

[Sarel]

I am suprized that you do get output. Look at the comments.

   Xor Esi, Esi                  ; Esi = 0
   Xor Ecx, Ecx               ; Ecx = 0
   Mov Ecx, subIntervals   ;its 32 bit code, why not.
   Lea Esi, OxAxis
   Fldz                              ;st(0) = 0 and st(1) is undefined. Where did you put 0.01 in it.
   Fstp Real10 Ptr [Esi]       ;The first value in the array gets a zero.
                                      ;st(0) = undefined. Because you poped the stack and st(1) was not initialized.
   Fld Real10 Ptr [Esi]         ;st(0) = zero again.
   Add Esi, 10                   ;Points to the next value in the array.
   .Repeat                        ; I personaly dont like these, but it reads better.
      Fadd St(0), St(1)        ;You add an undefined value, st(1), to zero ; amount + step | step | -
      Fstp Real10 Ptr [Esi]    ;Store the undefined answer and pop the stack.
                                      ;st(0) is undefined again as well as st(1)
      Fld Real10 Ptr [Esi]     ;            ; amount | step | -
      Add Esi, 10              ;    ; Esi = Esi + 10
   .UntilCxZ                   ;Nowhere do you subtract Your (E)cx register to change the flags.
                               ;sub Ecx,1
                               ;.Until Ecx = 0
   Ret

Enjoy fixing it
As for your print function, I use what comes with masm32. FpuFLtoA and FpuAtoFL

[Airam]

Thanks MichaelW, jj2007 and Sarel for your answers.

Sarel: I commited and error when i posted my question, because i forgot to say that the step was already at the fpu stack.
jj2007: The code was very big, so I decided not to include all here, but I know that it was real important for you to help me.
MichaelW: I actually was producing just two meningful decimals but i got problems with them.

The problem was in the printing routine, the one i didn't put here. Sorry.

Thank you very much for answering my question.