FPU status word

[jj2007]

So can we agree that if you load two whatever values with fld (or fild), the FPU holds them as REAL10, and if you save them to two locations in memory as REAL10, they can be compared correctly, showing eventually that 12345.6789 is not equal if one is REAl4 and the other is REAL8...?

[raymond]

The FPU loads and holds them in REAL10 format, but not necessarily in REAL10 precision. This may seem as playing on words but it is an important difference.

I do agree that if 12345.6789 was stored as a REAL4 value in memory (regardless of the precision at which it was generated) that it would be different from a 12345.6789 value generated as a REAL8 (or REAL10) before being stored in memory as a REAL8 (or REAL10).

If you load both of them, you can see with Ollydbg that they are different. And, if you now save both of them as REAL10 and look at the memory locations where they are stored (or print them with 15 significant digits), you would observe that they are slightly different.

HOWEVER, if you load the REAL4 value of 12345.6789 and store it as a REAL8 or REAL10 (thus not having been generated in such precision), it should not be any different than the REAL4 value when reloaded onto the FPU in REAL10 format.

Although the difference would be considerably smaller, values generated in REAL 10 precision but stored as REAL8 and REAL10 would be different when reloaded to the FPU. The REAL8 would have lost 8 bits of precision (out of 64) and rounded up or down based on the most significant bit lost.

[MichaelW]

Although the difference would be considerably smaller, values generated in REAL 10 precision but stored as REAL8 and REAL10 would be different when reloaded to the FPU. The REAL8 would have lost 8 bits of precision (out of 64) and rounded up or down based on the most significant bit lost.

Isn't that 64 - 53 bits of precision lost?

[raymond]

My bad. :red :eek :red You loose 10 bits of precision (out of 64).
It would not be 11 (64-53) because one bit is implied in the REAL4 and REAL8 formats.

[jj2007]

HOWEVER, if you load the REAL4 value of 12345.6789 and store it as a REAL8 or REAL10 (thus not having been generated in such precision), it should not be any different than the REAL4 value when reloaded onto the FPU in REAL10 format.

Raymond,
Thank you for your efforts to bring clarity in this tricky business. What I am claiming is that the FPU does not change what it gets when loading and storing a value, regardless of the precision set by the FPU control word. Below is a practical example - source attached, and sorry that it needs the MasmBasic version of today (I added the SetFpu macro).

Of course, if you perform operations (fadd, fmul, ...) with the loaded values, the result would depend on the precision. But that is not the case for the comparison algo.

Code Select Expand

Digits: 1234.567890123456789
MyR4=   1000.000000000000000
MyR8=   1000.00000000000011
MyR10=  999.999999999999999

64 bit precision, Fcmp 'top' :
MyR4 is lower than MyR8
MyR4 is higher than MyR10

53 bit precision:
MyR4 is lower than MyR8
MyR4 is higher than MyR10

24 bit precision:
MyR4 is lower than MyR8
MyR4 is higher than MyR10

64 bit precision, Fcmp 'high' :
MyR4 and MyR8 are equal
MyR4 and MyR10 are equal

53 bit precision:
MyR4 and MyR8 are equal
MyR4 and MyR10 are equal

24 bit precision:
MyR4 and MyR8 are equal
MyR4 and MyR10 are equal