FPU compare real to integer problem

[RuiLoureiro]

JJ,
        i read the tutes and i didnt find any help to the  :green2
        problem

Dave,
        they are as large as real10 numbers

[dedndave]

oh - ok
well - the routine i linked will generate integer exponentials to full precision

[qWord]

RuiLoureiro,
you want to test whether the content of a FPU register is an integer value or not?

[RuiLoureiro]

Dave,
        Ok but it doesnt help me

qWord,
        you want to test whether the content of a FPU register is an integer value or not?

        no.

[jj2007]

Rui,

Post:
1. full code
2. your expected value
3. what the FPU produces against your expectations.
Everything else is a waste of time - your time and our time.
:thumbu

[RuiLoureiro]

jj,
        1. thank you for your answer  :thumbu

        2. Sorry, full code is too long !
                       
        3. I have one formula to compute X^Y if X and Y are both real10
        and i have
        another formula to compute X^Y if X is real10 and Y is integer32.

        But i dont want to code Y in integer32,
        this seems to be the problem. But i dont know if i must code it
        in integer32.


        I know how to code Y in integer32 but i dont want to do that
        if i can solve the problem in a different way.
         
        If Y is in Real10, i am thinking that when i truncate Y to
        integer32 i get Yi (Yi is an integer32).

                        Yi = Trunc (Y)

        Now:
                               
        mov     eax, offset Y       ; Y  is 4 in real10
        mov     ebx, offset Yi      ; Yi is 4 in integer32
       
        fld     tbyte ptr [eax]
        fild    dword ptr [ebx]
        fcomip  st,st(1)
        jne     _isnotequal
        je      _isequal            ; why not ?
       
        It seems to be clear, no ? I get _isnotequal.         
       
        Did you understand the problem ?

        4. The question is: Can i solve the problem this way
           or i need to code Y in integer32 to use the 2nd formula ?

        Thanks   

[qWord]

... so you want to determinate if y is an integer or not and then select the corresponding algorithm  ::)

[RuiLoureiro]

... so you want to determinate if y is an integer or not and then select the corresponding algorithm  ::)

            Yes

[dedndave]

1) use FINIT once at the beginning of the program - it sets up the FPU for 80-bit precision
when you start a program under windows, it is set up for 64-bit precision

2) use REAL10 instead of DT or TBYTE PTR

3) this code leaves one of the FPU registers loaded - make sure you FFREE it   :P

Code Select Expand

        .XCREF
        .NOLIST
        INCLUDE \masm32\include\masm32rt.inc
        .686
        .LIST

;#################################################################################

        .DATA

Y       real10 4.0
Yi      dd     4

;*********************************************************************************

;        .DATA?

;#################################################################################

        .CODE

;*********************************************************************************

_main   PROC

        finit

        fld real10 ptr Y
        fild dword ptr Yi
        fcomip  st,st(1)
        jz      equal

not_equal:
        print   chr$('Not Equal'),13,10
        jmp short exit00

equal:
        print   chr$('Equal'),13,10

exit00:
        inkey
        exit

_main   ENDP


;#################################################################################

        END     _main

result:

Code Select Expand

Equal
Press any key to continue ...

[qWord]

... so you want to determinate if y is an integer or not and then select the corresponding algorithm  ::)

            Yes

see what I've found in my FPU box:

Code Select Expand

fld ...
sub esp,12
fst[p] REAl10 ptr [esp]
movzx eax,WORD ptr [esp+8]
and eax,7FFFh
mov ecx,eax
xor eax,7FFFh
.if !ZERO? ; number is not NaN,INFINTE,...
mov eax,DWORD ptr [esp]
or eax,DWORD ptr [esp+4]
or eax,ecx
setz al
.if !ZERO? ; value != +-0.0
xor eax,eax
sub ecx,3fffh
.if !CARRY? ; negativ exponent => not an integer
test ecx,0ffffffc0h
setnz al
.if ZERO? ; (exponent >= 64) ? integer : continue check
; clear out all integer bits and test for zero.
; (could be replaced by common x86 code...)
lea ecx,[ecx+1]
movq xmm0,QWORD ptr [esp]
movd xmm1,ecx
pcmpeqb xmm2,xmm2
pxor xmm3,xmm3
psrlq xmm2,xmm1
pand xmm0,xmm2
pcmpeqd xmm0,xmm3
pmovmskb eax,xmm0
movzx eax,al
xor eax,0ffh
setz al
.endif
.endif
.endif
.endif
add esp,12
; eax = 0/1
; eax=0: floating point value , eax != 0: integer
I've not used this offten, but AFAIKS it works well.

[jj2007]

The problem with fcomip is that the two numbers need to be really, really equal. And not every integer can be represented exactly in REAL10 format. To illustrate this, here an example where the 19th digit is a tick off:

include \masm32\MasmBasic\MasmBasic.inc   ; download
.data
Y   REAL10 4.000000000000000001
Yi   DWORD 4

   Init
   mov eax, offset Y   ; Y is 4 in real10
   mov ebx, offset Yi   ; Yi is 4 in integer32
   fld REAL10 ptr [eax]
   fild dword ptr [ebx]
   deb 4, "FPU", ST(0), ST(1)

   Print "fcomip, REAL10:   ", Tb$
   fcomip st,st(1)
   .if Zero?
      PrintLine "equal"
   .else
      PrintLine "not equal"
   .endif

   mov ecx, Yi

   Print "Fcmp, top precision:", Tb$
   Fcmp Y, ecx, top
   .if Zero?
      PrintLine "equal"
   .else
      PrintLine "not equal"
   .endif

   Print "Fcmp, high precision:", Tb$
   Fcmp Y, ecx, high
   .if Zero?
      PrintLine "equal"
   .else
      PrintLine "not equal"
   .endif

   Inkey "ok"
   Exit
end start

Code Select Expand

FPU
ST(0)           4.00000000000000000
ST(1)           4.00000000000000000
fcomip, REAL10:         not equal
Fcmp, top precision:    not equal
Fcmp, high precision:   equal

The debug macro "sees" the same number, but fcomip and the Fcmp macro in top precision mode see it as different. Fcmp in "high" precision, i.e. roughly REAL8, treats them as equal.

Now the obvious question is "what for"? Which precision does your app need? The "not equal" of fcomip would be misleading in most instances.
The question would be irrelevant if 4.0 came directly from memory - you declare Real10 4.0, and it works. However, if 4.0 are the result of calculations, then they are most likely a tick off, and you will almost never see "equal". Especially if the app does graphics, with a resolution of e.g. 2000 pixels, then you would need a much lower precision in order to see an equal where you want to see an equal.

P.S.: When testing this, I noticed a little bug in Fcmp - it would load a DWORD global variable as REAL4. The bug is fixed, and the example above works also with the old version because I used mov ecx, Yi to circumvent this problem.

[qWord]

This precision problem can be solved by using a threshold-factor:
pseudo code:

Code Select Expand

if abs(a-b) <= abs(a*factor) then equal
(requires: a != 0)

The SmplMath-macro faEQ(a,b,f) supports this method:

include \masm32\include\masm32rt.inc
.686
.mmx
.xmm
include \macros\smplmath\math.inc
.code
main proc
LOCAL fSlvTLS()   
   
   print "exact compare: ",13,10
   .if fEQ(1.01,1.02)
      print "    fEQ( 1.01, 1.02):           equal      (exact)",13,10
   .else
      print "    fEQ( 1.01, 1.02):           not equal  (exact)",13,10
   .endif

   print chr$(13,10)
   print "compare using a threshold factor: ",13,10

   .if faEQ(1.01,1.02)
      print "    faEQ( 1.01, 1.02, 1.0E-2):  equal      (+-1%) ",13,10
   .else
      print "    faEQ( 1.01, 1.02, 1.0E-2):  not equal  (+-1%)",13,10
   .endif

   .if faEQ(1.01,1.02,1.0E-3)
      print "    faEQ( 1.01, 1.02, 1.0E-3):  equal      (+-1 per mille) ",13,10
   .else
      print "    faEQ( 1.01, 1.02, 1.0E-3):  not equal  (+-1 per mille)",13,10
   .endif   
   
   print chr$(13,10)
   inkey
   exit
   
main endp
end main

Code Select Expand

exact compare:
    fEQ( 1.01, 1.02):           not equal  (exact)

compare using a threshold factor:
    faEQ( 1.01, 1.02, 1.0E-2):  equal      (+-1%)
    faEQ( 1.01, 1.02, 1.0E-3):  not equal  (+-1 per mille

[raymond]

another formula to compute X^Y if X is real10 and Y is integer32.

I will have to assume at this point that what you mean by integer32 is a 32-bit integer, i.e. a dword.

ALL dwords are represented exactly in REAL8 and REAL10 format.
In REAL8 format, the least significant 20 bits would ALWAYS be 0, while in the REAL10 format, the least significant 32 bits would ALWAYS be 0.

One way to check if a REAL10 number in memory is effectively a 32-bit signed integer could be done as follows:

Code Select Expand

.data
myREAL10 REAL10 ...
.code
mov eax,dword ptr myREAL10   ;get the lower 32 bits
test eax,eax
jnz not_a_dword_integer

mov eax,dword ptr myREAL10[32]   ;get the first 32 bits of the significand field
bsr ecx,eax
mov eax,31
sub eax,ecx      ;number of active bits
add eax,3FFFh   ;minimum biased exponent for those bits to be an integer

mov edx,dword ptr myREAL10[48]   ;get the upper 32 bits
shl edx,1      ;get rid of the sign bit
shr edx,17     ;retain only the biased exponent
cmp edx,4020h
jae not_a_dword_integer   ;the value of myREAL10 would be larger than 2^32
cmp edx,eax
jb not_a_dword_integer

;is a dword integer
...

not_a_dword_integer:
...

[raymond]

        3. I have one formula to compute X^Y if X and Y are both real10
        and i have
        another formula to compute X^Y if X is real10 and Y is integer32.

I don't know what "formulas" you are using to compute the powers of REAL numbers but, in my opinion, the simplest way is to use the FPU where it doesn't matter at all whether the "Y" is a float or an integer. Maybe you can enlighten us on those different formulas.

[vanjast]

Maybe you can compare the real10 numbers outside the FPU...

Num1 XOR Num2

Should give you and idea if it's at least equal (zero result).. if not then maybe it's the compiler/assembler not assigning equal values
:8)