The MASM Forum Archive 2004 to 2012

General Forums => The Campus => Topic started by: Jimg on November 16, 2007, 04:59:38 PM

Title: FPU truncate
Post by: Jimg on November 16, 2007, 04:59:38 PM
What is the proper way to truncate a value in an FPU register to an integer.  Not round but truncate.

Title: Re: FPU truncate
Post by: GregL on November 16, 2007, 05:44:56 PM
The example I posted here uses FISTTP if you have SSE3, CVTTSD2SI if you have SSE2, or the standard x87 way.

http://www.masm32.com/board/index.php?topic=7891.msg57858#msg57858

Title: Re: FPU truncate
Post by: Jimg on November 16, 2007, 08:03:00 PM
Thanks Greg.  That wasn't that long ago, I don't know why I didn't remember your post.

In the meantime, since I already had the value in the FPU, and I knew it was positive, I found I could just subtract .5 and let the FPU round like normal to get the result I wanted.  A lot easier and more efficient, in my case, than anything else I could think of.
Title: Re: FPU truncate
Post by: GregL on November 16, 2007, 08:47:55 PM
Jimq,

Just remember the normal rounding mode is "round to the nearest even integer".

Here's another method.


TruncateR8 PROC pR8:PTR REAL8, pInt32:PTR SDWORD
    .CONST
        dblMinusOneHalf REAL8 -0.5
    .DATA
        i SDWORD 0
    .CODE
        mov  edx, pR8
        finit
        fld  REAL8 PTR [edx]
        fadd st(0), st(0)
        fabs
        fld  dblMinusOneHalf
        fadd
        fistp i
        sar i, 1
        mov eax, pR8
        mov edx, [eax+4]
        bt  edx, 31
        jnc positive
        neg i
      positive:
        mov edx, i
        mov eax, pInt32
        mov [eax], edx
        ret
TruncateR8 ENDP


I got the idea for this from here (http://ldesoras.free.fr/doc/articles/rounding_en.pdf).

Title: Re: FPU truncate
Post by: Jimg on November 16, 2007, 09:12:56 PM
QuoteJust remember the normal rounding mode is "round to the nearest even integer".
That's only if it's exactly something.5, right?  I'll have to see if that's could be a problem for me.  Thanks.
Title: Re: FPU truncate
Post by: GregL on November 16, 2007, 09:17:30 PM
Jimq,

Yeah, only if it's something.5.