The MASM Forum Archive 2004 to 2012

Are large floating point numbers ( beyond what the fpu can handle ) represented the same way they are in HLLs ( an array of bytes with a byte for each digit in the large floating point number ) ? Is there a different strategy used in assembly language? Thanks for any input. :U

QuoteAre large floating point numbers ( beyond what the fpu can handle )

All depends on your interpretation of large floating point numbers. A REAL10 gives you a range up to approximately 10⁴⁹³² which is quite a large number. However, the precision of even a REAL10 cannot exceed approximately 19 significant digits. So, if you have a number such as:
12345678901234567890123456789012345678901234567890
it is well within the range of a REAL10 (and even of a REAL8) but it could be rounded to something like
1.23456789012345679E+49 in scientific notation with the last 30 or so digits being lost.

If the number is really beyond what the FPU can handle (up to 10³⁸ for REAL4, up to 10³⁰⁸ for REAL8), the number simply becomes the representation for INFINITY.

If you are interested in learning more about the floating point format, you may start by having a look at the following:
http://www.ray.masmcode.com/tutorial/fpuchap2.htm#floats

If you are talking about BIGNUM libraries available with some HLLs, that is a totally different subject not readily available in assembly.

Raymond