fpu which is more accurate

[petezl]

Hi all,
I have been puzzling over some fpu results
A calculation in excel: TanAlpha3 =TAN((E18)*PI()/180) == TAN((5)*PI()/180)
gives a result of 0.087489
open office org in linux gives 0.0874887
My calculator gives 5/180*PI gives 0.08726646261
using the fpu it's off the mark using real8
Trying Raymonds fpu commands with real10 I think is nearer! I'm getting 0.087266
Anyone know which is nearest the right figure?
I thought MS Excel worked to 12 dec places!

Peter.

[Eóin]

Microsoft PowerToy Calculator gives the answer as the following at max (512 bits) precision :wink
0.08748866352592400522201866943496145811945427636810822914523666222164142730261284161912373237006409752957505295565064132179197504723395097656772638240550816088681786476042856018615629997953994368412245211577252943326808245192301548223894701882536095207796938321998029495832208927055680552031107205670505244931359063087712328494867773906492311730999216684243619517797090934381289836682582088916830103854547159997968673583678069066615671169979767404922158399693800931489868900309047057012896546519599197946487196162

The best precision easily achievable in a program is to store the above number as a real10. Performing the calculation in the FPU will get you real10 level precision :U .

P.S. Excel quite possibly is working to 12 places internally but just displayed a rounded answer, try changing the cell format.

Additional P.S. When doing the calculation in FPU make sure to use fldpi when loading pi to ensure you have it at its highest precision.

[raymond]

A calculation in excel: TanAlpha3 =TAN((E18)*PI()/180) == TAN((5)*PI()/180)
gives a result of 0.087489
open office org in linux gives 0.0874887

Those are correct. You are computing the tangent of the 5 degree angle.

My calculator gives 5/180*PI gives 0.08726646261

That is also a correct result. You are simply converting an angle of 5 degrees into radians.

Anyone know which is nearest the right figure?

Which one do you want to know as the nearest the right figure: the angle in radians or its tangent?

For your information, the tangent of small angles is quite close to the angle expressed in radians. This would also apply to the sine of small angles.

N.B. MS Excel most probably works with extended double precision (i.e. REAL10) which is equivalent to approx. 19 significant digits.

Raymond

[petezl]

Thanks for the replies. I think I see now where I'm going wrong!
Math doesn't come naturally to me, Usually succeed but only by perserverence! I remember now that as you say small angles return quite small differences whether in rads or degs.
Raymond,
The result I'm after is the excel result using the fpu!

ps. I'm used to using the fpu stack directly so thought I'd try your fpu commands and have hit a glitch. will pm you some code when I get back home in a few days..

Peter.

[MichaelW]

Per the documentation Excel 97 stores numbers with 15-digit precision (double or REAL8), and I doubt that this has changed for the later versions.

[Mark Jones]

I believe the typical float is realized as exponent + (sign bit) + mantissa format. Slightly different order for different platforms but the concept remains the same. If the mantissa is 8 bits long, it produces a float with much more granularity (error) than a mantissa of 48 bits. So two floats with varying number of mantissa bits will each produce a similar value but with differing error.

Exponent + mantissa format is a very efficient method of representing a huge magnitude of numbers with very few bits, but the bigger the magnitude the larger the error. Because of the error, it might be beneficial to round the float. The granularity at extreme magnitudes (like 1.123E+4000) is quite large so rounding those might be pointless. If you need thousandths' accuracy at 1E+/-4000, float is not going to work for you... :) but the general idea is, the more bits you can supply to the mantissa portion of the calculation, the more accurate the result.