floating point

Matthew Fluet mfluet@intertrust.com
Mon, 13 Aug 2001 13:54:08 -0700 (PDT)


> I don't understand.  I just computed the square root of negative infinite
> in MLton and I did not get a floating point exception (thank god).  Do
> you explicitly test for negatives and return a NaN by hand in those cases?

I (i.e., the x86-codegen) does not.  Nor does the basis.
I gdb'ed the program that computes Math.sqrt(~1.0) and stepped through it;
it executed fsqrt with -1.0 at the top of the floating-point stack, and
that replaced it with QNaN.

Looking quickly at Vol1 of IA Soft. Dev. Manual, what's really going on is
that the floating point exceptions are being masked (i.e., slilently
handled by the floating-point unit.)  In the case of sqrt, it returns the
"real indefinite value to the destination operand."