doubles and cos(pi / 2)

Stephen Weeks MLton@sourcelight.com
Wed, 4 Oct 2000 13:22:26 -0700 (PDT)


I am confused about the correct value of cos(pi / 2) with IEEE compliant
doubles.  Below, I have the results of several compilers.  All agree that the
value is 6.123e-17, except for SML/NJ which reports zero.  The reason I bring
this up is that one of the ICFP functional programming contest tests
(features.gml) seems to have relied on cos(pi / 2.0) = 0.  I don't understand
how the OCAML entries got this test right, and I wonder if we were eliminated
because of it.  I am thinking of sending a query to the organizers.


compiler	value
--------------	----------------------
C		0.00000000000000006123
emacs lisp	0.00000000000000006123
ML Kit		0.00000000000000006123	
MLton           0.00000000000000006123
mosml		0.00000000000000006123
SML/NJ		0.00000000000000000000
OCAML		6.12303176911e-17

C:
	fprintf(stderr, "%.20f", cos(M_PI / 2))

OCAML:
	let x = cos(3.14159265358979323846 /. 2.0);;

emacs lisp:
	(format "%.20f" (cos (/ pi 2)))

SML:
	let open Real open Math 
        in fmt (StringCvt.FIX(SOME 20)) (cos(pi / 2.0)))
        end