[MLton-devel] Real.fromLargeInt

[Matthew Fluet]

Real.fromLargeInt's pos function doesn't seem to be affected by the
current rounding mode.  Furthermore, ~ (pos (IntInf.~ i)) won't always
give the correct result for some rounding modes. 

Real.toLargeInt doesn't seem to work for "large" real numbers.

Below is a quick test program I wrote.  check i n mode uses i and n to
construct a large, positive IntInf.int then does Real.fromLargeInt and
Real.toLargeInt on the IntInf.int and it's negation.  I get the following
results: 

check   2       17      TO_NEAREST
1010894056998587542
1010894056998587520.0
0
~1010894056998587542
~1010894056998587520.0
0

check   2       17      TO_NEGINF
1010894056998587542
1010894056998587520.0
0
~1010894056998587542
~1010894056998587520.0
0

check   2       17      TO_POSINF
1010894056998587542
1010894056998587520.0
0
~1010894056998587542
~1010894056998587520.0
0

check   2       17      TO_ZERO
1010894056998587542
1010894056998587520.0
0
~1010894056998587542
~1010894056998587520.0
0

TO_NEGINF and TO_POSINF give incorrect Real.fromLargeInt results for the
negative and positive IntInf.int's respectively.  TO_ZERO is o.k. from a
rounding point of view, and TO_NEAREST might be.  Clearly all the
Real.toLargeInt results are incorrect.

**********

fun mkInt i n = if n = 0
		  then i
		  else mkInt (IntInf.+ (IntInf.* (i, IntInf.fromInt 10),i)) 
                             (n - 1)

val mkInt = fn i => fn n => mkInt (IntInf.fromInt i) n

fun check i n mode = 
  let
    val _ = IEEEReal.setRoundingMode mode
    val _ = print "check\t"
    val _ = print (Int.toString i)
    val _ = print "\t"
    val _ = print (Int.toString n)
    val _ = print "\t"
    val _ = print (case mode of
		     IEEEReal.TO_NEAREST => "TO_NEAREST"
		   | IEEEReal.TO_NEGINF => "TO_NEGINF"
		   | IEEEReal.TO_POSINF => "TO_POSINF"
		   | IEEEReal.TO_ZERO => "TO_ZERO")
    val _ = print "\n"
    fun doit i =
      let
	val r = Real.fromLargeInt i
	val i' = Real.toLargeInt mode r
      in
	print (IntInf.toString i);
	print "\n";
	print (Real.fmt (StringCvt.FIX (SOME 1)) r);
	print "\n";
	print (IntInf.toString i');
	print "\n"
      end
    val i = mkInt i n
  in
    doit i;
    doit (IntInf.~ i);
    print "\n"
  end

val _ = check 2 17 IEEEReal.TO_NEAREST
val _ = check 2 17 IEEEReal.TO_NEGINF
val _ = check 2 17 IEEEReal.TO_POSINF
val _ = check 2 17 IEEEReal.TO_ZERO



-------------------------------------------------------
This sf.net email is sponsored by: Influence the future 
of Java(TM) technology. Join the Java Community 
Process(SM) (JCP(SM)) program now. 
http://ads.sourceforge.net/cgi-bin/redirect.pl?sunm0002en

_______________________________________________
MLton-devel mailing list
MLton-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mlton-devel