[MLton] semantics of PackReal

Matthew Fluet fluet@cs.cornell.edu
Tue, 25 Apr 2006 12:27:40 -0400 (EDT)


I don't believe that PackReal<N>{Big,Little} follow the Basis Library 
spec correctly.  The following program:

val v =
    Word8Vector.fromList
    [0wx0D,0wxE5,0wx35,0wx94,0wxD7,0wx00,0wx82,0wx40,
     0wx0D,0wxE5,0wx35,0wx94,0wxD7,0wx00,0wx82,0wx40]

val r = PackReal64Little.subVec(v, 0)

val () = print (concat [Real64.toString r, "\n"])

val r = PackReal64Little.subVec(v, 1)

val () = print (concat [Real64.toString r, "\n"])

val r = PackReal64Little.subVec(v, 2)

val () = print (concat [Real64.toString r, "\n"])

Behaves as follows:

[fluet@localhost temp]$ ./z
576.105263158
7.55515535919E~245
~5.92683407001E178

The Basis Library specifies:

subVec (seq, i)
subArr (seq, i)
     These functions extract the subsequence

        seq[bytesPerElem*i..bytesPerElem*(i+1)-1]

     of the aggregate seq and convert it into a real value according to
     the endianness of the structure. They raise the Subscript exception
     if i < 0 or if Word8Array.length seq < bytesPerElem * (i + 1).

So, it seems to me that the correct behavior is:

[fluet@localhost temp]$ ./z
576.105263158
576.105263158
unhandled exception: Subscript