[MLton] semantics of PackReal
Matthew Fluet
fluet@cs.cornell.edu
Tue, 25 Apr 2006 12:27:40 -0400 (EDT)
I don't believe that PackReal<N>{Big,Little} follow the Basis Library
spec correctly. The following program:
val v =
Word8Vector.fromList
[0wx0D,0wxE5,0wx35,0wx94,0wxD7,0wx00,0wx82,0wx40,
0wx0D,0wxE5,0wx35,0wx94,0wxD7,0wx00,0wx82,0wx40]
val r = PackReal64Little.subVec(v, 0)
val () = print (concat [Real64.toString r, "\n"])
val r = PackReal64Little.subVec(v, 1)
val () = print (concat [Real64.toString r, "\n"])
val r = PackReal64Little.subVec(v, 2)
val () = print (concat [Real64.toString r, "\n"])
Behaves as follows:
[fluet@localhost temp]$ ./z
576.105263158
7.55515535919E~245
~5.92683407001E178
The Basis Library specifies:
subVec (seq, i)
subArr (seq, i)
These functions extract the subsequence
seq[bytesPerElem*i..bytesPerElem*(i+1)-1]
of the aggregate seq and convert it into a real value according to
the endianness of the structure. They raise the Subscript exception
if i < 0 or if Word8Array.length seq < bytesPerElem * (i + 1).
So, it seems to me that the correct behavior is:
[fluet@localhost temp]$ ./z
576.105263158
576.105263158
unhandled exception: Subscript