[MLton] Re: [MLton-user] FFI and pointer relocation

[Wesley W. Terpstra]

On Nov 28, 2007, at 4:23 PM, Matthew Fluet wrote:
> Right, but doesn't allow you express times beyond that point.
>
> We used to implement Time.time as
>  datatype time = T of {sec: Int.int, usec: Int.int}
> but switched to using IntInf.int (representing microseconds since  
> the epoch) in Sep. 2003:
>  http://mlton.org/pipermail/mlton/2003-September/024318.html
> and then shortly thereafter using IntInf.int (representing  
> nanoseconds since the epoch).
>
> The only discussion I could find on the choice of Time.time  
> representation were back from July 2001:
>  http://mlton.org/pipermail/mlton/2001-July/019405.html

I concede the point. Further analysis has convinced me that this won't  
be a big deal even in a heavy gettimeofday user like a network  
application.

My comments were mostly motivated by the aesthetic cleanliness of  
using a single C call followed by _symbol invocations to grab the  
result and only noticed the cost of IntInf conversions afterwards.  
Your solution to the FFI issue raised on mlton-user was a good MLton  
FFI programming pattern I wanted to emphasize as being generally useful.

Since the TIME signature creates value via the IntInf conversion path,  
I doubt changing the representation will help without first solving  
the more crucial failure to catch (IntX.fromLarge o IntY.toLarge).  
I've looked into how to do this and the main problem seems to be that  
all the Overflow handling logic is inside the basis. This prevents a  
rewrite rule from being written in the SSA stages.

As a recap, the basis recommends programmers to use idioms like:
val convert1 = Int64.fromLarge o IntX.toLarge
val convert2 = Int64.fromLarge o Real.toLargeInt IEEReal.TO_ZERO
val convert3 = Int16.fromInt o Int8.toInt
val convert4 = Char.chr o Char.ord

A brief summary of the current state of affairs under MLton:

For convert1, the procedure depends on the size of ObjptrWord and X.  
Suppose ObjptrWord=32.
If X>=31 and the value is >=30bit big, a very expensive conversion  
involving arrays and recursion is triggered.
If 16<X<=30, the array path is generated but never used (compiler  
cannot determine that the range test always succeeds)
If X<=16, the value is packed/unpacked via an ObjptrWord intermediate  
+ bit twiddles.

A programmer probably expects X <= 64 to result in a noop. For X > 64,  
he probably expects a range test and a noop.

convert2 is implemented by converting the real to a textual string(!),  
converting that via the gmp string parser, and then converting into an  
int via the array path.

convert3 isn't so bad, but it does feel the need to unnecessarily  
check for overflow.

convert4 checks twice for overflow (and the optimizer doesn't  
recognize this -- one is a < comparison and another is a scale down/up  
primapp = comparison). The same double-checking happens for  
Word64.toInt.

I think the right thing to do is:
1. remove the conversion/overflow code from the basis
2. add smarter prim apps:
   RealX_{to,from}IntInf => checks for size mess-ups
   Word[1-64]_{to,from}[Un]SignedIntInf  => ditto
3. detect composition of the 'smart' prim apps
4. translate the smarter prim apps during SSA into the existing:
   Word_extdToWord -- does NOT check for bounds

As an aside, there is an mpz_set_d one can use for converting doubles  
into IntInfs.

However, this is a big project for another day. I know to avoid  
conversions through IntInf in critical sections. :-)

>> In a well written network application, an strace looks like:  
>> select() gettimeofday() maybe read() maybe write() gettimeofday()  
>> repeat.
> I'm curious as to why you need so many gettimeofday() calls.

You need to know the current time in order to be able to schedule  
relative events, like: 'read a value or timeout in 5 seconds'.  
Typically you cache the time immediately after select b/c nearly every  
event handler will use (at least) one of these. Also by doing it right  
after select, you can make sure things like 'do X 5ms after Y' are not  
too late due to handling another event in between.

The second gettimeofday is so you can determine how long you should  
sleep till the next event in the queue (because the event handler took  
time).