[MLton] RE: A MLton / C-- Experiment

[Matthew Fluet]

Both Norman and Simon gave (different) means of supporting multiple 
SML/Haskell threads in a C-- targetting compiler.  I think they are both 
fine solutions, but I felt they fell outside the scope of the experiment I 
wanted to run (for different reasons).

On Sun, 13 Mar 2005, Norman Ramsey wrote:

>  >  2) Programs that make use of MLton's thread library cannot be
>  >     supported.  Attempting to multiplex the one C-- stack among
>  >     multiple ML stacks would lead to disaster.
>
> We have some experimental support for multiple C-- stacks.
> But since this support cannot detect stack overflow, it is
> emphatically unsuitable for any sort of production use.
> (People at Bell Labs seem to be willing to run threads with no safety
> net, but I am not.)

Ultimately, multiple C-- stacks would be the way to go in any C-- 
targetting compiler.  Norman admits this is currently an experimental 
feature.  Still, my reasons for not pursuing this approach are twofold:

1) Starting a new C-- stack everytime we start a new SML stack would 
require more invasive hacking into the runtime system, which I wanted to 
avoid.

2) MLton doesn't actually ever create a new SML thread from scratch.  
Instead, the primitive operation is to copy an existing (including 
the currently executing) thread.  Using either the C-- stack as *the* 
stack or using a pair of a C-- stack and an SML stack, still requires a 
way to copy a C-- stack, including relocating all continuations.


On Mon, 14 Mar 2005, Simon Peyton-Jones wrote:

> That's exactly our plan for GHC.   Current we generate C exactly like
> Mlton does; that is, GHC lays out the GHC stack, and there's a parallel
> system stack which only the C compiler touches.  However, the system
> stack never grows, because every transfer is a tail call.  If we want a
> call we push a return address on the Haskell stack, and do a jump.  To
> return, we jump to the return address taken from the Haskell stack.  A
> "jump" is a tail call in C land -- which is hacked into a real jump by
> post-processing the asm code generated by gcc, or by using a trampoline.
> 
> So we plan to set up an alternative route that compiles to C-- instead
> of C, but still uses GHC to lay out the Haskell stack, making no use of
> the system (= C--) stack except for spilling live registers to.  Since
> the C-- stack never grows, there is no problem with concurrency.  The
> same C-- stack can be used for any number of threads, because the C--
> stack is always empty at every Haskell-level control transfer (including
> thread switch).   This route is mostly done; but needs to be tied up and
> tested.

Ensuring that the C-- stack never grows would be an alternative solution 
that would work in MLton; as noted, some trickery is needed to get C to 
behave in this manner.

My reasons for not pursuing this approach is that I felt it would not give
an accurate picture of C--, certainly not accurate enough to make any
decisions on whether or not to pursue using C-- as *the* stack.  My
impression from MLton is that adopting this solution would give rise to
lots of little C-- functions (at a minimum, corresponding to each non-tail
return and handler continuation (in the Machine IL), and in the limit
corresponding to each basic block) and some additional glue to ensure that 
values not on the ML stack are passed correctly between these little C-- 
functions.

In any case, I want to see how qc-- handles *BIG* functions, how fast cut 
to transfers work, how span lookups work, etc.  These are all aspects that 
would figure heavily in a solution using C-- as *the* stack.