[MLton] flattening of arrays and vectors

[Matthew Fluet]

> > With regards to the September release, is that version planned to
> > have the undumped (save world) processing of the basis library?  Now
> > that MLton can be used during development, it would be very nice to
> > have the time from start to detection of type/syntax errors be
> > shorter.
>
> I agree with Henry that this would be a nice feature.  I am thinking
> that something really simple might give a lot.  Matthew, do you have
> any thoughts on how hard this would be?

I don't imagine the implementation would be that hard, but I don't know of
a good design for this mechanism yet.

Essentially, it's a matter of exposing the MLB->Ast.Basdec.t and
MLB->Elaborate.Env.Basis.t maps, and having additions to them be preserved
between invocations of the compiler.  Saving the world and replacing the
mlton.world is one way of accomplishing it.

Right now, those maps are from OS.FileSys.file_id to their respective
ranges.  As we've already noted, file_id makes good sense for establishing
file identify during a compile, but is bad for moving directories around
and certainly aren't valid for install packages.  Also, one reason we went
to distributing the Basis Library sources was to make it easier for users
to fix bugs without waiting for a new release.  So, we either need to
expose how to build a new world when a user's Basis Library changes, or
explicitly check all the dependencies.

There is the other aspect that the Elaborate.Env.t isn't very
compositional.  In one sense the environment just keeps growing, although
things keep falling in and out of scope.  There doesn't seem to be any way
to remove/replace all the information that will be associated with a
cached .mlb file when a user decides to install an updated .mlb entry.