[MLton-user] Fwd: Plans for pickling in MLton?

[Vesa Karvonen]

Roland Olsson <roland.olsson at hiof.no>:
> I have been using MLton for MPI programming the past years and would like
> to have support for pickling, also known as serialization, in order to
> more easily be able to send and receive data objects. Even if there are
> combinators for pickling, such as the ones published by Andrew Kennedy and
> others, it would make life easier to have pickling "built-in" in MLton.

I believe that MLton used to have built-in pickling, but I think that
it was removed at some point (before I personally started using
MLton).

Certainly, there are some things that are impossible with a combinator
based approach.  For instance, I think that it might be possible to
build some very interesting frameworks if it were possible to pickle
threads (which, of course, aren't even Standard ML).  However, I have
recently been developing a generic programming library (using pure
SML'97) and it also happens to include a pickling generic.  Of course,
it cannot pickle closures (or threads for that matter), because it is
impossible within SML'97, but pretty much everything else can be
pickled (arbitrary user defined data types).

A possible workaround might be to write a simple interpreter and
pickle programs, which can, of course, be wrapped as closures using
the interpreter. Assuming that the programs written in that
interpreted language were mostly calling built-in functions of the
interpreter (running at MLton's full speed) then you might even get
acceptable performance that way. Doing it this way (having a custom
interpreter and pickling programs) would also probably have several
advantages.  For example, it could be made platform and compiler
independent.  It would also allow you to perform any checking and
verification on the programs that you want before running them to
ensure safety and security.

You can find the generics library from MLton's repository:

  http://mlton.org/cgi-bin/viewsvn.cgi/mltonlib/trunk/com/ssh/generic/unstable/

Here is the signature of the pickling generic in particular:

  http://mlton.org/cgi-bin/viewsvn.cgi/mltonlib/trunk/com/ssh/generic/unstable/public/value/pickle.sig?view=auto

The current implementation does not allow the user to specialize
pickling, but I know how to do it (see the Pretty generic) and plan to
add the support in the near future.

I do not claim (or even believe) that the current implementation of
the pickling generic would be particularly efficient, but I'm pretty
sure that the design admits efficient implementation.  If you want to
try the library, I would be happy to help.  Also, if you do try it, I
might be able to help by improving the implementation if you run into
problems (performance).

The implementation techniques used in the generics library are
discussed in my paper:

  [8] Generics for the Working ML'er.
      Vesa Karvonen.
      In Proceedings of the 2007 workshop on ML.

If you want, I can send you a copy of the published paper.  I also
hope to release a slightly revised and extended version of the paper
in the near future.

> Generally speaking, I am very satisfied with MLton, especially the
> robustness and relative lack of bugs in the compiler, and hope that its
> development will continue.

I second that!

-Vesa Karvonen