[MLton-user] The Generic Programming Library is great

[Vesa Karvonen]

On Jan 31, 2008 11:11 PM, Ville Laurikari <ville at laurikari.net> wrote:
> I've recently taken the Generic Programming Library by Vesa Karvonen
> into use and though it would be a good idea to share the experience.
> I urge all SML'ers to take a look at this library.  Chances are you
> can do away with a lot of your one-off ad-hoc code and offload the
> tasks to this library.

Thanks for the endorsement!  :-)

I'm interested in hearing about any problems anyone might encounter while
using or trying to use the library.  If you run into problems, please post
about them on the MLton users list or e-mail me.

> There's pickling:
>    http://mlton.org/cgi-bin/viewsvn.cgi/*checkout*/mltonlib/trunk/com/ssh/generic/unstable/public/value/pickle.sig
>
> Pickling means converting values to a platform independent and
> relatively space efficient binary format (and back, of course).  You can
> then dump that data to files, send it over the network, and so on.

Right.  A key feature of the pickling generic is that the pickle format is
platform independent.  SML does not completely fix the representations of
basic types like integers, which can be either arbitrary precision or have
some, even configurable, fixed precision.  Thus pickling involves
converting the values to a platform independent format and unpickling
involves the inverse.  The pickling generic has also been designed to be
flexible in that you can directly pickle to or unpickle from any input or
output stream (e.g. you can directly pickle to a file or to a socket).
These both imply some inherent overheads, although the current
implementation could likely be improved significantly in terms of
performance.

As a part of an IPC library, I've been thinking about introducing a
specialized pickling generic that would both use a platform dependent
format and only pickle to and unpickle from raw (usually shared) memory
[RAWMEM].  This should make amenable to highly efficient implementation.
The main usage would be for exchanging immutable data (messages) between
two (or more) processes running in parallel.

[RAWMEM] There is in fact a kind of prototype implementation of this at

  http://mlton.org/cgi-bin/viewsvn.cgi/mltonlib/trunk/com/ssh/ipc/unstable/

but it was written before the generics library.  I intend to rewrite the
RawMem pickler using the generics framework, making it essentially just
another generic.

> Even show/read and pickle/unpickle alone are wonderful but there's more,
> such as generic equality testing, hashing, and sorting.  Check out the
> README for more details.

In addition to the generics that you mention, generic "reduce" (a kind of
generic fold) and "transform" operations (a limited form of a generic
fmap) (and also an experimental generic fmap) can also be useful in
application programming.  During ICFP'07, I wrote a simple example of
using the reduce and transform generics to canonicalize the terms of
a trivial language.  I should add it to the examples...

The library currently provides some 19 "generics" of which 10, I believe,
are likely to be useful in many kinds of applications (dynamic, eq, fmap,
hash, ord, pickle, pretty, read, reduce, transform) while the others are
likely to be useful in only a few specific applications (arbitrary, size,
shrink) or the implementation or testing of other generics (data-rec-info,
seq, some, type-exp, type-hash, type-info).

If you find that you have an application that could use a kind of generic
function that isn't provided by the framework, I'd like to hear about it.

-Vesa Karvonen