[MLton-user] IOType 1.0, binary serialization preprocessor

[Matthew Fluet]

On Tue, Aug 23, 2011 at 4:00 PM, Christopher Cramer
<tsuyoshi at yumegakanau.org> wrote:
> I've just released the version 1.0 of IOType, which is a Standard ML
> preprocessor that generates binary serialization functions from type
> definitions.

Looks like an interesting and useful library.  I only looked at it
briefly, but here are some thoughts:

It would be good to note in the README that it supports polymorphic
types in the expected manner: for every polymorphic type variable,
readTy and writeTy take a reader/writer argument.

It would also be good to note how the library/preprocessor deals with
sharing and types with identity (like 'a ref and 'a array).  In
contrast to some serialization libraries, this one doesn't attempt to
preserve sharing or identity.  For example, in the following:

  iotype t = int list list
  val z = let val l = List.tabulate (1000, fn i => i) in List.tabulate
(1000, fn i => l) end
  ...
  val () = writeT outStream
  ...
  val a = readT inStream

The size of the data structure reachable from 'a' will be 1000 times
larger than the size of the data structure reachable form 'z'.

I like the use of (*#line ?.?? "???"*) directives in the preprocessor.
 You might explicitly introduce #line directives for the code added by
IOType; that is, the source location for readTy and writeTy ought to
be the output .sml file, not the input .ioml file.

On the one hand, I applaud the effort to use a full parser to handle
the input.  At the same time, I worry that it might be fragile.  As
you note in your TODO file, it would be nice to support some of the
Successor ML proposals, but the right place to do so would probably be
in the SML compiler; but then some Successor ML + IOType code might be
hard to handle --- if IOType doesn't support the extensions, it won't
process the files.  It is also fragile in the face of someone writing
an EqType or OrdType preprocessor/library, which also wants to add new
keywords, etc.  I don't have any good suggestions here, but just a
comment about this kind of preprocessing.