[MLton-user] IOType 1.0, binary serialization preprocessor

[Christopher Cramer]

On Thu, Aug 25, 2011 at 09:43:31AM -0400, Matthew Fluet wrote:
> It would be good to note in the README that it supports polymorphic
> types in the expected manner: for every polymorphic type variable,
> readTy and writeTy take a reader/writer argument.

Well... I do give an example of this in the README, but I guess I can
make it more explicit.

> It would also be good to note how the library/preprocessor deals with
> sharing and types with identity (like 'a ref and 'a array).  In
> contrast to some serialization libraries, this one doesn't attempt to
> preserve sharing or identity.

Yes, I shall put that in.

> I like the use of (*#line ?.?? "???"*) directives in the preprocessor.
>  You might explicitly introduce #line directives for the code added by
> IOType; that is, the source location for readTy and writeTy ought to
> be the output .sml file, not the input .ioml file.

I was actually trying to think of what to do in that case. That seems
like the right thing to do. It's just a little tricky to figure
out the location in the output file. I guess have an extra layer over
TextIO that tracks it...

> On the one hand, I applaud the effort to use a full parser to handle
> the input.  At the same time, I worry that it might be fragile.  As
> you note in your TODO file, it would be nice to support some of the
> Successor ML proposals, but the right place to do so would probably be
> in the SML compiler; but then some Successor ML + IOType code might be
> hard to handle --- if IOType doesn't support the extensions, it won't
> process the files.  It is also fragile in the face of someone writing
> an EqType or OrdType preprocessor/library, which also wants to add new
> keywords, etc.  I don't have any good suggestions here, but just a
> comment about this kind of preprocessing.

I have been thinking about this. It's basically an issue that Lisp
sidesteps by structuring everything with parentheses, and I suppose
this is exactly what Camlp4 is supposed to solve for OCaml. IOType is
basically the same idea as bin-prot for OCaml, which uses the whole
Camlp4 framework and thus works with any other Camlp4 extensions.

As a practical matter, in the case where you have extensions that are
not understood by the IOType preprocessor (for MLton this would be the
foreign function interface), you can simply put the types you want to
serialize in a separate file, and then include/open it in the file with
the extensions.

What I was thinking of doing when I put that stuff in the TODO was a
whole new preprocessor, incorporating both the IOType functionality
and the Successor ML proposals that can be easily implemented in a
preprocessor without type inference. If you put it into the compiler,
then you have the problem that one compiler supports some things and
then another compiler doesn't. And I don't expect another revision of the
definition (or even some sort of formalization of Successor ML) to
happen.

But... I'm not sure if it's worth the effort to implement any more
extensions. I've been using SML for about a year and a half now, having
switched from OCaml, and for the most part I don't miss any of the
extra stuff that OCaml has.