[MLton] Writing memory to disk ...

Stephen Weeks sweeks@sweeks.com
Wed, 24 May 2006 10:49:47 -0700


Writing the heap to disk in tight memory situations has proved useful
to more people than just Henry.  It is not viable to remove this
functionality.  Having an option to enable/disable it is fine.
History would argue for having the default be ON.  However, the usage
is esoteric enough that it's OK to go against that and make the
default be OFF.  Security and other least-surprise concerns argue this
way as well.

As to what directory gets used, the current behavior is to use
C:/WINNT/TEMP on Windows and /tmp elsewhere, overridable by an
environment variable, $TEMP on Windows and $TMPDIR elsewhere.

When the feature is enabled, it is important for MLton to choose a
good default across platforms so that people don't have to duplicate
the logic for this.  It is a bad idea to require the tempdir on the
command line, because of duplication of logic, because people are
likely to get it wrong, and because we already support overriding it
via environment variable.  I'm fine with making the default tempdir
whatever is most expected on each platform.  I also wouldn't object to
optionally specifying the tempdir on the command-line instead of via
an environment variable if people think that's less surprising.

I don't understand Wesley's claim that MLton doesn't check that the
write to disk succeeds.  The runtime uses the write_safe call (in
x86_64 branch lingo), which certainly checks, and fails if the write
was not complete.  If there's a problem here, it should be fixed, but
I don't see it. 

To sum up, I propose the following changes:

 * Disable this functionality by default.
 * Add a runtime option, "disk-backed" (I'm open to other names), that
   enables this functionality. 
 * Refactor as Ville suggests, moving the code to platform/*
 * Change the default directory on non-Windows machines to /var/tmp.
   Is there any other platform that should remain as /tmp or be
   something else entirely (AIX, *BSD, Darwin, HP-UX)?