[MLton-user] out of memory

[Matthew Fluet]

On Tue, Jun 1, 2010 at 9:09 PM, Adam Goode <adam at spicenitz.org> wrote:
> On 06/01/2010 03:57 PM, Matthew Fluet wrote:
>> On Tue, Jun 1, 2010 at 2:28 PM, Sean McLaughlin <seanmcl at gmail.com> wrote:
>>> Hello,
>>>
>>>   Both solutions you proposed worked.  Oddly I think, may-page-heap true was
>>> significantly faster than -drop-pass deepFlatten.
>>
>> That is odd.  Paging the heap to disk is generally quite expensive (in
>> your case, it requires writing and reading a >1GiB file).  It would
>> also mean that your program isn't quite a worst-case scenario for
>> DeepFlatten, since it was able to complete with just a little more
>> heap space.  On the other hand, with the greater heap space, the rest
>> of the compile needs fewer garbage collections.  It might also
>> indicate that deep flatten did enough work to significantly improve
>> the remainder of the compilation.
>>
>
> It might also be that the OS is using the remaining 2GB of memory as an
> efficient page cache. If the computation is fast enough, the data might
> not hit disk at all.

Good point.

> Is there a 64-bit MLton for Mac OS? That is probably the best solution
> for this issue.

A 64-bit MLton for Mac OS X is certainly possible.  Indeed, it is the
default on Snow Leopard.  On Leopard, you should be able to build a
64-bit MLton by doing "make TARGET_ARCH=amd64" from the top-level
Makefile.  The limiting factor is that you need a 64-bit build of GMP.
 MacPorts (my preferred Unix tools on MacOS "distribution") doesn't
include x86_64 in the universal build of GMP on Leopard, though it
does on Snow Leopard.