FW: [MLton] Crash fread(...) failed (only read 0) (Cannot allocate memory) during deepFlatten with MLton 20070826

[Nicolas Bertolotti]

Sorry, bad key combination ...

> BTW, if you are just changing the runtime, you might find it useful to
> rebuild the compiler with '-keep o', which will leave all of the object
> files from the compiler around.  Then you can compile with
> 'mlton -output mlton-compile *.o' which will just link the object files
> with the new runtime.

Sure ... it may also be useful to update the Makefile in order to build
the runtime as a shared library.

> I've previously suggested trying the 'hash-cons p' runtime option, that
> instructs the runtime to hash-cons every 1/p (major) garbage collections.
> This can reduce the live data by sharing data.  Of course, the
> hash-consing table requires some space, which is may be hard to come by in
> this situation.

Wow, I probably read your mail too quickly ...

I tried the option yesterday. Here is what I get:
...
            deepFlatten starting
82,763,496 bytes hash-consed (8.1%).
52,284,080 bytes hash-consed (3.6%).
16,717,560 bytes hash-consed (1.2%).
58,765,644 bytes hash-consed (3.4%).
10,277,672 bytes hash-consed (0.6%).
9,820,592 bytes hash-consed (0.6%).
22,914,436 bytes hash-consed (1.3%).
1,530,184 bytes hash-consed (0.1%).
26,915,764 bytes hash-consed (1.5%).
4,806,936 bytes hash-consed (0.3%).
6,266,360 bytes hash-consed (0.4%).
10,466,984 bytes hash-consed (1.1%).
81,219,412 bytes hash-consed (7.3%).
70,647,804 bytes hash-consed (4.0%).
13,195,012 bytes hash-consed (0.7%).
13,399,084 bytes hash-consed (0.7%).
3,415,028 bytes hash-consed (0.2%).
6,683,680 bytes hash-consed (0.3%).
360,096 bytes hash-consed (0.0%).
1,665,636 bytes hash-consed (0.1%).
24,136,472 bytes hash-consed (5.0%).
            deepFlatten finished in 41.52 + 1543.72 (97% GC)
...
which is unfortunately not good enough to avoid swapping.

>
> Beyond that, there aren't any runtime or compiler options that will affect
> the memory usage of the deepFlatten pass.  Either disable the pass
> completely ('-drop-pass deepFlatten') or you will need to investigate
> algorithmic changes to the deepFlatten pass.
>

OK, then my short term solution will be to submit a request to have more RAM on the build machines

Nicolas