[MLton] Crash fread(...) failed (only read 0) (Cannot allocate memory) during deepFlatten with MLton 20070826

[Nicolas Bertolotti]

> It is a different problem, in the sense that you succeeded (at least
> once) in writing the heap to disk, allocating a larger heap, and reading
> the heap from disk.  Getting an out of memory error suggests that even
> after writing the heap to disk and releasing it, the operating system was
> unable to mmap 1.6G.  That seems strange, even on a 32bit system, where
> the virtual address space is 4G, though I suppose it could happen with
> sufficient fragmentation.
>
> If you compile with @MLton gc-messages --, you should get a more detailed
> memory listing of what memory is reserved, which might indicate whether
> there is some extra memory being held.

It is exactly what I am doing right now...

When the failure occurs, there is still a lot of swap space available ... and, as you mentioned, the virtual address space is the same on the machine with 4Gb of RAM and the 2Gb one.

I just noticed that the failure occurs while the GC tries to allocate a new heap with a "minSize" value that is actually smaller than the size of the previous heap (that has just been released).

I have applied the following patch (just to give a try):
--- mlton/runtime/gc/heap.c     2008-04-22 18:27:34.000000000 +0200
+++ mltonp1/runtime/gc/heap.c   2008-04-11 19:12:58.000000000 +0200
@@ -208,9 +208,6 @@
                uintmaxToCommaString (backoff),
                uintmaxToCommaString (minSize));
     }
+    /* Make sure we always try to allocate at least 'minSize' bytes. */
+    if (h->size > minSize && (h->size - backoff) < minSize)
+      backoff = h->size - minSize;
   }
   h->size = 0;
   return FALSE;

Nicolas