[MLton] max-heap setting for 64-bit applications

[Matthew Fluet]

On Fri, Dec 11, 2009 at 1:15 PM, David Hansel
<hansel at reactive-systems.com> wrote:
> Thanks for the detailed description on MLton's memory usage.  This
> does explain what is going on.  I ran the example with "gc-messages
> max-heap 3000M" and I see the following before the trashing starts:
>
> [GC: Growing heap at 0x0000000080000000 of size 3,062,628,352 bytes
> (+ 23,986,176 bytes card/cross map),]
> [GC:    to desired size of 3,121,217,536 bytes and minimum size
> of 3,070,031,512 bytes.]
>
> When running with the "max-heap 2950M" setting,  all the other
> messages are the same but the final "growing" operation does not
> happen ("Out of memory" is reported).
>
> This seems to reflect exactly what Matthew suggested,  MLton is
> trying to grow the heap to its maximum size which causes the
> thrashing.

Right, and the annoying bit is that the previous heap was so close to
the max heap setting.  Perhaps a reasonable heuristic is that if a
desired heap is "close-to" the max-heap size, just round up.  Perhaps
0.75 of max heap?  In the max-heap 3G setting, this could still leave
you in the situation where you have a 2.25G allocation and a 3G
allocation at the same time to copy.  Or 0.55 of max heap; that could
require 1.65G+3G at the time of the copy.

> It seems like one strategy would be that keepomg the "max-heap"
> setting slightly under half the available physical memory should
> avoid the case where we're already using about 50% and then have
> to create another heap of the same size.

True.  Although, you would really need to use about 50% of the
physical memory that you want the MLton process to have access to,
else you will page via competition with other processes.

> Another observation I've made is that using "fixed-heap" instead
> of "max-heap" seems to avoid the thrashing (unless I set the
> size unreasonably high,  like 5GB on my 6GB machine running a
> number of other applications).  I had somewhat expected for MLton
> to immediately allocate the whole heap but (according to the
> task manager) that did not happen.  Are there any other drawbacks
> about using fixed-heap?

It would make sense for fixed-heap to grab the whole heap at once.
However, because the semi-spaces of the copying garbage collector are
two separate heaps (not two halves of one heap), MLton will initially
only allocate one of the heaps and won't allocate the secondary heap
until the first (major) garbage collection.  If you run with @MLton
fixed-heap 3G --, then MLton will allocate a 1.5G heap, wait for that
to fill, perform a major copy-collection into another 1.5G heap (now
we have two 1.5G heaps, matching our "fixed-heap" setting).  In your
example allocating program, where there is almost no garbage, after
the major copy-collection, MLton will see a heap at near capacity and
attempt to resize it to 3G (the "fixed-heap" setting), in order to
have a larger heap for mark-compact collection.  Going from 1.5G to
3G, even when both are allocated for the copy, probably isn't
noticeable as thrashing.  If, after the copying collection, the live
data was a small fraction of the heap, then MLton stick with copying
collection (using a 1.5G heap), but won't deallocate the secondary
heap unless keeping both of them exceeds the available RAM.

I don't think that there are any drawbacks to using fixed-heap, as
long as you are willing to devote that amount of memory to the
process.  Especially on a 64-bit system, if you set the fixed-heap to
a proportion of ram such that the jump from 1/2-fixed-heap to
fixed-heap isn't noticeable as thrashing, then you should have
predictable behavior.  On a 32-bit system, it isn't as nice.  Since we
now have lots of systems with >4G memory, but only 4G virtual address
space, it would be very nice to run processes with fixed-heap 3.25G.
Unfortunately, because of the above described semi-space behavior,
what end up happening is that a 1.125G heap is allocated and then it
can be difficult to allocate the other 1.125G heap in the address
space.  And can be even harder (and, indeed, in the absence of a
proper mremap, impossible) to grow the 1.125G heap to 3.25G without
the runtime dumping the 1.125G heap to disk.  On those systems, you
really want to grab the whole 3.25G up front.  Going to a single
contiguous heap, interpreting it as two semi-spaces when using a major
copying-collection would be nicer here, because fixed-heap would grab
the whole 3.25G up front.