[MLton-user] Profiling the live heap?

[Matthew Fluet]

On Wed, May 4, 2011 at 4:18 AM, Christof Leng <cleng at dvs.tu-darmstadt.de> wrote:
> I'm implementing a network simulator in SML with MLton. It seems that I
> have a memory leak in long running simulations. Although this adds up to
> GBs over time, only a very small fraction of all allocated memory does
> not cleaned up by GC correctly (presumably due to some bug in my
> application). Therefore "-profile alloc" didn't do the trick for me. Is
> there a way to identify only the allocations that are still in the live
> heap?

Unfortunately, no, there isn't a way to do this kind of heap profiling
at the current time.  (Though, it would be an interesting project.)
Right now, the runtime system only records the representation type of
an allocated object (i.e., the size of the object, and the number of
pointer fields); in particular, it doesn't record anything about its
point of allocation.

Wesley once wrote a patch that dumped the heap graph reachable from a
given root:
  http://mlton.org/pipermail/mlton/2008-November/030442.html
He was also interested in annotating objects with more interesting
information out their origins.

It might not be too difficult to tweak the garbage collector to
compute a histogram (indexed by object representation type) of the
live heap.  If the leaky object has a sufficiently distinct
representation, perhaps you could deduce the SML object to which it
corresponds and its likely allocations.

One difficulty with associating fine-grained allocation information
with objects is that the object header reserves 19 bits for the index
into the object types table.  2^19 is (supposedly) more than
sufficient for the number of different representations of objects
within a single program, but it probably falls short of the number of
different allocation sites in a large program.