[MLton] RE: card/cross map in heap

[Nicolas Bertolotti]

I recently experienced some issues with the patch when trying to build some big binaries.

The problem comes from the fact the remapHeap() function now succeeds too often.

As a matter of fact, it now always succeeds when it is not able to grow the heap and the current heap size is greater or equals minSize.

Then, when it is no longer possible to grow the size of the heap without dumping it to the disk, the application continues to run without dumping to disk until minSize really gets bigger than the current heap size.

This really slows down the execution as, it is much better in that case to dump the heap to disk and be able to allocate a bigger buffer than to keep going and entering GC every time we need to allocate a few bytes.

On the other hand, the fact that we guarantee that we try to allocate minSize bytes is a good thing in the context of the createHeap() function as it guarantees that the application won't crash until we really run out of memory.

In order to solve the issue, I have added the following piece of code at the beginning of the remapHeap() function (see the attached patch, based on the one you previously sent):
  if (desiredSize > minSize && h->size == minSize)
    minSize = align(minSize + backoff, s->sysvals.pageSize);
I am not sure it is the best thing to do, but it works fine in my case.

Nicolas

> -----Original Message-----
> From: Matthew Fluet [mailto:fluet at tti-c.org]
> Sent: Tuesday, June 24, 2008 3:24 PM
> To: Nicolas Bertolotti
> Cc: Matthew Fluet; mlton at mlton.org
> Subject: Re: [MLton] RE: card/cross map in heap
>
> On Mon, 16 Jun 2008, Nicolas Bertolotti wrote:
> >> If this patch also eliminates the out of memory crashes, then I think
> it
> >> is preferable, since it has a simpler management of the card/cross map.
> >
> > It should as it also allocates both card/cross map and heap in one shot.
> > I have already included it in my working version and it will be used to
> > build a next release of the product.
>
> I'm assuming that you had no issues with this version of the card/cross
> map in heap patch.  I'll plan on merging the patch in the not to distant
> future.
>
> > Also note that I actually faced the bug you found in my previous patch.
> > It crashed on Windows and Solaris ... but not on Linux.
>
> Interesting that it worked on Linux.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 02-card-cross-map-in-heap-fluet.patch
Type: application/octet-stream
Size: 30444 bytes
Desc: 02-card-cross-map-in-heap-fluet.patch
Url : http://mlton.org/pipermail/mlton/attachments/20080627/5f147f4d/02-card-cross-map-in-heap-fluet-0001.obj