[MLton] Crash fread(...) failed (only read 0) (Cannot allocate memory) during deepFlatten with MLton 20070826

[Nicolas Bertolotti]

> The only "hint" would be to move the struct GC_generationalMaps from the
> struct GC_state to the struct GC_heap and 'reinterpret' the mmap-ed
> memory for the heap as
>
>   ------------------------------------------------------------------------
> -
>   |    old generation    |               |  nursery  | cardMap | crossMap
> |
>   ------------------------------------------------------------------------
> -
>   |------oldGenSize------|
>   |-----------------------size-----------------------|
>   ^                                      ^           ^         ^
>   start                                  nursery     cardMap   crossMap
>
> Then it is just propagating this interpretation through...
>
>
> The secondary heap is used as the to-space for major cheney-copy garbage
> collections; at the end of the GC, the secondary heap is swapped with the
> primary heap (swapHeapsForCheneyCopy).  It never holds live data while the
> mutator is running, and hence doesn't need its own card/cross map.  We try
> to hold on to a secondary heap (resizeHeapSecondary) so long as it doesn't
> cause paging and we can keep it as large as the primary heap.

I tried a number of different solutions. It seems that moving the GC_generationalMaps struct to the GC_heap struct requires lots of changes in different parts of the code and would also lead to some useless computations when we manipulate the secondary heap (either adding some code to determine that we don't need to manipulate the card/cross map in that case or manipulate it anyway).

I finally implemented a solution in which the data structures don't change and which is less intrusive.

While I was debugging this solution, it appeared that the function growHeap() may actually shrink the heap when the minimum size is less than the current size which, I guess, is not the expected behavior. Now the card/cross map is stored at the end of the heap, it is important to ensure that the heap grows in that case so I added some conditions here.

Using this patch, I don't have any sporadic crashes when building my application (or MLton itself) on the 4 Gb debian box.

I am interested in getting some feed-back about this patch so don't hesitate.

Last point: the function growHeap() stores the current heap to disk when it can not allocate enough memory for the new heap. As this function is sometimes called with a minimum size that is less than the current heap size and this operation is quite costly, don't you think we could avoid it and keep the heap as is in that case?

Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: card-cross-map-in-heap.patch
Type: application/octet-stream
Size: 17187 bytes
Desc: card-cross-map-in-heap.patch
Url : http://mlton.org/pipermail/mlton/attachments/20080505/8ec146bf/card-cross-map-in-heap.obj