[MLton] Address space in windows

[Wesley W. Terpstra]

On Tue, Oct 14, 2008 at 6:01 AM, Wesley W. Terpstra <wesley at terpstra.ca> wrote:
>>> 1. Eliminate the use of VirtualAlloc/Free on cygwin

I have elected not to do this. While it is true that mmap and VA can
grab the same memory, cygwin likes to re-use munmap'd memory ASAP
(probably because it just decommits it like us). However, this means
that if MLton ever shrinks it's heap all it takes is one allocation by
cygwin to make it impossible for us to recover the memory we gave up.
When using VA, this fragmentation doesn't happen because cygwin
doesn't know about our decommitted memory.

Both use-mmap and the default (VA) acquire > 1.1G of RAM now without
problems on cygwin. On MinGW, we easily grab >1.9GB of memory.

As a side note, the reason cygwin memory is smaller is due to the
unfortunate position of the cygwin DLLs at address:
61000000-61200000 r-xs 61054740 8CBD:AB20 1125899906875461
/usr/bin/cygwin1.dll

The limit to 2G on MinGW comes from the windows system DLLs which are
loaded around 7FFF0000.

Another thing I've noticed is that MLton's estimate of the available
RAM can trick it into making bad choices for the heap size. You can
only possibly get 1.2G on cygwin, but MLton thinks it should be able
to get an amount = physical ram. It sets the minSize and desiredSize
both too high and is the remap then fails. However, if it had a less
aggressive minSize, it could have allocated another 300MB. I'm not
sure what to do about this. Perhaps cap the physical RAM based on the
platform?

> Yes. I'm not sure what the downside of many small mappings is vs. one
> big mapping. Is there a performance penalty? It's all anonymous
> memory. Does mmap'ing two pages side-by-side differ from one mmap of
> two pages? On windows, yes. On *nix? Isn't this how the C heap is
> grown?

These questions all remain open.

> We could easily include a moving copy in the fake mremap. eg: try
> first to expand in-place, if that fails, try an out-of-place
> mmap+memmove.

My patch now does this, so "fragmentation" with many concatenated maps
only happens if you really need mremap to do it.