[MLton] max-heap setting for 64-bit applications

[David Hansel]

Hi all,

I have a few questions about the max-heap setting for 64-bit
applications built by MLton.

The code below is an example that is supposed to allocate more
and more memory until it is terminated.  While it is running,  it
outputs roughly how much memory it has allocated so far.

I compiled the example with MLton (r7353 or r7369,  no difference)
on 64-bit Windows 7 as a 64-bit application.   The computer has
6GB of RAM, of which 3.5-4.0GB are available to the application.

If I run the example with "@MLton max-heap 2950M --" run-time
parameters then it keeps allocating memory,  the numbers printed
by the example being roughly in sync with what Windows' Task
Manager shows.   Around 3GB it is terminated with the message
"Out of memory with max heap size 3,093,299,200",
which seems like exactly what should happen.

However,  if I run the example with "@MLton max-heap 3000M --"
then around the the same point where it was was stopped before
(the program reports roughly 3.05GB of allocated memory), my
system starts thrashing and I can see in the Task Manager
that the program has allocated around 4.5GB of memory and is still
trying to get more.

Is this what you would expect?  I thought that the max-heap setting
would include the amount of additional memory required for garbage
collection.  Is that true?  Or do I always have to just allow
half of the system memory for max-heap to make sure to not allocate
too much memory?

Thanks,

David


Example code:


fun mk10MBArray () =
    let
      val n = 2500000
      val a = Array.array (n, 0)
      fun loop i =
	  if i >= n then
	    ()
	  else
	    (Array.update (a, i, i);
	     loop (i+1))
    in
      loop 0;
      a
    end

fun processArray (a, x : IntInf.int) =
    let
      val n = Array.length a
      fun loop (i, x) =
	  if i >= n then
	    x
	  else
	    loop (i+10000, x + (IntInf.fromInt (Array.sub (a, i))))
    in
      loop (0, x)
    end

fun processMem mls =
    let
      val n = length mls
      val x = IntInf.fromInt 0
      fun loop (i, x) =
	  if i >= n then
	    x
	  else
	    let
	      val a = List.nth (mls, i)
	      val x = processArray (a, x)
	    in
	      loop (i+1, x)
	    end
    in
      loop (0, IntInf.fromInt 0)
    end

fun allocUntilDeath () =
    let
      val timer = Timer.startRealTimer ()
      fun loop (mls, i) =
	  let
	    val t  = Timer.checkRealTimer timer
	    val mls = (mk10MBArray ())::mls
	  in
	    processMem mls;
	    print (Time.toString t);
	    print ": ";
	    print (Real.toString i);
	    print " GB\n";
	    loop (mls, i+0.01)
	  end
    in
      loop ([], 0.0)
    end

val () = (allocUntilDeath ())


-- 
  ----------------------------------------------------------
  David Hansel
  http://www.reactive-systems.com/
  OpenPGP (GnuPG) public key file:
  http://www.reactive-systems.com/~hansel/pgp_public_key.txt
  ----------------------------------------------------------