setrlimit can only increase the soft limit and only up to the hard limit. MLton uses its own stack for the ML stack, but there isn't much you can do for the C stack that C code runs in. (We could make the code use stack space allocated some where else, but there is no way to make a GC happen and the stack grow if you run out.)