[MLton-user] Explosion in MLton compilation time

[Matthew Fluet]

On Thu, 18 Jun 2009, Dan DuVarney wrote:
> We've been encountering cases where small changes to a program cause
> MLton compilation times to increase by a factor of 10 or more (i.e.,
> from less than 1 hour to almost 10 hours). Some logs are inlined below
> which show that the simplifyTypes and refFlatten passes seem to be the
> culprit. In particular, during the simplifyTypes pass, the numPeeks
> count increases by a factor of nearly 30. These logs were generated with
> MLton R7016 under WinXP/MinGW using two very similar sets of source
> code. I have tried R7153 and am getting similar behavior. I can provide
> logs for R7153 once the slow compile finishes, if that will help.

I doubt that anything will have changed with r7153.

> The trigger seems to be some combination of program size and complex
> datatypes. I have been able to get the compile-time to revert to normal
> by replacing some datatypes with records. However, the subsystem
> containing the "offending" datatypes compiles normally when part of a
> smaller program. Hence the amount of source code required to reproduce
> the problem is quite massive (and also proprietary). For these reasons
> it will be difficult for me to send a source code example which
> reproduces the compilation-time increase.

In the absence of a source code example, it is very difficult to
investigate the issue.

> One question I have is: Are there any compile-time constants which might
> prevent the compile-time increase? We are already dropping the
> deepFlatten pass.

Obviously, you can '-drop-pass simplifyTypes -drop-pass refFlatten' to
omit the offending passes.

> Let me know if there is any other information I can provide.

Compiling (both the normal and the slow code) with
'-keep-pass simplifyTypes -diag-pass simplifyTypes' and comparing the 
resulting *.simplifyTypes.pre.ssa, *.simplifyTypes.post.ssa, 
*.simplifyTypes.diagnostic files might reveal something about the types 
at that stage of compilation that differ between the two programs. 
Providing the files would be useful information, but they are effectively 
dumps of the program (so you may consider them proprietary as well).