[MLton-user] Java and MLton

[Stephen Weeks]

> However something is very suspicious about SML/NJ being faster than
> Ocaml.

Yes, this is clearly wrong.  There are a number of broken SML/NJ
benchmarks that show a running time of zero, and hence get maximum
score for the benchmark.  For ary, the benchmark code doesn't export
the right heap.  Quite a few of the benchmarks are not type correct:
lists, moments, spellcheck, sumcol, strcat, wordfreq.  I'd guess there
is a problem with the benchmark harness that doesn't check if the heap
has been exported by SML/NJ.  Hopefully Brent, who reads MLton-user
and is updating the shootout, can easily fix this.  At least it should
be easy enough to disable the broken benchmarks until they are fixed.

There is also the difficult-to-fix known problem that many of the
benchmarks only run for a very short period of time, hundredths of
seconds. 

There is also the problem that the scoring method effectively divides
by the *slowest* program's running time, which clusters all of the
fast programs together.  It might be more reasonable to divide by the
median running time, or 10th best, or something.

Anyways, Brent is interested in improving the shootout, so you can
send mail directly to him with suggestions.