[MLton] Flatten Benchmarks

Lukasz S Ziarek lziarek@cs.purdue.edu
Thu, 26 Feb 2004 13:00:34 -0500 (EST)


Well I ran my flattener first then regular mlton, which I realize now goes
against the convention of the benchmark suite =). However, the numbers are
still correct as are the ratios, just that big ratios are now good and
small ratios are bad =).  Once I figure out the two benchmarks that just
compile forever, I will post a new set of final benchmarks and abide by
the correct conventions.  In the next couple of days I hope to turn out a
new labeling scheme and see what type of performance gain it may have on
programs.  Also I want to see what performance can be gained from
flattening refs of tuples, into refs of single elements. Currently
references are a "choke point" that forces a coercion, potentially
building up a very complex tuple structure. Finally I am going to write up
a formal paper based on the results that I obtain. Once it is complete, I
will post it for everyone on the list.

Luke


MLton0 -- /homes/lziarek/mlton/build/bin/mlton -flatten true
MLton1 -- /homes/lziarek/mlton/build/bin/mlton -flatten false
run time ratio
benchmark         MLton0 MLton1
barnes-hut          1.00   0.89
boyer               1.00   0.86
checksum            1.00   1.00
count-graphs        1.00   1.09
DLXSimulator        1.00   1.01
fft                 1.00   1.03
fib                 1.00   1.00
imp-for             1.00   0.92
knuth-bendix        1.00   1.14
lexgen              1.00   1.07
life                1.00   0.93
logic               1.00   1.00
mandelbrot          1.00   0.98
matrix-multiply     1.00   1.00
md5                 1.00   1.07
merge               1.00   1.00
mlyacc              1.00   1.01
mpuz                1.00   1.02
nucleic             1.00   0.27     <-- worst case
output1             1.00   0.98
peek                1.00   1.00
psdes-random        1.00   1.49     <-- best case
ratio-regions       1.00   1.03
ray                 1.00   0.75
raytrace            1.00   0.78
simple              1.00   1.02
smith-normal-form   1.00   1.00
tailfib             1.00   1.00
tak                 1.00   1.00
tensor              1.00   1.00
tsp                 1.00   0.98
tyan                1.00   1.05
vector-concat       1.00   1.14
vector-rev          1.00   1.00
vliw                1.00   0.83
wc-input1           1.00   1.01
wc-scanStream       1.00   0.97
zebra               1.00   0.87
zern                1.00   1.00
size
benchmark          MLton0  MLton1
barnes-hut        140,139 138,811
boyer             156,741 157,637
checksum           73,497  73,473
count-graphs       90,041  90,593
DLXSimulator      161,687 166,567
fft                92,050  92,162
fib                71,769  71,777
imp-for            71,705  71,713
knuth-bendix      139,395 143,075
lexgen            226,832 230,656
life               89,281  90,833
logic             137,529 130,393
mandelbrot         71,545  71,553
matrix-multiply    73,089  73,065
md5               114,131 114,395
merge              72,929  72,841
mlyacc            563,720 547,448
mpuz               76,249  76,241
nucleic           288,605 221,733
output1           118,509 119,573
peek              111,835 113,995
psdes-random       72,681  72,657
ratio-regions      95,641  97,729
ray               214,811 212,939
raytrace          314,761 312,633
simple            303,706 266,022
smith-normal-form 207,295 209,327
tailfib            71,513  71,521
tak                71,641  71,649
tensor            132,486 135,462
tsp               114,315 115,947
tyan              168,191 172,255
vector-concat      73,049  73,001
vector-rev         72,329  72,337
vliw              493,376 427,488
wc-input1         141,927 144,751
wc-scanStream     136,751 138,399
zebra             204,063 175,479
zern              128,700 128,780
compile time
benchmark         MLton0 MLton1
barnes-hut          3.52   3.56
boyer               4.26   4.01
checksum            2.42   2.45
count-graphs        2.85   2.81
DLXSimulator        4.21   4.04
fft                 2.68   2.70
fib                 2.42   2.40
imp-for             2.42   2.43
knuth-bendix        4.78   3.27
lexgen              4.87   4.86
life                2.70   2.72
logic               3.40   3.32
mandelbrot          2.41   2.44
matrix-multiply     2.48   2.43
md5                 2.83   2.80
merge               2.38   2.41
mlyacc             14.50  14.79
mpuz                2.48   2.48
nucleic             8.41   6.75
output1             2.81   2.77
peek                2.74   2.71
psdes-random        2.43   2.44
ratio-regions       3.09   3.12
ray                 4.38   4.30
raytrace            7.28   6.70
simple              6.49   5.35
smith-normal-form   5.70   4.64
tailfib             2.39   2.41
tak                 2.43   2.36
tensor              3.75   3.74
tsp                 2.96   2.94
tyan                4.07   4.09
vector-concat       2.44   2.42
vector-rev          2.41   2.42
vliw               10.21   9.04
wc-input1           3.23   3.23
wc-scanStream       3.26   3.21
zebra               4.31   4.00
zern                2.74   2.75
run time
benchmark         MLton0 MLton1
barnes-hut         23.63  21.05
boyer              19.63  16.88
checksum           67.30  67.51
count-graphs       17.40  19.03
DLXSimulator       19.27  19.39
fft                38.63  39.72
fib                32.29  32.29
imp-for            32.01  29.34
knuth-bendix       17.22  19.55
lexgen             25.53  27.30
life                8.32   7.77
logic              19.57  19.62
mandelbrot         35.84  35.23
matrix-multiply     4.86   4.86
md5                17.81  19.10
merge              17.82  17.88
mlyacc             18.04  18.18
mpuz               23.83  24.30
nucleic            75.84  20.55
output1             6.67   6.54
peek               20.95  20.95
psdes-random       23.12  34.48
ratio-regions      22.47  23.05
ray                22.10  16.51
raytrace           26.71  20.74
simple             21.09  21.51
smith-normal-form  19.59  19.52
tailfib            24.37  24.37
tak                12.32  12.31
tensor             28.37  28.35
tsp                25.13  24.69
tyan               19.59  20.64
vector-concat      28.53  32.40
vector-rev         19.15  19.07
vliw               23.16  19.28
wc-input1          21.02  21.24
wc-scanStream      15.62  15.20
zebra              30.11  26.30
zern               22.49  22.44