[MLton] Flatten Benchmarks
Lukasz S Ziarek
lziarek@cs.purdue.edu
Thu, 26 Feb 2004 13:00:34 -0500 (EST)
Well I ran my flattener first then regular mlton, which I realize now goes
against the convention of the benchmark suite =). However, the numbers are
still correct as are the ratios, just that big ratios are now good and
small ratios are bad =). Once I figure out the two benchmarks that just
compile forever, I will post a new set of final benchmarks and abide by
the correct conventions. In the next couple of days I hope to turn out a
new labeling scheme and see what type of performance gain it may have on
programs. Also I want to see what performance can be gained from
flattening refs of tuples, into refs of single elements. Currently
references are a "choke point" that forces a coercion, potentially
building up a very complex tuple structure. Finally I am going to write up
a formal paper based on the results that I obtain. Once it is complete, I
will post it for everyone on the list.
Luke
MLton0 -- /homes/lziarek/mlton/build/bin/mlton -flatten true
MLton1 -- /homes/lziarek/mlton/build/bin/mlton -flatten false
run time ratio
benchmark MLton0 MLton1
barnes-hut 1.00 0.89
boyer 1.00 0.86
checksum 1.00 1.00
count-graphs 1.00 1.09
DLXSimulator 1.00 1.01
fft 1.00 1.03
fib 1.00 1.00
imp-for 1.00 0.92
knuth-bendix 1.00 1.14
lexgen 1.00 1.07
life 1.00 0.93
logic 1.00 1.00
mandelbrot 1.00 0.98
matrix-multiply 1.00 1.00
md5 1.00 1.07
merge 1.00 1.00
mlyacc 1.00 1.01
mpuz 1.00 1.02
nucleic 1.00 0.27 <-- worst case
output1 1.00 0.98
peek 1.00 1.00
psdes-random 1.00 1.49 <-- best case
ratio-regions 1.00 1.03
ray 1.00 0.75
raytrace 1.00 0.78
simple 1.00 1.02
smith-normal-form 1.00 1.00
tailfib 1.00 1.00
tak 1.00 1.00
tensor 1.00 1.00
tsp 1.00 0.98
tyan 1.00 1.05
vector-concat 1.00 1.14
vector-rev 1.00 1.00
vliw 1.00 0.83
wc-input1 1.00 1.01
wc-scanStream 1.00 0.97
zebra 1.00 0.87
zern 1.00 1.00
size
benchmark MLton0 MLton1
barnes-hut 140,139 138,811
boyer 156,741 157,637
checksum 73,497 73,473
count-graphs 90,041 90,593
DLXSimulator 161,687 166,567
fft 92,050 92,162
fib 71,769 71,777
imp-for 71,705 71,713
knuth-bendix 139,395 143,075
lexgen 226,832 230,656
life 89,281 90,833
logic 137,529 130,393
mandelbrot 71,545 71,553
matrix-multiply 73,089 73,065
md5 114,131 114,395
merge 72,929 72,841
mlyacc 563,720 547,448
mpuz 76,249 76,241
nucleic 288,605 221,733
output1 118,509 119,573
peek 111,835 113,995
psdes-random 72,681 72,657
ratio-regions 95,641 97,729
ray 214,811 212,939
raytrace 314,761 312,633
simple 303,706 266,022
smith-normal-form 207,295 209,327
tailfib 71,513 71,521
tak 71,641 71,649
tensor 132,486 135,462
tsp 114,315 115,947
tyan 168,191 172,255
vector-concat 73,049 73,001
vector-rev 72,329 72,337
vliw 493,376 427,488
wc-input1 141,927 144,751
wc-scanStream 136,751 138,399
zebra 204,063 175,479
zern 128,700 128,780
compile time
benchmark MLton0 MLton1
barnes-hut 3.52 3.56
boyer 4.26 4.01
checksum 2.42 2.45
count-graphs 2.85 2.81
DLXSimulator 4.21 4.04
fft 2.68 2.70
fib 2.42 2.40
imp-for 2.42 2.43
knuth-bendix 4.78 3.27
lexgen 4.87 4.86
life 2.70 2.72
logic 3.40 3.32
mandelbrot 2.41 2.44
matrix-multiply 2.48 2.43
md5 2.83 2.80
merge 2.38 2.41
mlyacc 14.50 14.79
mpuz 2.48 2.48
nucleic 8.41 6.75
output1 2.81 2.77
peek 2.74 2.71
psdes-random 2.43 2.44
ratio-regions 3.09 3.12
ray 4.38 4.30
raytrace 7.28 6.70
simple 6.49 5.35
smith-normal-form 5.70 4.64
tailfib 2.39 2.41
tak 2.43 2.36
tensor 3.75 3.74
tsp 2.96 2.94
tyan 4.07 4.09
vector-concat 2.44 2.42
vector-rev 2.41 2.42
vliw 10.21 9.04
wc-input1 3.23 3.23
wc-scanStream 3.26 3.21
zebra 4.31 4.00
zern 2.74 2.75
run time
benchmark MLton0 MLton1
barnes-hut 23.63 21.05
boyer 19.63 16.88
checksum 67.30 67.51
count-graphs 17.40 19.03
DLXSimulator 19.27 19.39
fft 38.63 39.72
fib 32.29 32.29
imp-for 32.01 29.34
knuth-bendix 17.22 19.55
lexgen 25.53 27.30
life 8.32 7.77
logic 19.57 19.62
mandelbrot 35.84 35.23
matrix-multiply 4.86 4.86
md5 17.81 19.10
merge 17.82 17.88
mlyacc 18.04 18.18
mpuz 23.83 24.30
nucleic 75.84 20.55
output1 6.67 6.54
peek 20.95 20.95
psdes-random 23.12 34.48
ratio-regions 22.47 23.05
ray 22.10 16.51
raytrace 26.71 20.74
simple 21.09 21.51
smith-normal-form 19.59 19.52
tailfib 24.37 24.37
tak 12.32 12.31
tensor 28.37 28.35
tsp 25.13 24.69
tyan 19.59 20.64
vector-concat 28.53 32.40
vector-rev 19.15 19.07
vliw 23.16 19.28
wc-input1 21.02 21.24
wc-scanStream 15.62 15.20
zebra 30.11 26.30
zern 22.49 22.44