[MLton-devel] benchmarks with -profile {no,alloc,time}

Stephen Weeks MLton@mlton.org
Sat, 2 Nov 2002 13:58:32 -0800


Here are the results of some benchmarks I ran last night with the new
allocation profiling, as well as the existing time profiling.  They
show that the code size cost of allocation profiling is not too bad,
about 10%-20%, and that the runtime hit is also not too bad, again no
worse than 20%.  Given that, I don't think it's worth the effort to
inline the C function call that does the counter increment, until we
allready have 64 bit ops for other reasons.

There is some bug with time profiling that causes the profiled
executable to fail fairly often.  Unfortunately, this isn't an easy
one to track down, since it's not repeatable.  Different programs fail
on different runs of the benchmarks, and I haven't been able to get
any to fail except when running inside the benchmark program.  If
anyone has any ideas, I'd appreciate hearing them.


MLton0 -- mlton -profile no
MLton1 -- mlton -profile alloc
MLton2 -- mlton -profile time
compile time
benchmark         MLton0 MLton1 MLton2
barnes-hut           1.5    1.7    1.7
boyer                3.2    3.7    3.5
checksum             0.4    0.5    0.5
count-graphs         1.1    1.3    1.3
DLXSimulator         2.4    2.9    2.8
fft                  0.8    0.9    0.9
fib                  0.4    0.5    0.5
hamlet              33.6   38.7   36.2
imp-for              0.4    0.5    0.5
knuth-bendix         1.3    1.5    1.4
lexgen               3.7    4.1    4.0
life                 0.8    1.0    1.0
logic                1.9    2.1    2.1
mandelbrot           0.4    0.5    0.5
matrix-multiply      0.5    0.6    0.6
md5                  0.8    0.9    0.9
merge                0.4    0.5    0.6
mlyacc              15.4   18.9   16.6
model-elimination   14.3   17.0   16.2
mpuz                 0.5    0.7    0.7
nucleic              3.6    3.8    3.8
peek                 0.6    0.7    0.7
psdes-random         0.4    0.5    0.6
ratio-regions        1.5    1.8    1.7
ray                  2.2    2.4    2.3
raytrace             6.8    7.7    7.4
simple               4.5    5.1    4.8
smith-normal-form    4.8    5.0    5.0
tailfib              0.4    0.5    0.5
tak                  0.4    0.5    0.5
tensor               2.0    2.2    2.1
tsp                  0.9    1.0    1.1
tyan                 2.5    2.7    2.7
vector-concat        0.4    0.5    0.6
vector-rev           0.4    0.5    0.5
vliw                 8.0    9.5    8.8
wc-input1            1.0    1.2    1.2
wc-scanStream        1.1    1.2    1.2
zebra                4.3    4.8    4.8
zern                 0.7    0.8    0.8
run time
benchmark         MLton0 MLton1 MLton2
barnes-hut          30.4   31.5      *
boyer               36.8   36.7   45.7
checksum            45.6   45.5      *
count-graphs        31.0   36.9   32.2
DLXSimulator        72.7   78.5      *
fft                 40.2   40.6      *
fib                 27.2   31.1   28.2
hamlet              35.6   39.0      *
imp-for             35.0   31.9   31.8
knuth-bendix        31.3   32.2      *
lexgen              29.7   34.9   30.5
life                38.0   43.9   39.8
logic               39.5   41.5      *
mandelbrot          39.6   39.6   36.3
matrix-multiply     34.1   34.1   34.1
md5                 96.0  105.4  105.4
merge               54.9   58.4      *
mlyacc              31.8   34.1   32.6
model-elimination   46.4   50.7   47.7
mpuz                24.5   25.1   25.4
nucleic             38.6   40.4   39.9
peek                18.8   18.8   18.8
psdes-random        18.0   17.3   17.4
ratio-regions       30.7   31.5      *
ray                 16.1   16.3   15.8
raytrace            22.6   24.1      *
simple              30.6   33.1   30.9
smith-normal-form   32.3   32.1      *
tailfib             29.6   24.4   24.5
tak                 54.7   59.0   59.0
tensor              19.9   18.4   18.5
tsp                 36.0   36.2      *
tyan                37.0   40.1   37.1
vector-concat       59.7   60.6   61.4
vector-rev          74.1   74.9      *
vliw                30.4   33.7      *
wc-input1           33.4   34.8   35.3
wc-scanStream       31.1   31.2   31.2
zebra               36.6   44.6   37.8
zern                38.9   38.9      *
run time ratio
benchmark         MLton1 MLton2
barnes-hut           1.0      *
boyer                1.0    1.2
checksum             1.0      *
count-graphs         1.2    1.0
DLXSimulator         1.1      *
fft                  1.0      *
fib                  1.1    1.0
hamlet               1.1      *
imp-for              0.9    0.9
knuth-bendix         1.0      *
lexgen               1.2    1.0
life                 1.2    1.0
logic                1.0      *
mandelbrot           1.0    0.9
matrix-multiply      1.0    1.0
md5                  1.1    1.1
merge                1.1      *
mlyacc               1.1    1.0
model-elimination    1.1    1.0
mpuz                 1.0    1.0
nucleic              1.0    1.0
peek                 1.0    1.0
psdes-random         1.0    1.0
ratio-regions        1.0      *
ray                  1.0    1.0
raytrace             1.1      *
simple               1.1    1.0
smith-normal-form    1.0      *
tailfib              0.8    0.8
tak                  1.1    1.1
tensor               0.9    0.9
tsp                  1.0      *
tyan                 1.1    1.0
vector-concat        1.0    1.0
vector-rev           1.0      *
vliw                 1.1      *
wc-input1            1.0    1.1
wc-scanStream        1.0    1.0
zebra                1.2    1.0
zern                 1.0      *
size
benchmark            MLton0    MLton1    MLton2
barnes-hut          104,064   110,810   110,666
boyer               141,119   165,049   159,617
checksum             44,375    50,457    50,945
count-graphs         65,399    75,481    73,497
DLXSimulator        103,056   118,434   115,458
fft                  53,547    60,437    59,149
fib                  44,415    50,537    51,017
hamlet            1,228,776 1,456,320 1,312,840
imp-for              44,431    50,553    51,073
knuth-bendix         86,928    97,538    92,074
lexgen              173,645   196,045   184,373
life                 62,751    71,497    70,105
logic               104,423   117,801   113,529
mandelbrot           44,479    50,617    51,057
matrix-multiply      44,919    51,097    51,585
md5                  53,544    58,706    58,850
merge                45,703    52,105    52,417
mlyacc              537,453   609,565   565,661
model-elimination   636,752   743,826   677,938
mpuz                 48,551    55,545    55,705
nucleic              83,527    91,017    90,473
peek                 52,616    57,330    57,290
psdes-random         45,447    51,593    52,033
ratio-regions        63,167    78,281    77,849
ray                 104,944   115,184   109,496
raytrace            278,653   314,861   290,165
simple              200,331   233,877   218,269
smith-normal-form   182,364   188,446   188,118
tailfib              44,143    50,265    50,753
tak                  44,543    50,649    51,121
tensor              105,099   113,613   111,757
tsp                  60,128    64,898    64,914
tyan                107,984   124,754   118,882
vector-concat        44,879    51,001    51,457
vector-rev           44,767    50,873    51,361
vliw                324,457   379,529   346,633
wc-input1            66,877    74,237    72,229
wc-scanStream        67,549    75,021    72,869
zebra               143,128   176,994   165,170
zern                 51,250    56,692    56,692


-------------------------------------------------------
This sf.net email is sponsored by: See the NEW Palm 
Tungsten T handheld. Power & Color in a compact size!
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0001en
_______________________________________________
MLton-devel mailing list
MLton-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mlton-devel