release benchmark numbers

Stephen Weeks MLton@sourcelight.com
Fri, 6 Jul 2001 14:28:09 -0700


I just finished the benchmarks for the release.  Here are the final numbers.

compile time
benchmark         ML Kit MLton Moscow ML SML/NJ
barnes-hut           8.1   2.5       0.8    1.7
checksum               *   0.7         *    0.2
count-graphs         2.6   1.7       0.2    1.2
fft                  2.1   1.5       0.1    0.9
fib                  1.0   0.6       0.1    0.2
hamlet                 *  52.8      43.3   94.5
knuth-bendix         5.3   2.3       0.4    2.1
lexgen              10.2   5.5       0.8    5.2
life                 2.9   1.4       0.2    0.7
logic                6.9   7.3       0.4    2.0
mandelbrot           1.2   0.7       0.1    0.2
matrix-multiply      1.1   0.7       0.1    0.3
md5                    *   2.2         *    2.1
merge                1.0   0.7       0.1    0.2
mlyacc              60.2  19.5       7.4   24.0
mpuz                 1.3   0.9       0.1    0.4
nucleic             28.3   4.3       2.1    2.9
peek                 1.1   1.1       0.1    0.2
psdes-random         1.1   0.7         *    0.3
ratio-regions        4.3   3.1       0.4    2.1
ray                  3.7   3.6       0.2    1.1
raytrace               *  10.2         *    6.7
simple              14.7   7.3       0.9    4.5
smith-normal-form      *   8.0         *    3.5
tak                  1.0   0.6       0.1    0.2
tensor                 *   3.1         *    3.4
tsp                  2.7   1.8       0.3    0.8
vector-concat        1.0   0.7       0.1    0.2
vector-rev           1.1   0.7       0.1    0.2
vliw                36.5  12.4       3.0   18.2
wc-input1            1.1   1.7       0.1    0.3
wc-scanStream        1.1   1.8       0.1    0.3
zebra                2.8   5.0       0.2    0.8

run time
benchmark         ML Kit MLton Moscow ML SML/NJ
barnes-hut          19.1   5.4      66.2    4.3
checksum               *   5.0         *   17.2
count-graphs        45.2   7.1     149.5   10.3
fft                    *   9.6     253.5    9.8
fib                  4.0   4.5      20.9    4.8
hamlet                 *  10.7      97.9   15.9
knuth-bendix        40.6   8.6     105.9   19.3
lexgen              32.7  13.7      76.2   20.6
life                77.2  11.0     223.9   10.5
logic                  *  30.9     110.5   20.3
mandelbrot          39.0   8.9     294.8   12.7
matrix-multiply     71.1   6.6     245.3   21.4
md5                    *   4.9         *   17.2
merge                  *  39.6         *  428.1
mlyacc                 *  10.8      83.2   18.3
mpuz                60.6   6.9     330.2   21.0
nucleic                *   8.6     135.4    5.7
peek                26.7   4.4     139.3    9.4
psdes-random        81.3   9.2         *   21.9
ratio-regions      116.1   9.6     438.4   49.7
ray                    *   6.0      70.0    5.0
raytrace               *   6.5         *   15.8
simple              16.5   7.2      92.4    9.7
smith-normal-form      *   1.1         *  110.2
tak                 18.5  11.4      83.8   15.2
tensor                 *   8.8         *   51.1
tsp                 30.8  12.5     181.2   20.8
vector-concat       58.1   6.2     105.9   36.5
vector-rev          46.7   4.6     147.9   87.3
vliw                19.3   7.6      58.4    9.9
wc-input1              *   3.0         *   20.7
wc-scanStream          *   8.8         *   13.4
zebra               37.9   3.1      74.3   22.4

run time ratio
benchmark         ML Kit Moscow ML SML/NJ
barnes-hut           3.6      12.3    0.8
checksum               *         *    3.4
count-graphs         6.4      21.2    1.5
fft                    *      26.4    1.0
fib                  0.9       4.7    1.1
hamlet                 *       9.2    1.5
knuth-bendix         4.7      12.3    2.2
lexgen               2.4       5.6    1.5
life                 7.0      20.4    1.0
logic                  *       3.6    0.7
mandelbrot           4.4      33.0    1.4
matrix-multiply     10.8      37.3    3.3
md5                    *         *    3.5
merge                  *         *   10.8
mlyacc                 *       7.7    1.7
mpuz                 8.8      47.7    3.0
nucleic                *      15.8    0.7
peek                 6.1      31.8    2.1
psdes-random         8.8         *    2.4
ratio-regions       12.1      45.8    5.2
ray                    *      11.6    0.8
raytrace               *         *    2.4
simple               2.3      12.8    1.3
smith-normal-form      *         *  102.2
tak                  1.6       7.3    1.3
tensor                 *         *    5.8
tsp                  2.5      14.5    1.7
vector-concat        9.4      17.2    5.9
vector-rev          10.1      32.0   18.9
vliw                 2.6       7.7    1.3
wc-input1              *         *    6.9
wc-scanStream          *         *    1.5
zebra               12.2      23.9    7.2


size
benchmark          ML Kit   MLton Moscow ML    SML/NJ
barnes-hut        179,980  44,371    94,990   331,768
checksum                *  21,282         *   332,504
count-graphs      109,948  40,466    84,575   355,376
fft               107,284  35,098    84,095   332,808
fib                68,588  21,082    79,878   310,968
hamlet                  * 976,777   277,168 1,263,816
knuth-bendix      115,252  59,571    88,439   321,504
lexgen            227,444 131,778   104,883   390,136
life              100,220  37,754    83,390   305,120
logic             136,068 148,738    87,252   331,744
mandelbrot        101,612  20,914    81,341   311,992
matrix-multiply   118,348  21,834    81,879   338,632
md5                     *  34,595         *   331,792
merge              68,796  22,194    80,091   307,904
mlyacc            525,404 424,914   148,286   700,456
mpuz               89,500  26,138    82,381   320,184
nucleic           234,220  58,194   207,154   354,288
peek               77,420  29,235    81,618   311,016
psdes-random       84,540  22,226         *   313,016
ratio-regions     111,308  57,586    87,485   335,856
ray               123,356  73,969    89,860   384,072
raytrace                * 181,398         *   510,056
simple            194,364 169,426    94,397   641,056
smith-normal-form       * 142,314         *   483,400
tak                68,356  20,978    79,928   306,872
tensor                  *  53,682         *   342,048
tsp               115,732  37,555    86,140   322,552
vector-concat      77,660  21,730    80,191   317,128
vector-rev         77,860  21,874    80,073   317,128
vliw              418,076 263,922   135,386   618,576
wc-input1         144,652  41,507    86,900   311,992
wc-scanStream     145,100  44,043    87,076   313,016
zebra              85,404 109,411    83,419   310,256