[MLton] latest benchmarks

[Jesper Louis Andersen]

I've attached a benchmark run for x86-FreeBSD on my machine. An interesting
thing from the benchmark-run of the recent trunk:

testing real
666c666
< 11.59195423
---
> 11.59195518
817c817
< 11.59195423
---
> 11.59195518

I note that

http://mlton.org/pages/PortStatus/attachments/x86-freebsd.log

has the regression as well.

On 6/21/07, Matthew Fluet <fluet at tti-c.org> wrote:
>
> Jesper Louis Andersen wrote:
> > Yup, that was indeed the problem. With -codegen x86 it runs. I can add
> the
> > benchmark results besides yours in the same format in SVN if you like
> it,
> > or I can post them here.
>
> I "reset" the http://www.mlton.org/PortStatus wiki page.
> I think it makes sense to accumulate logs and benchmarks there.
>
> > It will probably take some time to run anyway.
>
> Definitely.  It's multiple hours with 8 different compilers.
>
>
> > On 6/21/07, Matthew Fluet <fluet at tti-c.org> wrote:
> >>
> >> Jesper Louis Andersen wrote:
> >> > Hmm, it failed the -codegen native as well as -codegen amd64. I'll
> >> > investigate further.
> >>
> >> For mlton-20051205 (on x86 or x86_64), -codegen native implies using
> the
> >> x86 codegen.
> >>
> >> For mlton.svn.trunk, -codegen native has been eliminated.
> >> On x86, -codegen x86 implies using the x86 codegen.
> >> On amd64, -codegen amd64 implies using the amd64 codegen.
> >>
> >> It would be easy to restore the -codegen native option on
> >> mlton.svn.trunk and choose the appropriate native codegen for the
> >> architecture.
> >>
> >> > On 6/21/07, Jesper Louis Andersen <jesper.louis.andersen at gmail.com>
> >> wrote:
> >> >>
> >> >> I have a 32bit FreeBSD compile ready. Benchmarks to trickle in
> >> tomorrow
> >> >> when I
> >> >> get it to crunch while at work (It takes a fair amount of time for
> it
> >> to
> >> >> finish and I don't
> >> >> want to mess too much with the laptop while it processes the
> >> benchmarks).
> >> >>
> >> >> There are also a few tweaks needed to run on 64-bit FreeBSD. I hope
> to
> >> be
> >> >> able to
> >> >> look into them around the 1st of July.
> >> >>
> >> >> On 6/20/07, Matthew Fluet <fluet at tti-c.org> wrote:
> >> >> >
> >> >> > I've merged the x86_64 branch into trunk.  Since the previous
> >> >> > announcement of the experimental release, there were only two
> minor
> >> >> bugs
> >> >> > reported:
> >> >> >   1) Bug with -align 8 on x86_64
> >> >> >   2) Inconsistent behavior with -const 'MLton.detectOverflowfalse'
> >> >> > These have both been fixed, and I'm pretty happy with the state of
> >> the
> >> >> > x86_64 port.
> >> >> >
> >> >> > I ran the benchmark suite to compare the last public release to
> the
> >> >> > current trunk.  It is a bit of an apples-to-oranges comparison,
> >> since
> >> I
> >> >> > ran the benchmarks on an AMD Opteron (64-bit) system.  So, the
> >> 20051205
> >> >> > compiler (and its resulting executables) are running in 32-bit
> mode,
> >> >> > while the trunk compiler (and its resulting executables) are
> running
> >> in
> >> >> > 64-bit mode.
> >> >> >
> >> >> > [BTW, it would be nice if someone could run a corresponding
> >> benchmark
> >> >> > suite on a 32-bit system, for a more apples-to-apples comparison.]
> >> >> >
> >> >> > You can see all of the results at:
> >> >> >
> >> >> >
> >> >>
> >>
> http://mlton.org/cgi-bin/viewsvn.cgi/*checkout*/mlton/trunk/doc/x86_64-port-notes/bench-20070619.txt?rev=5659
> >>
> >> >>
> >> >> >
> >> >> > Some of the highlights:
> >> >> >
> >> >> > * Benchmarks were run on a uni-core, dual-processor AMD Opteron
> >> >> 2.0GHz ,
> >> >> > 8GB Memory, Fedora Core 6 machine (with gcc version 4.1.1 and
> linux
> >> >> > version 2.6.20 (x86_64)).
> >> >> >
> >> >> > * compile time and code size is up across the board on trunk vs
> >> >> > 20051205.  I suspect that part of the code size increase can be
> >> >> > attributed to the comparison of 32-bit executables to 64-bit
> >> >> > executables.  Any 64-bit operation requires an additional 8bit
> >> >> > instruction prefix (as do 32-bit ops that touch the extended
> >> register
> >> >> > set).  Compile time is probably partly explained by the bigger
> Basis
> >> >> > Library implementation (increasing elaboration time and carrying
> >> more
> >> >> > code through early optimizations), and partly by the fact that the
> >> >> trunk
> >> >> > compiler is executing a little slower than the 20051205 compiler.
> >> >> >
> >> >> > * recent versions of gcc are doing fairly well with the C
> >> code.  (Note
> >> >> > that using -codegen c with 20051205 uses the version of gcc on the
> >> host
> >> >> > machine.)  Indeed, the flat-array.sml benchmark needs to be
> revised,
> >> as
> >> >> > gcc recognizes that the inner loop is pure (Overflow exceptions
> are
> >> >> > handled within the loop) and unused.  The SSA{,2} optimizer should
> >> also
> >> >> > discover that the loop may be optimized, but that is another
> issue.
> >> >> > GCC also does fairly well on the checksum benchmark with 20051205,
> >> >> > though it does horribly on the checksum benchmark with trunk.
> >> >> > I suspect that the later behavior is due to the fact that on
> x86_64,
> >> >> > sequences (arrays/vectors) are indexed by 64-bit integers in the
> >> >> > primitive operations (sub, update, etc), but indexed by 32-bit
> >> integers
> >> >> > in the user code (Array.sub, Array.update, etc. since Int.int
> >> >> > corresponds to Int32.int ).  Hence, there are quite a few 64/32
> >> >> > conversions going on.
> >> >> >
> >> >> > * I note that with both native codegens and C codegens, with both
> >> >> > 20051205 and trunk, that -align 8 often has a positive impact on
> >> >> > runtime, and rarely has a significant negative impact.  This
> >> might be
> >> >> > due to the Opteron memory system.  Aligned reads probably help
> most
> >> on
> >> >> > Real64 intensive benchmarks.
> >> >> >
> >> >> > * The amd64 codegen is doing alright as compared to the x86
> >> codegen.  I
> >> >> > see at most a factor of 2 slowdown, and a few speedups.  Again,
> I'm
> >> not
> >> >> > sure what real conclusions can be drawn.  Some slowdowns are going
> >> >> to be
> >> >> > due to the changes to the runtime and Basis Library since
> 20051205;
> >> to
> >> >> > isolate those, I need a comparison of 20051205 to trunk on a
> 32-bit
> >> >> > system.  Some slowdowns are probably going to be due to the
> sequence
> >> >> > indexing discussed above.
> >> >> >
> >> >> >
> >> >> >
> >> >> > _______________________________________________
> >> >> > MLton mailing list
> >> >> > MLton at mlton.org
> >> >> > http://mlton.org/mailman/listinfo/mlton
> >> >> >
> >> >> >
> >> >>
> >> >
> >> >
> >> >
> >>
> ------------------------------------------------------------------------
> >> >
> >> > _______________________________________________
> >> > MLton mailing list
> >> > MLton at mlton.org
> >> > http://mlton.org/mailman/listinfo/mlton
> >>
> >>
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > MLton mailing list
> > MLton at mlton.org
> > http://mlton.org/mailman/listinfo/mlton
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20070622/9c6754ef/attachment.html