Thu, 23 Aug 2001 19:29:28 -0700
I thought a bit about your scoring system, and invented a new formula :-) that
better captures the information I care about, which is comparing the good
compilers. The problem with your current formula, as I see it, is that the
performance of the bad compilers has a tremendous impact on the scoring. For
example, if a new compiler shows up that is worse than the worst compiler by a
factor of two, then everybody's score is compressed into the top half of the
5-10 range. The effect is that the scores do not distinguish much between the
To address, this, I propose the following. The score for compiler C is
10 * (best compiler CPU time / C's CPU time)
This gives a number between 0 and 10, but will compress the bad compilers
(e.g. giving all the compilers more than 10X slower than the best a score less
I would be interested to see how this formula would affect things.
My 2 cents.
Thanks for keeping up the site.