wordfreq slowness explained

Henry Cejtin henry@sourcelight.com
Tue, 12 Jun 2001 21:59:07 -0500


Silly  bug: the reason that wordfreq is SO slow is that the hash that you use
is completely broken.  It produces exactly 26  different  hash  values.   The
reason for that is that in the `loop' function inside it, you do not loop.

I  fixed  this,  and  the  CPU time went down from .76 CPU seconds to .13 CPU
seconds.  Assuming that this ratio  holds  on  Doug's  machine,  we  will  be
exactly the same speed as Perl, but still worse than half the speed of OCaml.

I'm going to hack a different implementation that is  less  strange  and  see
what can be done.