fast I/O

[Stephen Weeks]

To get a feeling for the penalty for using TextIO.scanStream I wrote the
following two simple benchmarks, which read an entire file, a character at a
time, from stdin.

(* normal *)
open TextIO
val ins = stdIn
fun loop() =
   case input1 ins of
      NONE => ()
    | SOME _ => loop()
val _ = loop()

(* scan *)
open TextIO
val _ =
   scanStream
   (fn reader =>
    let
       fun loop s =
	  case reader s of
	     NONE => SOME((), s)
	   | SOME(_, s) => loop s
    in loop
    end)
   stdIn


Here is some data.

			normal	scan
			------	-----------
running time (s)	0.94	2.51
bytes allocated		4,348	327,684,380

As you can see, you pay about a factor of 3 hit for using scanStream, and it is
mostly due to the extra allocation that goes on to represent the functional
stream.  So, my recommendation is that you avoid using scanstream if at all
possible.  You should be able to design a format that doesn't require lookahead
to tokenize and then use Int.fromString.