TextIO.scanStream is wrong

[Henry Cejtin]

Ok,  I  have  been thinking about this for a bit, and I finally realized that
something is REALLY  wrong  with  MLton  TextIO.   For  a  concrete  example,
consider the following code:

    fun getSome (n, chreader, state) =
           if n = 0
              then ([], state)
              else case chreader state of
                     NONE => ([], state)
                   | SOME (ch, state') =>
                        let val (ac, state'') = getSome (n - 1,
                                                         chreader,
                                                         state')
                        in (ch::ac, state'')
                        end
    
    fun fake chreader state =
           let val (bef, state') = getSome (10, chreader, state)
               val (after, state'') = getSome (10, chreader, state')
               val (bef', state''') = getSome (10, chreader, state)
               val bef = String.implode bef
               val after = String.implode after
               val bef' = String.implode bef'
           in print ("1st = |" ^ bef ^ "|\n");
              print ("2nd = |" ^ after ^ "|\n");
              print ("3rd = |" ^ bef' ^ "|\n");
              NONE
           end
    ;
    
    val z = TextIO.openIn "/etc/passwd"
    
    val _ = TextIO.scanStream fake z
    val _ = TextIO.scanStream fake z

If  you  run  this using SML/NJ, you get the `correct' answer (one I think is
right):

    1st = |chars 0-9 of /etc/passwd|
    2nd = |chars 10-19 of /etc/passwd|
    3rd = |chars 0-9 of /etc/passwd|
    1st = |chars 0-9 of /etc/passwd|
    2nd = |chars 10-19 of /etc/passwd|
    3rd = |chars 0-9 of /etc/passwd|

If you run it under MLton you get

    1st = |chars 0-9 of /etc/passwd|
    2nd = |chars 10-19 of /etc/passwd|
    3rd = |chars 0-9 of /etc/passwd|
    1st = |chars 20-29 of /etc/passwd|
    2nd = |chars 30-39 of /etc/passwd|
    3rd = |chars 20-29 of /etc/passwd|

Don't get me wrong: I think that the TextIO.scanStream interface is  slightly
goofy  (i.e.,  I  can't  off  hand  think of a reasonable way to make it work
fast), but what we have is clearly wrong.