[MLton-user] StreamIOExtra.anon.first.doit

[Matthew Fluet]

> In the "real" program, it grows to over 100MB when there's nothing
> written to the files it's watching. I think I've also solved my problem,
> because I'm keeping the stream around, then reading many EOF's which
> causes it to grow, but I still have the original stream in a list.
> 
> So in summary, if reading EOF didn't allocate anything, my program would
> be fixed, but I think can fix it without any changes to the Basis
> implementation by updating my list.

I understand the scenario you are describing.  I don't know if there is
any other means of implementing the StreamIO layer without suffering the
problem you are experiencing.  The Basis Library specification says:

   Note that even if endOfStream returns true, subsequent input operations
   may succeed if more data becomes available. A stream can have multiple
   end-of-streams interspersed with normal elements. This can happen on
   Unix, for example, if a user types control-D (#"\^D") on a terminal
   device, and then keeps typing characters; it may also occur on file
   descriptors connected to sockets.

   Multiple end-of-streams is a property of the underlying reader. Thus,
   readVec on a reader may return an empty string, then another call to
   readVec on the same reader may return a nonempty string, then a third
   call may return an empty string.

Multiple calls to inputLine when the underlying reader has no further 
input necessarily commits the stream to having an EOF.  As you note, if 
you don't keep a reference to the beginning of the stream, then the GC can 
collect the portion of the stream not reachable from the current position.

I believe that canInput is the right function to call here.  That should 
not commit the stream to an EOF (hence, no space leak), but it will let 
you determine when futher data is available, at which point you could call 
an input function (which will commit the stream to the new data).