[MLton] WideChar

[Matthew Fluet]

> > The problem with both NONE and exceptions for 1.2 style converters is that
> > the invalidity of the input stream is not discovered until sufficient
> > input is read; i.e., not at the point where the conversion is applied.
>
> Why does this matter, since these are streams?  I.E. who can tell that
> we've looked a few characters ahead?

It's not that someone can tell.  It's that the lazy nature of streams
necessarily means that readers accumulate the exceptional effects of their
conversions.  So, you may be calling Int.scan, and get the InvalidUnicode
exception, because you passed the reader through a converter.  Unless you
scan the entire stream at the point of the conversion, every use of the
reader may result in the exception.

I'm not saying that there is anything we can do about it -- it is the
nature of a lazy stream.  I'm simply saying that one can get less than
intuitive behavior.