[MLton] getting String.fromString to return NONE

[Andreas Rossberg]

"Stephen Weeks" <sweeks@sweeks.com>:
> > mlton 20041109 doesn't seem to want to ever return NONE from this
> > function.  Rather it seems to be returning an acceptable prefix of the
> > string I pass it.  For example
> >
> >   String.fromString "4\t"
> >
> > just returns SOME "4", and String.fromString "\n" returns SOME "".
> >
> > (As opposed to String.fromString "4\\t", which returns SOME "4\t", and
> > String.fromString "\\n", which returns SOME "\n", as you might
> > expect.)
> >
> > This return of a prefix doesn't seem called for by the Basis
> > description, and it certainly makes it difficult to tell if the input
> > is well-formed or not.

IMO, the key sentence of the spec is:

  "They only return NONE in the case where the prefix of the input cannot be
scanned at all."

So returning SOME "4" for your example is indeed correct. Whether it is
useful is another question. In my opinion, no fromString function should be
allowed to ignore trailing characters, but that's not what the spec says.
Short of that, the best option would have been what Stephen proposed.

I wouldn't be as strict as Stephen saying that you shouldn't use
String.fromString at all. But the way it is, it's surely best to only use
any of the fromString functions in the basis when you already know that the
input is well-formed.

> Michael, here are your examples, with today's implementations.
>
>                spec   Hamlet  ML Kit  MLton  Mosml            SML/NJ
>                -----  ------  ------  -----  -----            -------
> 8. "4\t"       "4"    "4"     "4\t"   "4"    "4\t"            NONE
> 9. "\n"        ""     NONE    "\n"    ""     "\n"             NONE
> 10. "4\\t"     "4\t"  "4\t"   "4\t"   "4\t"  "4\t"            "4\t"
> 11. "\\n"      "\n"   "\n"    "\n"    "\n"   "\n"             "\n"

My interpretation of the above quote is that the spec agrees with Hamlet and
NJ on example 9. Because if "prefix" were meant to include the empty one,
then the whole sentence wouldn't make any sense, and NONE would indeed never
be returned.

> and that tabs and newlines would not be allowed, since they are not
> allowed in SML strings (some implementations don't even agree on that:
> Hamlet and the ML Kit allow tabs, and Hamlet allows newlines).

Mh, looking at the above table, you probably mean Mosml, not Hamlet. :-)

- Andreas