Fwd: [MLton-user] unicode yet?

[Matthew Fluet]

On Wed, 2 Sep 2009, Sean McLaughlin wrote:
>  1) unicode in strings
>
>     val x = "ñ"
>
>     does not compile in MLton.  I see there's a WideString type, but
>     I can't get MLton to accept "..." syntax for any Wide* structure.  Can
> you do that?
>     SML/NJ does fine with the above, by the way.

MLton supports \Uxxxxxxxx escape sequences for describing characters with 
ordinal value greater than 2^16.  (The SML Definition allows \uxxxx.)
Note that the overload resolution depends on how the value is used, not 
the constant that defines it; so, you might need a WideString.string 
constraint.

The lexer knows nothing about Unicode; it expects the source to be plain 
ASCII characters (not even UTF-8), so dropping a multibyte sequence into a 
string literal will yield a normal String.string value with some odd 
characters.

>  2) unicode identifiers
>  3) unicode constructors

You wouldn't get one without the other.  But, as noted above, the lexer 
currently doesn't handle it.