[MLton-user] compiler crash

[Stephen Weeks]

Hi Andrew.  Thanks for the bug report.

> just got an error message that's 37000 lines long and appears to show some
> kind of intermediate code representation, followed by an "unhandled
> exception: TypeError" message.

The problem you see is due to a known bug in MLton's front end that
causes it to accept some programs that should be rejected.  Your guess
as to the error message is correct -- the bug then leads to some type
incorrect code that is caught by a type checker on one of our internal
ILs.  I recently checked into our CVS a fix for this bug.  Feeding
your program to a recently built version of MLton gives the following
error message.

Error: Lexer.sml 2.20: variable type in structure disagrees with signature
   variable: lexer
   unable to generalize: 'a
   signature: ({char: int,
		line: int,
		r: 'a -> (char * 'a) option,
		s: 'a,
		src: string}
	       -> (token
		   * {char: int,
		      line: int,
		      r: 'a -> (char * 'a) option,
		      s: 'a,
		      src: string}) option)
	      -> ({char: int,
		   line: int,
		   r: 'a -> (char * 'a) option,
		   s: 'a,
		   src: string}
		  -> ((tag * token)
		      * {char: int,
			 line: int,
			 r: 'a -> (char * 'a) option,
			 s: 'a,
			 src: string}) option)

Here is a simple example of a program (which should be rejected) that
tickles this bug.

------------------------------------------------------------
structure S:
   sig
      val r: 'a option ref
   end =
   struct
      val r = ref NONE
   end
val _ = S.r := SOME 13
val _ =
   case !S.r of
      NONE => ()
    | SOME f => f ()
------------------------------------------------------------

From this, you can see that the bug is that MLton incorrectly
polymorphically generalizes a variable defined in a structure when
that variable has an unknown monotype (typically due to the value
restriction).

As to your code, the new MLton's error message indicates that
Lexer.lexer does not have as general a type as you think.  This is
likely due to the value restriction making something ungeneralizable.
Adding some type annotations to Lexer.lexer and the functions it calls
should help you track it down.

Because of the nature of the bug, hopefully you can continue with
MLton 20040227 using a few more annotations in your program.
Alternatively, if you want to get the bug fix (and be bleeding edge in
other ways), you could build MLton from the CVS.  If it is difficult
for you to build MLton from CVS, let me know and I will put up an
experimental release that contains this bug fix.