[MLton] fix for bug in mlyacc

[Matthew Fluet]

Thanks Michael.  I committed your patch.

I guess it says something that no one before ever felt compelled to
comment their semantic actions.  I'm just not sure what the something
is...

The SML/NJ version of the tool will still have this bug; you might
push the patch to them as well.

On Wed, Jan 12, 2011 at 12:26 AM, Michael Norrish
<Michael.Norrish at nicta.com.au> wrote:
> Fix bug in comment-handling in lexer for mlyacc's input language.
>
> In particular, if a comment was successfully lexed, it returned a
> BOGUS_VALUE
> token, which then messed up the higher-level parsing.  This was due to
> the use of the idiom
>
>  continue() before YYBEGIN SOMESTATE
>
> making the return value of the continue the return value for the lexer at
> that point.
>
> This bug prevented the use of comments in code sections entirely.
>
> It was somewhat masked in the prefix section by the fact that most
> supposed comments there were never lexed as such anyway.  This could
> be observed by a file that included %% in a comment in the prefix
> section.
>
>
> diff --git a/mlyacc/src/yacc.lex b/mlyacc/src/yacc.lex
> --- a/mlyacc/src/yacc.lex
> +++ b/mlyacc/src/yacc.lex
> @@ -71,11 +71,11 @@
>  qualid ={id}".";
>  %%
>  <INITIAL>"(*"  => (Add yytext; YYBEGIN COMMENT; commentLevel := 1;
> -                   continue() before YYBEGIN INITIAL);
> +                   continue(); YYBEGIN INITIAL; continue());
>  <A>"(*"                => (YYBEGIN EMPTYCOMMENT; commentLevel := 1;
> continue());
>  <CODE>"(*"     => (Add yytext; YYBEGIN COMMENT; commentLevel := 1;
> -                   continue() before YYBEGIN CODE);
> -<INITIAL>[^%\n]+ => (Add yytext; continue());
> +                   continue(); YYBEGIN CODE; continue());
> +<INITIAL>[^(%\n]+ => (Add yytext; continue());
>  <INITIAL>"%%"   => (YYBEGIN A; HEADER (concat (rev
> (!text)),!lineno,!lineno));
>  <INITIAL,CODE,COMMENT,F,EMPTYCOMMENT>\n  => (Add yytext; inc lineno;
> continue());
>  <INITIAL>.      => (Add yytext; continue());
>
> _______________________________________________
> MLton mailing list
> MLton at mlton.org
> http://mlton.org/mailman/listinfo/mlton
>