[MLton] fix for bug in mlyacc
Michael Norrish
Michael.Norrish at nicta.com.au
Tue Jan 11 21:26:03 PST 2011
Fix bug in comment-handling in lexer for mlyacc's input language.
In particular, if a comment was successfully lexed, it returned a BOGUS_VALUE
token, which then messed up the higher-level parsing. This was due to
the use of the idiom
continue() before YYBEGIN SOMESTATE
making the return value of the continue the return value for the lexer at
that point.
This bug prevented the use of comments in code sections entirely.
It was somewhat masked in the prefix section by the fact that most
supposed comments there were never lexed as such anyway. This could
be observed by a file that included %% in a comment in the prefix
section.
diff --git a/mlyacc/src/yacc.lex b/mlyacc/src/yacc.lex
--- a/mlyacc/src/yacc.lex
+++ b/mlyacc/src/yacc.lex
@@ -71,11 +71,11 @@
qualid ={id}".";
%%
<INITIAL>"(*" => (Add yytext; YYBEGIN COMMENT; commentLevel := 1;
- continue() before YYBEGIN INITIAL);
+ continue(); YYBEGIN INITIAL; continue());
<A>"(*" => (YYBEGIN EMPTYCOMMENT; commentLevel := 1; continue());
<CODE>"(*" => (Add yytext; YYBEGIN COMMENT; commentLevel := 1;
- continue() before YYBEGIN CODE);
-<INITIAL>[^%\n]+ => (Add yytext; continue());
+ continue(); YYBEGIN CODE; continue());
+<INITIAL>[^(%\n]+ => (Add yytext; continue());
<INITIAL>"%%" => (YYBEGIN A; HEADER (concat (rev (!text)),!lineno,!lineno));
<INITIAL,CODE,COMMENT,F,EMPTYCOMMENT>\n => (Add yytext; inc lineno; continue());
<INITIAL>. => (Add yytext; continue());
More information about the MLton
mailing list