[MLton] A few mllex patches

[Florian Weimer]

Here are a few patches for mllex.

I tried to fix the large token performance issue by introducing a
buffer with an exponential growth policy, but the code is really
brittle in this area, thanks to the way look-behind for ^ is
implemented.  As is, it's easier to tweak the input function to return
a string which definitely contains a token boundary.

-- 
Florian Weimer                <fweimer at bfk.de>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstraße 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-mllex-Fix-TeX-formatting-bug-in-documentation.patch
Type: text/x-diff
Size: 985 bytes
Desc: not available
Url : http://mlton.org/pipermail/mlton/attachments/20071002/745a620d/0001-mllex-Fix-TeX-formatting-bug-in-documentation.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-mllex-Add-r-escape-sequence-to-the-documentation.patch
Type: text/x-diff
Size: 774 bytes
Desc: not available
Url : http://mlton.org/pipermail/mlton/attachments/20071002/745a620d/0002-mllex-Add-r-escape-sequence-to-the-documentation.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0003-mllex-Add-largefile-directive-to-change-yypos-to-I.patch
Type: text/x-diff
Size: 5614 bytes
Desc: not available
Url : http://mlton.org/pipermail/mlton/attachments/20071002/745a620d/0003-mllex-Add-largefile-directive-to-change-yypos-to-I.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0004-mllex-Document-the-largefile-directive.patch
Type: text/x-diff
Size: 933 bytes
Desc: not available
Url : http://mlton.org/pipermail/mlton/attachments/20071002/745a620d/0004-mllex-Document-the-largefile-directive.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0005-mllex-Document-that-a-token-should-not-span-many-in.patch
Type: text/x-diff
Size: 1120 bytes
Desc: not available
Url : http://mlton.org/pipermail/mlton/attachments/20071002/745a620d/0005-mllex-Document-that-a-token-should-not-span-many-in.bin