[MLton-devel] Fwd: Mark Tuttle

[Stephen Weeks]

> Henry, I see in then open(2) man page that there is the O_LARGEFILE
> option that we could pass to open.  Would there be any harm in adding
> that to all our open calls?

Here's what I've learned about large (>2G) files.  There appear to be
three ways to use them.

1. Compile -D_LARGEFILE64_SOURCE and pass O_LARGEFILE to open.
2. Compile -D_LARGEFILE64_SOURCE and use open64.
3. Compile -D_FILE_OFFSET_BITS=64, which replaces open with open64,
   and similarly for all other file ops.

I went ahead and checked in (1), which seemed to be the smallest
change.  Someday, we'd like to do (3), but that might introduce some
other bugs so I'd like to hold off until it can be tested more.

Now, when I run the test program that writes out 4G, I get the
following error, after successfully writing 2,147,483,648 bytes
(i.e. exactly 2G).

unhandled exception: output /tmp/z.large: Overflow

So, the O_LARGEFILE is working (we got one more byte than before), but
we are dying because MLton's IO library is using an Int32 to maintain
the file position, and that Int overflows.  I see three possible
fixes:

1. Use IntInf
2. Use Int64
3. Use Word

(1) is probably to costly performance-wise, but maybe we could try and
    see? 
(2) will take a while to implement, but is the way we should go long
    term. 
(3) would have as good a performance as we have now, and would allow
    up to 4G, but I don't think it's allowed by the Basis Library
    spec, which requires Position:> INTEGER

Thoughts?

Hmmm.  In thinking about it further, maybe (2) isn't so bad if we go
with the FFI library for Int64.
 


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
MLton-devel mailing list
MLton-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mlton-devel