From wesley at terpstra.ca  Fri Sep  5 01:56:24 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Fri Sep  5 01:56:27 2008
Subject: [MLton] Re: i386-darwin external _symbols
In-Reply-To: <Pine.LNX.4.64.0808301800180.21738@nagoya.uchicago.edu>
References: <162de7480808231800h766edb57q13c4d9901436bc6b@mail.gmail.com>
	<Pine.LNX.4.64.0808301800180.21738@nagoya.uchicago.edu>
Message-ID: <162de7480809050156m15709c5ekfa47beff8b546189@mail.gmail.com>

On Sun, Aug 31, 2008 at 1:08 AM, Matthew Fluet <fluet@tti-c.org> wrote:

> I've committed your patch plus a few more that gets external functions and
> symbols working on x86-darwin.


Thanks!

The above code works with the native codegen, but not with the C codegen ---
> due to the fact that 'sin' is declared as a function in <math.h> and then
> declared as an untyped extern (defaulting to int; would it be better to
> declare _address symbols as 'extern void'?) in the generated C code.


Making them void makes more sense to me. I just assumed there was a reason
it had no type information. It will still conflict for the 'sin' case,
though. I'm not really sure what the trade-offs are. Probably the only
difference would be if the target had different pointer sizes for different
types (where void must be the largest). So far every MLton target has only
one pointer type, I believe.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080905/e0d4beb7/attachment.html
From wesley at terpstra.ca  Fri Sep  5 02:41:37 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Fri Sep  5 02:41:42 2008
Subject: [MLton] symbol visibility
In-Reply-To: <Pine.LNX.4.64.0808312025410.25742@nagoya.uchicago.edu>
References: <Pine.LNX.4.64.0808312025410.25742@nagoya.uchicago.edu>
Message-ID: <162de7480809050241y376a2d39wf936621d097b3c3f@mail.gmail.com>

On Mon, Sep 1, 2008 at 5:10 AM, Matthew Fluet <fluet@tti-c.org> wrote:

> Commit r6680 adds and commit r6785 modifies the CFunction.SymbolScope.t
> datatype.  When emitting a c-prototype, the symbol scope is translated thus:
>      val symbolScope =
>         case symbolScope of
>            SymbolScope.External => "IMPORTED "
>          | SymbolScope.Private => "INTERNAL "
>          | SymbolScope.Public => "EXPORTED "
> which is then expanded by cpp via the "export.h" header.
>
> Is there a good reason to have a different names (i.e., External for
> SymbolScope.t, but "IMPORTED" for the macro)?


The IMPORTED/EXPORTED/INTERNAL macros I stole from someone else who used
those names. They nicely describe what is done operationally, IMO. When
thinking about how I wanted my ML code to look, I wanted a higher-level
point-of-view: what the compiler needs to know is where a symbol is coming
from. The SymbolScope datatype matches the ML-source-level names.

That said, it would be better to use macros of the same name as in the ML
source. I prefer the "private/public/external" labelling as it's more
declarative in nature. Also '_import "foo" imported' or '_export "foo"
exported' seems silly.

Which do you prefer?

Regarding the MLLIB_{PRIVATE/PUBLIC/whatever} macros, somehow what I'd
really like to do is define them based on if the header is used by C-code
within the library or C-code outside the library, eg:

#ifdef PART_OF_LIBRARY_<NAME-OF-LIBRARY>
#define MLLIB_PRIVATE(x) PRIVATE x
#define MLLIB_PUBLIC(x) PUBLIC x
#else
#define MLLIB_PRIVATE(x)
#define MLLIB_PUBLIC(x) EXTERNAL x
#endif

MLLIB_PRIVATE(int foo();)
MLLIB_PUBLIC(int bar();)

#undef MLLIB_PRIVATE
#undef MLLIB_PUBLIC

This would then need inclusion (or inline-copy) of exports.h.

A user using FFI as pary of his library would then do this:
#define PART_OF_LIBRARY_FOO
#include "foo.h"

Meanwhile, anyone using libfoo would just do:
#include "foo.h"

Thoughts?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080905/a45397a7/attachment.htm
From wesley at terpstra.ca  Fri Sep  5 07:55:44 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Fri Sep  5 07:55:48 2008
Subject: [MLton] mlb path maps ... and spaces
Message-ID: <162de7480809050755r498df3d4s83c002f9179bda78@mail.gmail.com>

fun parseMlbPathVar (line: String.t) =
   case String.tokens (line, Char.isSpace) of
      [var, path] => SOME {var = var, path = path}
    | _ => NONE

:-(

What's the right way to fix this, given the documentation says: "The format
of an mlb-path-map file is a sequence of lines; each line consists of two,
white-space delimited tokens. The first token is a path variable VAR and the
second token is the path to which the variable is mapped. The path may
include path variables, which are recursively expanded."

Could we change the semantics to be: cut first word up to first whitespace
=> that's the variable. Trim leading and trailing whitespace of the
remainder => that's the path? It would require changing the "specification",
but would be backwards compatible.

Leading and trailing whitespace are always supported via
CRAZY ./  my evil directory name /
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080905/a17d314f/attachment.html
From vesa.a.j.k at gmail.com  Fri Sep  5 10:05:24 2008
From: vesa.a.j.k at gmail.com (Vesa Karvonen)
Date: Fri Sep  5 10:05:58 2008
Subject: [MLton] mlb path maps ... and spaces
In-Reply-To: <162de7480809050755r498df3d4s83c002f9179bda78@mail.gmail.com>
References: <162de7480809050755r498df3d4s83c002f9179bda78@mail.gmail.com>
Message-ID: <9e43b9a0809051005v7e89e5d9o4060d3f2199adfce@mail.gmail.com>

On Fri, Sep 5, 2008 at 5:55 PM, Wesley W. Terpstra <wesley@terpstra.ca> wrote:
> Could we change the semantics to be: cut first word up to first whitespace
> => that's the variable. Trim leading and trailing whitespace of the
> remainder => that's the path? It would require changing the "specification",
> but would be backwards compatible.

Sounds reasonable to me, but don't forget to trim leading whitespace
from the variable name.

It would probably be useful to allow comments in a mlb-path-map file.

-Vesa Karvonen

From fluet at tti-c.org  Fri Sep  5 14:07:39 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Fri Sep  5 13:16:39 2008
Subject: [MLton] Re: i386-darwin external _symbols
In-Reply-To: <162de7480809050156m15709c5ekfa47beff8b546189@mail.gmail.com>
References: <162de7480808231800h766edb57q13c4d9901436bc6b@mail.gmail.com>
	<Pine.LNX.4.64.0808301800180.21738@nagoya.uchicago.edu>
	<162de7480809050156m15709c5ekfa47beff8b546189@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0809051605360.23257@nagoya.uchicago.edu>

On Fri, 5 Sep 2008, Wesley W. Terpstra wrote:
> On Sun, Aug 31, 2008 at 1:08 AM, Matthew Fluet <fluet@tti-c.org> wrote:
>> The above code works with the native codegen, but not with the C codegen ---
>> due to the fact that 'sin' is declared as a function in <math.h> and then
>> declared as an untyped extern (defaulting to int; would it be better to
>> declare _address symbols as 'extern void'?) in the generated C code.
>
> Making them void makes more sense to me. I just assumed there was a reason
> it had no type information.

Just because there was no type information from the front-end.  Using 
'void' would seem to communicate lack of definitive information.

> It will still conflict for the 'sin' case, though.

Yeah, gcc (and C standard?) complain bitterly about symbols being 
redeclared with different kinds.

From fluet at tti-c.org  Fri Sep  5 14:20:43 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Fri Sep  5 13:29:44 2008
Subject: [MLton] symbol visibility
In-Reply-To: <162de7480809050241y376a2d39wf936621d097b3c3f@mail.gmail.com>
References: <Pine.LNX.4.64.0808312025410.25742@nagoya.uchicago.edu>
	<162de7480809050241y376a2d39wf936621d097b3c3f@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0809051611540.23257@nagoya.uchicago.edu>

On Fri, 5 Sep 2008, Wesley W. Terpstra wrote:
> That said, it would be better to use macros of the same name as in the ML
> source. I prefer the "private/public/external" labelling as it's more
> declarative in nature. Also '_import "foo" imported' or '_export "foo"
> exported' seems silly.
>
> Which do you prefer?

I do think the 'private/public/external' seem a little nicer.

> Regarding the MLLIB_{PRIVATE/PUBLIC/whatever} macros, somehow what I'd
> really like to do is define them based on if the header is used by C-code
> within the library or C-code outside the library, eg:
>
> #ifdef PART_OF_LIBRARY_<NAME-OF-LIBRARY>
> #define MLLIB_PRIVATE(x) PRIVATE x
> #define MLLIB_PUBLIC(x) PUBLIC x
> #else
> #define MLLIB_PRIVATE(x)
> #define MLLIB_PUBLIC(x) EXTERNAL x
> #endif
>
> MLLIB_PRIVATE(int foo();)
> MLLIB_PUBLIC(int bar();)
>
> #undef MLLIB_PRIVATE
> #undef MLLIB_PUBLIC
>
> This would then need inclusion (or inline-copy) of exports.h.
>
> A user using FFI as pary of his library would then do this:
> #define PART_OF_LIBRARY_FOO
> #include "foo.h"
>
> Meanwhile, anyone using libfoo would just do:
> #include "foo.h"
>
> Thoughts?

Seems reasonable.  The '#define PART_OF_LIBRARY_FOO' idiom would only be 
needed if the library required a C proxy that itself required access to 
the _export-ed functions of the SML library ---  a sufficiently complex 
and rare situtation that the extra syntax seems fine.

From fluet at tti-c.org  Fri Sep  5 15:46:52 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Fri Sep  5 14:55:53 2008
Subject: [MLton] mlb path maps ... and spaces
In-Reply-To: <9e43b9a0809051005v7e89e5d9o4060d3f2199adfce@mail.gmail.com>
References: <162de7480809050755r498df3d4s83c002f9179bda78@mail.gmail.com>
	<9e43b9a0809051005v7e89e5d9o4060d3f2199adfce@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0809051735250.23257@nagoya.uchicago.edu>

On Fri, 5 Sep 2008, Vesa Karvonen wrote:
> On Fri, Sep 5, 2008 at 5:55 PM, Wesley W. Terpstra <wesley@terpstra.ca> wrote:
>> Could we change the semantics to be: cut first word up to first whitespace
>> => that's the variable. Trim leading and trailing whitespace of the
>> remainder => that's the path? It would require changing the "specification",
>> but would be backwards compatible.
>
> Sounds reasonable to me, but don't forget to trim leading whitespace
> from the variable name.

I would make the syntax for an mlb-path-map path the same as an mlb path:

   Paths may include path variables and are expanded according to a path
   map.  Unquoted paths may include alpha-numeric characters and the
   symbols "-" and "_", along with the arc separator "/" and extension
   separator ".". More complicated paths, including paths with spaces, may
   be included by quoting the path with ". A quoted path is lexed as an SML
   string constant.

That is, if we let [[:file:]] denote the epoynmous regexp from 
<src>/front-end/mlb.lex and let [[:smlstring:]] be a regexp for SML string 
constants, then an mlb-path-map entry line could be taken by the regexp:

^[[:space:]]*\([A-Z_][A-Z0-9_]*\)[[:space:]]+\([[:file:]]\|[[:smlstring:]]\)[[:space:]]$

> It would probably be useful to allow comments in a mlb-path-map file.

That might be nice.


From wesley at terpstra.ca  Sat Sep  6 01:38:46 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Sat Sep  6 01:38:50 2008
Subject: [MLton] mlb path maps ... and spaces
In-Reply-To: <Pine.LNX.4.64.0809051735250.23257@nagoya.uchicago.edu>
References: <162de7480809050755r498df3d4s83c002f9179bda78@mail.gmail.com>
	<9e43b9a0809051005v7e89e5d9o4060d3f2199adfce@mail.gmail.com>
	<Pine.LNX.4.64.0809051735250.23257@nagoya.uchicago.edu>
Message-ID: <162de7480809060138k250ff1aahad6ab4b277ac3b9b@mail.gmail.com>

On Sat, Sep 6, 2008 at 12:46 AM, Matthew Fluet <fluet@tti-c.org> wrote:

> I would make the syntax for an mlb-path-map path the same as an mlb path:
>
>  Paths may include path variables and are expanded according to a path
>  map.  Unquoted paths may include alpha-numeric characters and the
>  symbols "-" and "_", along with the arc separator "/" and extension
>  separator ".". More complicated paths, including paths with spaces, may
>  be included by quoting the path with ". A quoted path is lexed as an SML
>  string constant.
>
> That is, if we let [[:file:]] denote the epoynmous regexp from
> <src>/front-end/mlb.lex and let [[:smlstring:]] be a regexp for SML string
> constants, then an mlb-path-map entry line could be taken by the regexp:
>
>
> ^[[:space:]]*\([A-Z_][A-Z0-9_]*\)[[:space:]]+\([[:file:]]\|[[:smlstring:]]\)[[:space:]]$


This does seem better than my proposal.

 It would probably be useful to allow comments in a mlb-path-map file.
>

As normal SML-style comments then?

Of course, it turns the 3-line code there is now into yet another lexer.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080906/3cc873d7/attachment.htm
From wesley at terpstra.ca  Sat Sep  6 03:28:52 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Sat Sep  6 03:29:31 2008
Subject: [MLton] symbol visibility
In-Reply-To: <Pine.LNX.4.64.0809051611540.23257@nagoya.uchicago.edu>
References: <Pine.LNX.4.64.0808312025410.25742@nagoya.uchicago.edu>
	<162de7480809050241y376a2d39wf936621d097b3c3f@mail.gmail.com>
	<Pine.LNX.4.64.0809051611540.23257@nagoya.uchicago.edu>
Message-ID: <162de7480809060328q3adedc92g4c78b8ed7ac1233@mail.gmail.com>

On Fri, Sep 5, 2008 at 11:20 PM, Matthew Fluet <fluet@tti-c.org> wrote:

> I do think the 'private/public/external' seem a little nicer.


Committed.


>
>  Regarding the MLLIB_{PRIVATE/PUBLIC/whatever} macros, somehow what I'd
>> really like to do is define them based on if the header is used by C-code
>> within the library or C-code outside the library, eg:
>>
>> #ifdef PART_OF_LIBRARY_<NAME-OF-LIBRARY>
>> #define MLLIB_PRIVATE(x) PRIVATE x
>> #define MLLIB_PUBLIC(x) PUBLIC x
>> #else
>> #define MLLIB_PRIVATE(x)
>> #define MLLIB_PUBLIC(x) EXTERNAL x
>> #endif
>>
>> MLLIB_PRIVATE(int foo();)
>> MLLIB_PUBLIC(int bar();)
>>
>> #undef MLLIB_PRIVATE
>> #undef MLLIB_PUBLIC
>>
>> This would then need inclusion (or inline-copy) of exports.h.
>>
>> A user using FFI as pary of his library would then do this:
>> #define PART_OF_LIBRARY_FOO
>> #include "foo.h"
>>
>> Meanwhile, anyone using libfoo would just do:
>> #include "foo.h"
>>
>> Thoughts?
>>
>
> Seems reasonable.  The '#define PART_OF_LIBRARY_FOO' idiom would only be
> needed if the library required a C proxy that itself required access to the
> _export-ed functions of the SML library ---  a sufficiently complex and rare
> situtation that the extra syntax seems fine.
>

Hmm that reasoning suggests to make the #ifdef PART_OF_LIBRARY_XXX into #if
1 when Control.Format = Executable. Done.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080906/7cea044a/attachment.html
From wesley at terpstra.ca  Mon Sep  8 01:28:03 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Mon Sep  8 01:28:26 2008
Subject: [MLton] gettimeofday or clock_gettime
Message-ID: <162de7480809080128n6854b6c3j8dfe390e894ade41@mail.gmail.com>

The basis library doesn't specify the behaviour of Time.now very precisely,
but my reading of it suggests that it is supposed to work as
clock_gettime(CLOCK_REALTIME). MLton currently implements it with
gettimeofday. Why?

Also, clock_gettime(CLOCK_MONOTONIC) seems to be missing from the basis.
Perhaps it should be in a MLton extension?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080908/9aa781cc/attachment.htm
From fluet at tti-c.org  Mon Sep  8 08:14:01 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Mon Sep  8 07:23:10 2008
Subject: [MLton] gettimeofday or clock_gettime
In-Reply-To: <162de7480809080128n6854b6c3j8dfe390e894ade41@mail.gmail.com>
References: <162de7480809080128n6854b6c3j8dfe390e894ade41@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0809081009450.7743@nagoya.uchicago.edu>

On Mon, 8 Sep 2008, Wesley W. Terpstra wrote:
> The basis library doesn't specify the behaviour of Time.now very precisely,
> but my reading of it suggests that it is supposed to work as
> clock_gettime(CLOCK_REALTIME). MLton currently implements it with
> gettimeofday. Why?

Portablility.  E.g., clock_gettime isn't defined on Mac OS X 10.4.
gettimeofday seems much more widely available.


From Nicolas.Bertolotti at mathworks.fr  Mon Sep  8 07:46:54 2008
From: Nicolas.Bertolotti at mathworks.fr (Nicolas Bertolotti)
Date: Mon Sep  8 07:47:39 2008
Subject: [MLton] Crash "Vector.foldi2From" with MLton 20070826
Message-ID: <8320D98DA9A5C54C926D397795FE7CEA29F531C1A6@EXCHANGE-UK.ad.mathworks.com>

Skipped content of type multipart/related-------------- next part --------------
A non-text attachment was scrubbed...
Name: crash.log
Type: application/octet-stream
Size: 24159 bytes
Desc: crash.log
Url : http://mlton.org/pipermail/mlton/attachments/20080908/0b79dbde/crash-0001.obj
From fluet at tti-c.org  Mon Sep  8 08:54:18 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Mon Sep  8 08:03:33 2008
Subject: [MLton] Crash "Vector.foldi2From" with MLton 20070826
In-Reply-To: <8320D98DA9A5C54C926D397795FE7CEA29F531C1A6@EXCHANGE-UK.ad.mathworks.com>
References: <8320D98DA9A5C54C926D397795FE7CEA29F531C1A6@EXCHANGE-UK.ad.mathworks.com>
Message-ID: <Pine.LNX.4.64.0809081044120.7743@nagoya.uchicago.edu>

On Mon, 8 Sep 2008, Nicolas Bertolotti wrote:
> I am facing the following crash :
>                removeUnused4 starting
>                removeUnused4 raised in 0.09 + 0.00 (0% GC)
>             ssaSimplify raised in 78.67 + 14.34 (15% GC)
>      pre codegen raised in 120.05 + 29.36 (20% GC)
>   Compile SML raised in 120.05 + 29.36 (20% GC)
> MLton raised in 120.06 + 29.36 (20% GC)
> Vector.foldi2From
>
> Unfortunately, I can not identify a simple change in the code that 
> caused the crash because it appeared after we implement a complex change 
> in order to reduce the complexity of some big data types.
>
> You will find the full build log in attachment.

Logs are significantly less helpful than example programs.

> Also note that the crash is still reproducible using a compiler built from a recent checkout from the SVN trunk.
>
> Any idea is welcome.

If there is an internal compiler error, always try compiling with 
'-type-check true'.  An error like the above is usually indicative of the 
IL program being ill-formed.  For example, Vector.foldi2From is used by 
Vector.foreach, which is used by removeUnused to process matching vectors 
of actuals and formals; a mismatch in the lengths of the actuals and 
formals triggers the error in Vector.foldi2From, but the real error is 
that the IL is ill-formed: the lengths of the actuals and formals in a 
well-typed IL program should always be equal.  If the IL program is 
ill-formed, then the bug is actually an earlier pass that produced an 
ill-formed program.  Compiling with '-type-check true' should reveal which 
pass produced an ill-typed IL program.

From Nicolas.Bertolotti at mathworks.fr  Mon Sep  8 09:17:06 2008
From: Nicolas.Bertolotti at mathworks.fr (Nicolas Bertolotti)
Date: Mon Sep  8 09:17:56 2008
Subject: [MLton] Crash "Vector.foldi2From" with MLton 20070826
In-Reply-To: <Pine.LNX.4.64.0809081044120.7743@nagoya.uchicago.edu>
References: <8320D98DA9A5C54C926D397795FE7CEA29F531C1A6@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809081044120.7743@nagoya.uchicago.edu>
Message-ID: <8320D98DA9A5C54C926D397795FE7CEA29F531C206@EXCHANGE-UK.ad.mathworks.com>

> Logs are significantly less helpful than example programs.

Sorry, I understand that but 157916 lines of code is the best I have for now and unfortunately, I can not send them.

I will send an example code when I am able to reproduce the crash with a small piece of code.

> If there is an internal compiler error, always try compiling with
> '-type-check true'.  An error like the above is usually indicative of the
> IL program being ill-formed.  For example, Vector.foldi2From is used by
> Vector.foreach, which is used by removeUnused to process matching vectors
> of actuals and formals; a mismatch in the lengths of the actuals and
> formals triggers the error in Vector.foldi2From, but the real error is
> that the IL is ill-formed: the lengths of the actuals and formals in a
> well-typed IL program should always be equal.  If the IL program is
> ill-formed, then the bug is actually an earlier pass that produced an
> ill-formed program.  Compiling with '-type-check true' should reveal which
> pass produced an ill-typed IL program.

Thanks for the hint.

I relaunched with the option and here is what I have now :
"
         ssaSimplify starting
...
            redundant starting
            redundant finished in 0.53 + 0.00 (0% GC)
            post.ssa size is 46,241,204 bytes
            numPeeks = 939036180
            maxLength = 4
            average position in property list = 0.665
            numPeeks = 9202801
            average position in bucket = 1.490
            typeCheck starting
               typeCheck starting
                  checkScopes starting
                  checkScopes finished in 0.49 + 0.00 (0% GC)
Type error: analyze raised exception Analyze.coerces length mismatch: returns

               typeCheck raised in 0.68 + 0.78 (54% GC)
"

Is there any way to trace the execution of the "redundant" phase ?

Thanks in advance

Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: crash.log
Type: application/octet-stream
Size: 33968 bytes
Desc: crash.log
Url : http://mlton.org/pipermail/mlton/attachments/20080908/f9a1a6d3/crash-0001.obj
From fluet at tti-c.org  Mon Sep  8 10:18:32 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Mon Sep  8 09:28:39 2008
Subject: [MLton] Crash "Vector.foldi2From" with MLton 20070826
In-Reply-To: <8320D98DA9A5C54C926D397795FE7CEA29F531C206@EXCHANGE-UK.ad.mathworks.com>
References: <8320D98DA9A5C54C926D397795FE7CEA29F531C1A6@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809081044120.7743@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA29F531C206@EXCHANGE-UK.ad.mathworks.com>
Message-ID: <Pine.LNX.4.64.0809081213150.7743@nagoya.uchicago.edu>

On Mon, 8 Sep 2008, Nicolas Bertolotti wrote:
>> If there is an internal compiler error, always try compiling with
>> '-type-check true'.  An error like the above is usually indicative of the
>> IL program being ill-formed.  For example, Vector.foldi2From is used by
>> Vector.foreach, which is used by removeUnused to process matching vectors
>> of actuals and formals; a mismatch in the lengths of the actuals and
>> formals triggers the error in Vector.foldi2From, but the real error is
>> that the IL is ill-formed: the lengths of the actuals and formals in a
>> well-typed IL program should always be equal.  If the IL program is
>> ill-formed, then the bug is actually an earlier pass that produced an
>> ill-formed program.  Compiling with '-type-check true' should reveal which
>> pass produced an ill-typed IL program.
>
> Thanks for the hint.
>
> I relaunched with the option and here is what I have now :
> "
>         ssaSimplify starting
> ...
>            redundant starting
>            redundant finished in 0.53 + 0.00 (0% GC)
>            post.ssa size is 46,241,204 bytes
>            numPeeks = 939036180
>            maxLength = 4
>            average position in property list = 0.665
>            numPeeks = 9202801
>            average position in bucket = 1.490
>            typeCheck starting
>               typeCheck starting
>                  checkScopes starting
>                  checkScopes finished in 0.49 + 0.00 (0% GC)
> Type error: analyze raised exception Analyze.coerces length mismatch: returns
>
>               typeCheck raised in 0.68 + 0.78 (54% GC)
> "
>
> Is there any way to trace the execution of the "redundant" phase ?

Compile with '-keep-pass redundant -diag-pass redundant' will save the 
input and output IL programs (foo.redundant.pre.ssa; 
foo.redundant.post.ssa), as well as a diagnostic file 
(foo.redundant.diagostic).


From Nicolas.Bertolotti at mathworks.fr  Mon Sep  8 09:40:23 2008
From: Nicolas.Bertolotti at mathworks.fr (Nicolas Bertolotti)
Date: Mon Sep  8 09:41:00 2008
Subject: [MLton] Scaling issue during refFlatten when building the type
 dependencies graph
Message-ID: <8320D98DA9A5C54C926D397795FE7CEA29F531C218@EXCHANGE-UK.ad.mathworks.com>

Skipped content of type multipart/alternative-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 4473 bytes
Desc: image001.gif
Url : http://mlton.org/pipermail/mlton/attachments/20080908/becf96a2/image001-0001.gif
From Nicolas.Bertolotti at mathworks.fr  Mon Sep  8 10:16:32 2008
From: Nicolas.Bertolotti at mathworks.fr (Nicolas Bertolotti)
Date: Mon Sep  8 10:17:08 2008
Subject: [MLton] RE: card/cross map in heap
In-Reply-To: <Pine.LNX.4.64.0808191607020.28969@nagoya.uchicago.edu>
References: <8320D98DA9A5C54C926D397795FE7CEA2875DAC115@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0804300922520.13105@nagoya.uchicago.edu>
	<9e43b9a0804300659l406493bhbb0380d7aa2247ee@mail.gmail.com>
	<8320D98DA9A5C54C926D397795FE7CEA2875EB4D20@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0804301311550.13105@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA2875EB4D47@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0805010957310.7918@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA2875EB4E67@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0805312214510.13307@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA28767C6469@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0806240819520.20664@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA28777EAEE3@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0807161454290.13678@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA28778304D4@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0807171446110.17106@nagoya.uchicago.edu>
	<Pine.LNX.4.64.0808191607020.28969@nagoya.uchicago.edu>
Message-ID: <8320D98DA9A5C54C926D397795FE7CEA29F531C224@EXCHANGE-UK.ad.mathworks.com>

I like the idea of recording a mmapMaxHeap statistic.

Anyway, and this is the only reason why I have not already implemented it in order to give a try, I am concerned about the fact there may be some memory pressure from other processes.

In order to exploit the multi-core architectures, we would like to run multiple processes which perform parts of the verification process in parallel.

Then, I would like to be able to suspend the execution of one process (after it has dumped its heap to the disk) and wait until the other process uses less memory to continue (This is easy to do by replacing the "die" when "createHeap" fails with a sleep/retry loop).

In that case, the mmapMaxHeap statistic is not exactly the appropriate answer and using a fixed heap is also not what we want.

Also, there are 2 very different situations which can cause a "createHeap" to fail:
- not enough RAM to satisfy the allocation
- not a big enough logical address range to satisfy the allocation

In the first situation, assuming that the whole multi-process program is the only one that will really use a large amount of RAM, there is a way to determine whether or not the allocation can succeed in the future (based on the total amount of RAM and the fraction of RAM that is used by the system. We could approximate it by computing the amount of memory that is already used at the beginning of the execution of the process).

I am more concerned about the second situation. In that case, we should be able to determine (before we actually try to page to disk) that the allocation will never succeed and always choose to continue with the existing heap. On Linux, the file /proc/<pid>/maps can be used to answer that question but I could not find a way to implement such an algorithm.

> -----Original Message-----
> From: Matthew Fluet [mailto:fluet@tti-c.org]
> Sent: Wednesday, August 20, 2008 12:53 AM
> To: Nicolas Bertolotti
> Cc: mlton@mlton.org
> Subject: RE: [MLton] RE: card/cross map in heap
>
> On Thu, 17 Jul 2008, Matthew Fluet wrote:
> > The ideal solution, especially for a situation like yours, where you are
> > happy to use lots of memory on a dedicated machine, is to use
> > @MLton fixed-heap 3.5G -- to grab as large a heap as you can (that
> > comfortably fits in physical memory) at the begining of the program and
> never
> > bother resizing it.  As I understand it, resizing is only to 'play nice'
> with
> > other processes in the system.
> >
> > The problem with fixed-heap, though, is that the runtime starts off
> trying to
> > use the Cheney-copy collector (so, it really grabs 1/2 * 3.5G) and it
> may be
> > some time before it is forced to use the mark-compact collector, and it
> is
> > only at that point that the runtime will try to grab the 3.5G. Since
> > fixed-heap affects the desiredSize (but not the minSize), you really
> need to
> > set fixed-heap to the size that is actually able to be allocated, so
> that
> > desiredSize == currentSize, and no resizing occurs.
>
> Some thoughts that have occured to me.
>
> First, I remarked earlier that we would sometimes like to know "if I unmap
> this memory, can I mmap this size?"  While there is no general way of
> answering this question, I believe that (for the tight memory situations
> where it becomes an issue) we have some useful information around.  In
> particular, after paging the heap to disk, if the subsequent createHeap is
> forced to back off, then we have a reasonable upper bound on the amount of
> memory we should ever ask for in the future.  In particular, because we
> have paged the heap to disk, we know that we have freed up as much memory
> as we possible can.  If mmap can't satisfy our request in this situation,
> then we might have gone over the size of a contiguous mapping in our
> address space.  If that is the case, then there is no need to subsequently
> page the heap to disk and try to allocate a larger heap -- we've already
> got as large a heap as we can get.  (Given that, we may want createHeap to
> use a finer-grained backoff when used after paging the heap to disk; that
> would really find the largest sized heap.)
>
> Of course, the other reason mmap may fail is that the operating system
> virtual memory mangager can't currently satisfy our request along with the
> outstanding requests of other processes.  That is, mmap may fail because
> the MM won't over commit the physical/swap pages.  It is possible that a
> subsequent mmap will succeed, if in the intervening time, other processes
> have given up memory.
>
> That seems to suggest the following policy:
>
>   - record a  mmapMaxHeap  statistic.
>     This statistic is updated whenever createHeap is forced to backoff
> when
>     no extra memory is being used (that is, at the initial createHeap and
> a
>     createHeap after the heap has been paged to disk).
>
>   - take  mmapMaxHeap  into account when resizing the heap.
>     In particular, don't let the desiredSize exceed the mmapMaxHeap; it is
>     better to stick with a heap that is at mmapMaxHeap size than to try
>     paging to disk.
>     There is one exception to this policy.  If minSize > mmapMaxHeap, then
>     we should allow the runtime to page the heap to disk and try to mmap
>     the desired memory.  This helps handle the case that we saw an
>     mmapMaxHeap because of memory pressure in the system.
>
> Technically, it is possible that an inability of mmap to satisfy a minSize
> request could be a temporary situation, due to memory pressure from other
> processes.  One could always wait and try again in this situation.
> However, anyone running a program that needs >2.5G memory probably knows
> better than to run other high-memory processes at the same time and/or
> provides a decent swap file/partition.  So, I suspect that it is fairly
> safe to assume that mmap failing to satisfy a minSize request corresponds
> to a hard limit in the virtual address space for a contiguous map, and
> thus corresponds to a true out-of-memory situation.
>
>
> The second thought is that I wonder if the heap would be easier to
> predict/control if we used one contiguous heap at all times (that is, even
> for major copying collections).  In particular, it would be nice to have
> the behavior described above for fixed-heap --- namely, that one could use
> fixed-heap to grab a large block of memory at the beginning of the program
> and there would be no subsequent resizing, even if the runtime switched
> over from major copying collections to major mark-compact collections.  I
> note that the original Samson91 paper
> (http://mlton.org/References#Sansom91) works with a single contiguous
> heap.  I'm not sure that I understand the advantage of the current
> implementation, where the secondaryHeap is managed separately, potentially
> created and released a number of times during the execution of a program.

From fluet at tti-c.org  Mon Sep  8 19:12:49 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Mon Sep  8 18:22:10 2008
Subject: [MLton] Scaling issue during refFlatten when building the type
	dependencies graph
In-Reply-To: <8320D98DA9A5C54C926D397795FE7CEA29F531C218@EXCHANGE-UK.ad.mathworks.com>
References: <8320D98DA9A5C54C926D397795FE7CEA29F531C218@EXCHANGE-UK.ad.mathworks.com>
Message-ID: <Pine.LNX.4.64.0809082042040.29027@nagoya.uchicago.edu>

On Mon, 8 Sep 2008, Nicolas Bertolotti wrote:
> I recently faced a scaling issue during the refFlatten. The issue occurs 
> during the construction of a graph which uses a very large amount of RAM 
> because the edges are stored as lists and the redundant edges are not 
> automatically filtered during the construction of the graph itself.

I believe that the DirectedGraph module is meant to support graphs with 
multiple edges between nodes.  While representing the set of edges by a 
data structure other than a list might be more efficient, I don't believe 
that it would be appropriate the automatically combine edges between the 
same nodes.

> I have been able to workaround it by modifying the ref-flatten.fun file and replace the following construct:
>                      | Datatype tc =>
>                                    (ignore o Graph.addEdge)
>                                     (graph, {from = n, to = tyconNode tc})
> with:
>                      | Datatype tc =>
>                            (* NBe: Avoid redundant edges in the graph. *)
>                            let
>                                val m = tyconNode tc
>                            in
>                                if (Graph.Node.hasEdge {from = n, to = m}) then
>                                    ()
>                                else
>                                    ((ignore o Graph.addEdge)
>                                     (graph, {from = n, to = m}))
>                            end
>
> This change drastically reduced the memory usage (from millions of edges 
> to one thousand edges) and solved the problem.

Wow!  That is quite a difference (and suggests a large number of 
datatype/variants in the input program).

> I don't expect this fix to be included as is in the compiler as there 
> are probably better ways to handle this (use an other data type to store 
> the edges, use a different algorithm to build the graph ...).

Actually, I think this is the best solution to the immediate problem.  In 
any case, even with a different datatype for edges, there is no need for 
the RefFlatten graph to include duplicate edges.  It is true that the 
Node.hasEdge requires a linear search through the list of edges; that 
could be improved to constant time with another property.  But, I'm not 
sure that you would see a difference in practice; even with your large 
problem, the list of edges is now the "short" list of non-redundant edges 
and for any given node it can grow to at most the number of datatypes in 
the IL program (and will typically be much, much shorter in practice).

From vesa.a.j.k at gmail.com  Tue Sep  9 20:29:16 2008
From: vesa.a.j.k at gmail.com (Vesa Karvonen)
Date: Tue Sep  9 20:29:49 2008
Subject: [MLton] Scaling issue during refFlatten when building the type
	dependencies graph
In-Reply-To: <Pine.LNX.4.64.0809082042040.29027@nagoya.uchicago.edu>
References: <8320D98DA9A5C54C926D397795FE7CEA29F531C218@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809082042040.29027@nagoya.uchicago.edu>
Message-ID: <9e43b9a0809092029k70e1805v873705efb4d22e58@mail.gmail.com>

On Tue, Sep 9, 2008 at 5:12 AM, Matthew Fluet <fluet@tti-c.org> wrote:
> On Mon, 8 Sep 2008, Nicolas Bertolotti wrote:
[...change to ref-flatten...]
>> This change drastically reduced the memory usage (from millions of edges
>> to one thousand edges) and solved the problem.
>
> Wow!  That is quite a difference (and suggests a large number of
> datatype/variants in the input program).
[...]

This also seems to reduce space requirements when compiling mlkit with
mlton.  Without this change, compiling mlkit with ref-flatten takes a
lot of memory (8GB is not enough on a 64-bit machine).  With this
change it is possible to compile mlkit with ref-flatten (max live was
about 2.5GB (IIRC)), but it takes a very long time, about 11 hours
(IIRC), almost all of it in ref-flatten, on a C2Q machine clocked at
2GHz.  (The numbers 2.5GB and 11 hours are approximate as someone
managed to close the console window when I left the machine for 2
minutes.)

-Vesa Karvonen

From fluet at tti-c.org  Wed Sep 10 06:31:44 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Wed Sep 10 05:41:14 2008
Subject: [MLton] Scaling issue during refFlatten when building the type
	dependencies graph
In-Reply-To: <9e43b9a0809092029k70e1805v873705efb4d22e58@mail.gmail.com>
References: <8320D98DA9A5C54C926D397795FE7CEA29F531C218@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809082042040.29027@nagoya.uchicago.edu>
	<9e43b9a0809092029k70e1805v873705efb4d22e58@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0809100815150.19938@nagoya.uchicago.edu>

On Wed, 10 Sep 2008, Vesa Karvonen wrote:
> On Tue, Sep 9, 2008 at 5:12 AM, Matthew Fluet <fluet@tti-c.org> wrote:
>> On Mon, 8 Sep 2008, Nicolas Bertolotti wrote:
> [...change to ref-flatten...]
>>> This change drastically reduced the memory usage (from millions of edges
>>> to one thousand edges) and solved the problem.
>>
>> Wow!  That is quite a difference (and suggests a large number of
>> datatype/variants in the input program).
> [...]
>
> This also seems to reduce space requirements when compiling mlkit with
> mlton.  Without this change, compiling mlkit with ref-flatten takes a
> lot of memory (8GB is not enough on a 64-bit machine).  With this
> change it is possible to compile mlkit with ref-flatten (max live was
> about 2.5GB (IIRC)), but it takes a very long time, about 11 hours
> (IIRC), almost all of it in ref-flatten, on a C2Q machine clocked at
> 2GHz.  (The numbers 2.5GB and 11 hours are approximate as someone
> managed to close the console window when I left the machine for 2
> minutes.)

I don't doubt your numbers, but this might not be the whole story on the 
RefFlatten scaling issue.  The graph construction to avoid flattening into 
(mutually) recursive data-structures was added by r6395 (20080213), and, 
IIRC, the MLKit has had scaling issues with the RefFlatten and DeepFlatten 
passes from before the 20070826 release.  OTOH, by excluding (mutually) 
recursive data-structures, there are less candidates for flattening in the 
main body of the RefFlatten pass, so there is probably some help from 
that.


From vesa.a.j.k at gmail.com  Wed Sep 10 13:00:10 2008
From: vesa.a.j.k at gmail.com (Vesa Karvonen)
Date: Wed Sep 10 13:00:12 2008
Subject: [MLton] nj-mlton on amd64 linux broken?
Message-ID: <9e43b9a0809101300u49193f16y6724ab9dcabecea5@mail.gmail.com>

I've been looking at committing the FP constant folding stuff.  One of
the problems was that the code didn't compile with SML/NJ.  I've now
changed the code to also compile with SML/NJ (and the optimization is
disabled when MLton has been compiled with SML/NJ) and I'd now like to
make a sanity check that nothing broke.  However, it seems that MLton
built with SML/NJ on amd64 linux is broken.  Here is what happens when
I run bin/regression after make nj-mlton on amd64 linux from
unmodified SVN trunk:

$ ./bin/regression
MLton MLTONVERSION (built Wed Sep 10 22:44:30 2008 on caterpillar)
flags = -type-check true
testing 10
        movq (gcState+0x18)(%rip),%r15
.p2align 0x4
.globl F_0
.hidden F_0
F_0:
L_0:
        movq (gcState+0x18)(%rip),%r15
        cmpq %rbp,%r15
        setb %al
        movzbl %al,%eax
        testl %eax,%eax
        movl %eax,(globalWord32+0x0)(%rip)
        jnz L_7
L_1:
        movq (gcState+0x8)(%rip),%r14
        cmpq %r12,%r14
        setb %bl
        movzbl %bl,%ebx
        testl %ebx,%ebx
        movl %ebx,(globalWord32+0x4)(%rip)
        jnz L_25
L_2:
        movq (c_stackP+0x0)(%rip),%r15
        movq %r15,%rsp
        movq $0x11,%r15
        movq %r15,%rcx
        xorq %r15,%r15
        movq %r15,%rdx
        movq $0x18,%r15
        movq %r15,%rsi
        leaq gcState(%rip),%r15
        movq %r15,%r14
        movq %r14,%rdi
        addq $0x8,%rbp
        leaq (L_3+0x0)(%rip),%r15
        movq %r15,0xFFFFFFFFFFFFFFF8(%rbp)
        movq %r12,(gcState+0x0)(%rip)
        movq %rbp,(gcState+0x10)(%rip)
        call GC_arrayAllocate
        movq (gcState+0x0)(%rip),%r15
        movq %r15,%r12
        movq (gcState+0x10)(%rip),%r15
        movq %r15,%rbp
        jmp L_3
assertion failure: amd64CodeGen.outputChunk
compilation of 10 failed with -type-check true

(And the same seems to happen on other tests, too.)

So, am I doing something wrong here or should I just forget about
testing with SML/NJ built MLton on amd64 linux?

-Vesa Karvonen

From fluet at tti-c.org  Wed Sep 10 15:43:02 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Wed Sep 10 14:52:20 2008
Subject: [MLton] nj-mlton on amd64 linux broken?
In-Reply-To: <9e43b9a0809101300u49193f16y6724ab9dcabecea5@mail.gmail.com>
References: <9e43b9a0809101300u49193f16y6724ab9dcabecea5@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0809101651300.28827@nagoya.uchicago.edu>

On Wed, 10 Sep 2008, Vesa Karvonen wrote:
> I've been looking at committing the FP constant folding stuff.  One of
> the problems was that the code didn't compile with SML/NJ.  I've now
> changed the code to also compile with SML/NJ (and the optimization is
> disabled when MLton has been compiled with SML/NJ)

In the previous thread, I suggested looking at Real32.precision to 
determine if Real32 was really a 32-bit IEEE float or if it was bound to 
Real62 in the basis-stubs.  It occurs to me that you could also use 
MLton.isMLton (which is present via mlton-stubs even in a SML/NJ compile) 
to disable the optimization under SML/NJ.

> and I'd now like to make a sanity check that nothing broke.

Always a good thing!

> However, it seems that MLton built with SML/NJ on amd64 linux is broken.

MLton built with SML/NJ on amd64 has worked in the past.  It would be a 
good thing to get it working again.

> Here is what happens when I run bin/regression after make nj-mlton on 
> amd64 linux from unmodified SVN trunk:
>
> $ ./bin/regression
> MLton MLTONVERSION (built Wed Sep 10 22:44:30 2008 on caterpillar)
> flags = -type-check true
> testing 10
>        movq (gcState+0x18)(%rip),%r15
...
> assertion failure: amd64CodeGen.outputChunk
> compilation of 10 failed with -type-check true

An SML/NJ compiled MLton does all the Assert.assert checks (while a MLton 
compiled MLton does not), so the final assembly is being run through the 
amd64Validate module 
(<src>/mlton/codegen/amd64-codegen/amd64-validate.fun), which (in its x86 
incarnation) was meant to more informative than gas syntax errors during 
the initial development of the native codegen.  My guess is that %rip 
isn't being recognized as a valid register in this context --- indeed, it 
isn't a general purpose register, so there are lots of contexts in which 
it isn't valid, but it is o.k. as a base register.

Something like the following should help:

Index: mlton/codegen/amd64-codegen/amd64-validate.fun
===================================================================
--- mlton/codegen/amd64-codegen/amd64-validate.fun	(revision 6845)
+++ mlton/codegen/amd64-codegen/amd64-validate.fun	(working copy)
@@ -26,7 +26,9 @@
              else true

        fun validate_base {register}
-        = if not (validate {register = register}
+        = if not (eq (register, rip))
+             andalso
+             not (validate {register = register}
                    andalso
                    List.contains(baseRegisters,
                                  register,

From fluet at tti-c.org  Wed Sep 10 15:50:55 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Wed Sep 10 15:00:13 2008
Subject: [MLton] nj-mlton on amd64 linux broken?
In-Reply-To: <Pine.LNX.4.64.0809101651300.28827@nagoya.uchicago.edu>
References: <9e43b9a0809101300u49193f16y6724ab9dcabecea5@mail.gmail.com>
	<Pine.LNX.4.64.0809101651300.28827@nagoya.uchicago.edu>
Message-ID: <Pine.LNX.4.64.0809101747250.28827@nagoya.uchicago.edu>

On Wed, 10 Sep 2008, Matthew Fluet wrote:
> An SML/NJ compiled MLton does all the Assert.assert checks (while a MLton 
> compiled MLton does not), so the final assembly is being run through the 
> amd64Validate module (<src>/mlton/codegen/amd64-codegen/amd64-validate.fun), 
> which (in its x86 incarnation) was meant to more informative than gas syntax 
> errors during the initial development of the native codegen.

Another alternative is to kill off the {amd64,x86}Validate modules 
entirely.  Their utility has waned considerably and they have been a 
hindrance more often than a help when tweaking the codegens.

From Nicolas.Bertolotti at mathworks.fr  Mon Sep 15 10:44:27 2008
From: Nicolas.Bertolotti at mathworks.fr (Nicolas Bertolotti)
Date: Mon Sep 15 10:45:12 2008
Subject: [MLton] compiled binary freezes on solaris during
	LargeReal.fromLargeInt()
Message-ID: <8320D98DA9A5C54C926D397795FE7CEA29F536E061@EXCHANGE-UK.ad.mathworks.com>

Hello,

When you compile and run the following program on Solaris, using a compiler based on the latest SVN sources (r6853):
val _ = print("OK1\n");
val x = Time.toNanoseconds(Time.now());
val _ = print("OK2\n");
val y = LargeReal.fromLargeInt(x);
val _ = print("OK3\n");

It freezes (using all CPU resources) after printing the string < OK2 >.

The same piece of code runs fine when it is built using the latest official release.

Any idea?

Thanks in advance

Nicolas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080915/7c4210eb/attachment.html
From fluet at tti-c.org  Mon Sep 15 12:33:25 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Mon Sep 15 11:43:14 2008
Subject: [MLton] compiled binary freezes on solaris during
	LargeReal.fromLargeInt()
In-Reply-To: <8320D98DA9A5C54C926D397795FE7CEA29F536E061@EXCHANGE-UK.ad.mathworks.com>
References: <8320D98DA9A5C54C926D397795FE7CEA29F536E061@EXCHANGE-UK.ad.mathworks.com>
Message-ID: <Pine.LNX.4.64.0809151412000.2305@nagoya.uchicago.edu>

On Mon, 15 Sep 2008, Nicolas Bertolotti wrote:
> When you compile and run the following program on Solaris, using a compiler based on the latest SVN sources (r6853):

I assume this is on a Sparc (as opposted to an x86 or amd64).

> val _ = print("OK1\n");
> val x = Time.toNanoseconds(Time.now());
> val _ = print("OK2\n");
> val y = LargeReal.fromLargeInt(x);
> val _ = print("OK3\n");
>
> It freezes (using all CPU resources) after printing the string < OK2 >.
>
> The same piece of code runs fine when it is built using the latest official release.
>
> Any idea?

The codepath for LargeReal.fromLargeInt is mostly in the GnuMP and gdtoa 
libraries (see 'val fromIntInf' in <src>/basis-library/real/real.sml).

I would suggest compiling the program with "-debug true" and running the 
executable under gdb; a Ctrl-C when the program spins should reveal 
whether it is in GnuMP (if you see "IntInf_toString" in the backtrace) or 
in gdtoa (if you see "Real64_strto" in the backtrace).

I would also suggest building MLton 20070826 from source on this machine 
and compiling the program.  That would reveal whether there is something 
specific about this machine (version of gcc, etc.) that triggers the 
behavior or there is some change to the compiler from 20070826 to HEAD 
that triggers the behavior.

From Nicolas.Bertolotti at mathworks.fr  Mon Sep 15 13:38:32 2008
From: Nicolas.Bertolotti at mathworks.fr (Nicolas Bertolotti)
Date: Mon Sep 15 13:39:42 2008
Subject: [MLton] compiled binary freezes on solaris during
	LargeReal.fromLargeInt()
In-Reply-To: <Pine.LNX.4.64.0809151412000.2305@nagoya.uchicago.edu>
References: <8320D98DA9A5C54C926D397795FE7CEA29F536E061@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809151412000.2305@nagoya.uchicago.edu>
Message-ID: <8320D98DA9A5C54C926D397795FE7CEA29F536E076@EXCHANGE-UK.ad.mathworks.com>

Humm, I had forgotten that the other difference between my 2 compilers was the GMP version.

MLton 20070826 was compiled using gmp 4.2.1 whereas the new sources were compiled using gmp 4.2.2.

I switched back to version 4.2.1 and the example now works fine.

Thanks for your help ... it was not MLton's fault

I'm quite surprised anyway ... the bug really occurs at compilation time. After I build the binary with a runtime compiled using gmp 4.2.1, I can run the binary successfully with the gmp 4.2.2 shared library in LD_LIBRARY_PATH.

> -----Original Message-----
> From: Matthew Fluet [mailto:fluet@tti-c.org]
> Sent: Monday, September 15, 2008 9:33 PM
> To: Nicolas Bertolotti
> Cc: mlton@mlton.org
> Subject: Re: [MLton] compiled binary freezes on solaris during
> LargeReal.fromLargeInt()
>
> On Mon, 15 Sep 2008, Nicolas Bertolotti wrote:
> > When you compile and run the following program on Solaris, using a
> compiler based on the latest SVN sources (r6853):
>
> I assume this is on a Sparc (as opposted to an x86 or amd64).
>
> > val _ = print("OK1\n");
> > val x = Time.toNanoseconds(Time.now());
> > val _ = print("OK2\n");
> > val y = LargeReal.fromLargeInt(x);
> > val _ = print("OK3\n");
> >
> > It freezes (using all CPU resources) after printing the string < OK2 >.
> >
> > The same piece of code runs fine when it is built using the latest
> official release.
> >
> > Any idea?
>
> The codepath for LargeReal.fromLargeInt is mostly in the GnuMP and gdtoa
> libraries (see 'val fromIntInf' in <src>/basis-library/real/real.sml).
>
> I would suggest compiling the program with "-debug true" and running the
> executable under gdb; a Ctrl-C when the program spins should reveal
> whether it is in GnuMP (if you see "IntInf_toString" in the backtrace) or
> in gdtoa (if you see "Real64_strto" in the backtrace).
>
> I would also suggest building MLton 20070826 from source on this machine
> and compiling the program.  That would reveal whether there is something
> specific about this machine (version of gcc, etc.) that triggers the
> behavior or there is some change to the compiler from 20070826 to HEAD
> that triggers the behavior.

From fluet at tti-c.org  Mon Sep 15 14:53:06 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Mon Sep 15 14:02:58 2008
Subject: [MLton] compiled binary freezes on solaris during
	LargeReal.fromLargeInt()
In-Reply-To: <8320D98DA9A5C54C926D397795FE7CEA29F536E076@EXCHANGE-UK.ad.mathworks.com>
References: <8320D98DA9A5C54C926D397795FE7CEA29F536E061@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809151412000.2305@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA29F536E076@EXCHANGE-UK.ad.mathworks.com>
Message-ID: <Pine.LNX.4.64.0809151642560.2305@nagoya.uchicago.edu>

On Mon, 15 Sep 2008, Nicolas Bertolotti wrote:
> MLton 20070826 was compiled using gmp 4.2.1 whereas the new sources were 
> compiled using gmp 4.2.2.

I would think that gmp 4.2.1 and 4.2.2 would be binary compatible.

> I'm quite surprised anyway ... the bug really occurs at compilation 
> time.

MLton does use IntInf to represent integer and word constants and for 
constant folding, etc., so GMP will be used by the compiler at compile 
time.

> After I build the binary with a runtime compiled using gmp 4.2.1, 
> I can run the binary successfully with the gmp 4.2.2 shared library in 
> LD_LIBRARY_PATH.

Weird.

>> -----Original Message-----
>> From: Matthew Fluet [mailto:fluet@tti-c.org]
>> Sent: Monday, September 15, 2008 9:33 PM
>> To: Nicolas Bertolotti
>> Cc: mlton@mlton.org
>> Subject: Re: [MLton] compiled binary freezes on solaris during
>> LargeReal.fromLargeInt()
>>
>> On Mon, 15 Sep 2008, Nicolas Bertolotti wrote:
>>> When you compile and run the following program on Solaris, using a
>> compiler based on the latest SVN sources (r6853):
>>
>> I assume this is on a Sparc (as opposted to an x86 or amd64).
>>
>>> val _ = print("OK1\n");
>>> val x = Time.toNanoseconds(Time.now());
>>> val _ = print("OK2\n");
>>> val y = LargeReal.fromLargeInt(x);
>>> val _ = print("OK3\n");
>>>
>>> It freezes (using all CPU resources) after printing the string < OK2 >.
>>>
>>> The same piece of code runs fine when it is built using the latest
>> official release.
>>>
>>> Any idea?
>>
>> The codepath for LargeReal.fromLargeInt is mostly in the GnuMP and gdtoa
>> libraries (see 'val fromIntInf' in <src>/basis-library/real/real.sml).
>>
>> I would suggest compiling the program with "-debug true" and running the
>> executable under gdb; a Ctrl-C when the program spins should reveal
>> whether it is in GnuMP (if you see "IntInf_toString" in the backtrace) or
>> in gdtoa (if you see "Real64_strto" in the backtrace).
>>
>> I would also suggest building MLton 20070826 from source on this machine
>> and compiling the program.  That would reveal whether there is something
>> specific about this machine (version of gcc, etc.) that triggers the
>> behavior or there is some change to the compiler from 20070826 to HEAD
>> that triggers the behavior.
>
> _______________________________________________
> MLton mailing list
> MLton@mlton.org
> http://mlton.org/mailman/listinfo/mlton
>
>

From Nicolas.Bertolotti at mathworks.fr  Tue Sep 16 02:15:10 2008
From: Nicolas.Bertolotti at mathworks.fr (Nicolas Bertolotti)
Date: Tue Sep 16 02:15:53 2008
Subject: [MLton] compiled binary freezes on solaris during
	LargeReal.fromLargeInt()
In-Reply-To: <Pine.LNX.4.64.0809151642560.2305@nagoya.uchicago.edu>
References: <8320D98DA9A5C54C926D397795FE7CEA29F536E061@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809151412000.2305@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA29F536E076@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809151642560.2305@nagoya.uchicago.edu>
Message-ID: <8320D98DA9A5C54C926D397795FE7CEA29F536E0F8@EXCHANGE-UK.ad.mathworks.com>

I actually misunderstood problem and symptoms...

The issue was kind of sporadic and I finally found that the program freezes in gdtoa.

This was actually a build problem during the construction of my cross compilers. It appears that a "make clean" in the "runtime" directory does not clean the files in "gdtoa" (more precisely the file "arith.h").

Then, as I successively built cross compilers for various targets, the same arith.h file was used to build all the libgdtoa.a libraries.

I might be a good thing to extend "make clean" in order to avoid such issues.

Nicolas

> -----Original Message-----
> From: Matthew Fluet [mailto:fluet@tti-c.org]
> Sent: Monday, September 15, 2008 11:53 PM
> To: Nicolas Bertolotti
> Cc: Matthew Fluet; mlton@mlton.org
> Subject: RE: [MLton] compiled binary freezes on solaris during
> LargeReal.fromLargeInt()
>
> On Mon, 15 Sep 2008, Nicolas Bertolotti wrote:
> > MLton 20070826 was compiled using gmp 4.2.1 whereas the new sources were
> > compiled using gmp 4.2.2.
>
> I would think that gmp 4.2.1 and 4.2.2 would be binary compatible.
>
> > I'm quite surprised anyway ... the bug really occurs at compilation
> > time.
>
> MLton does use IntInf to represent integer and word constants and for
> constant folding, etc., so GMP will be used by the compiler at compile
> time.
>
> > After I build the binary with a runtime compiled using gmp 4.2.1,
> > I can run the binary successfully with the gmp 4.2.2 shared library in
> > LD_LIBRARY_PATH.
>
> Weird.
>
> >> -----Original Message-----
> >> From: Matthew Fluet [mailto:fluet@tti-c.org]
> >> Sent: Monday, September 15, 2008 9:33 PM
> >> To: Nicolas Bertolotti
> >> Cc: mlton@mlton.org
> >> Subject: Re: [MLton] compiled binary freezes on solaris during
> >> LargeReal.fromLargeInt()
> >>
> >> On Mon, 15 Sep 2008, Nicolas Bertolotti wrote:
> >>> When you compile and run the following program on Solaris, using a
> >> compiler based on the latest SVN sources (r6853):
> >>
> >> I assume this is on a Sparc (as opposted to an x86 or amd64).
> >>
> >>> val _ = print("OK1\n");
> >>> val x = Time.toNanoseconds(Time.now());
> >>> val _ = print("OK2\n");
> >>> val y = LargeReal.fromLargeInt(x);
> >>> val _ = print("OK3\n");
> >>>
> >>> It freezes (using all CPU resources) after printing the string < OK2
> >.
> >>>
> >>> The same piece of code runs fine when it is built using the latest
> >> official release.
> >>>
> >>> Any idea?
> >>
> >> The codepath for LargeReal.fromLargeInt is mostly in the GnuMP and
> gdtoa
> >> libraries (see 'val fromIntInf' in <src>/basis-library/real/real.sml).
> >>
> >> I would suggest compiling the program with "-debug true" and running
> the
> >> executable under gdb; a Ctrl-C when the program spins should reveal
> >> whether it is in GnuMP (if you see "IntInf_toString" in the backtrace)
> or
> >> in gdtoa (if you see "Real64_strto" in the backtrace).
> >>
> >> I would also suggest building MLton 20070826 from source on this
> machine
> >> and compiling the program.  That would reveal whether there is
> something
> >> specific about this machine (version of gcc, etc.) that triggers the
> >> behavior or there is some change to the compiler from 20070826 to HEAD
> >> that triggers the behavior.
> >
> > _______________________________________________
> > MLton mailing list
> > MLton@mlton.org
> > http://mlton.org/mailman/listinfo/mlton
> >
> >

From wesley at terpstra.ca  Tue Sep 16 05:59:06 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Tue Sep 16 05:59:12 2008
Subject: [MLton] compiled binary freezes on solaris during
	LargeReal.fromLargeInt()
In-Reply-To: <8320D98DA9A5C54C926D397795FE7CEA29F536E0F8@EXCHANGE-UK.ad.mathworks.com>
References: <8320D98DA9A5C54C926D397795FE7CEA29F536E061@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809151412000.2305@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA29F536E076@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809151642560.2305@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA29F536E0F8@EXCHANGE-UK.ad.mathworks.com>
Message-ID: <162de7480809160559t6a667514u92c9faee1cd41216@mail.gmail.com>

On Tue, Sep 16, 2008 at 11:15 AM, Nicolas Bertolotti <
Nicolas.Bertolotti@mathworks.fr> wrote:

> This was actually a build problem during the construction of my cross
> compilers. It appears that a "make clean" in the "runtime" directory does
> not clean the files in "gdtoa" (more precisely the file "arith.h").


Actually, it rm -rf's the entire gdtoa directory ...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080916/a8bfabee/attachment.html
From Nicolas.Bertolotti at mathworks.fr  Tue Sep 16 06:20:13 2008
From: Nicolas.Bertolotti at mathworks.fr (Nicolas Bertolotti)
Date: Tue Sep 16 06:20:54 2008
Subject: [MLton] compiled binary freezes on solaris during
	LargeReal.fromLargeInt()
In-Reply-To: <162de7480809160559t6a667514u92c9faee1cd41216@mail.gmail.com>
References: <8320D98DA9A5C54C926D397795FE7CEA29F536E061@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809151412000.2305@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA29F536E076@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809151642560.2305@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA29F536E0F8@EXCHANGE-UK.ad.mathworks.com>
	<162de7480809160559t6a667514u92c9faee1cd41216@mail.gmail.com>
Message-ID: <8320D98DA9A5C54C926D397795FE7CEA29F536E190@EXCHANGE-UK.ad.mathworks.com>

It did rm -rf the directory in MLton 20070826.

It does not with the current SVN sources (r6853).

... and I don't know why. There is no explicit < rm -rf gdtoa > call and I don't understand what used to be done exactly in the clean scripts to achieve this.

________________________________
From: Wesley W. Terpstra [mailto:wesley@terpstra.ca]
Sent: Tuesday, September 16, 2008 2:59 PM
To: Nicolas Bertolotti
Cc: Matthew Fluet; mlton@mlton.org
Subject: Re: [MLton] compiled binary freezes on solaris during LargeReal.fromLargeInt()

On Tue, Sep 16, 2008 at 11:15 AM, Nicolas Bertolotti <Nicolas.Bertolotti@mathworks.fr<mailto:Nicolas.Bertolotti@mathworks.fr>> wrote:
This was actually a build problem during the construction of my cross compilers. It appears that a "make clean" in the "runtime" directory does not clean the files in "gdtoa" (more precisely the file "arith.h").

Actually, it rm -rf's the entire gdtoa directory ...

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080916/9493535b/attachment.htm
From fluet at tti-c.org  Tue Sep 16 07:56:01 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Tue Sep 16 07:05:48 2008
Subject: [MLton] compiled binary freezes on solaris during
	LargeReal.fromLargeInt()
In-Reply-To: <8320D98DA9A5C54C926D397795FE7CEA29F536E190@EXCHANGE-UK.ad.mathworks.com>
References: <8320D98DA9A5C54C926D397795FE7CEA29F536E061@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809151412000.2305@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA29F536E076@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809151642560.2305@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA29F536E0F8@EXCHANGE-UK.ad.mathworks.com>
	<162de7480809160559t6a667514u92c9faee1cd41216@mail.gmail.com>
	<8320D98DA9A5C54C926D397795FE7CEA29F536E190@EXCHANGE-UK.ad.mathworks.com>
Message-ID: <Pine.LNX.4.64.0809160949420.28710@nagoya.uchicago.edu>

On Tue, 16 Sep 2008, Nicolas Bertolotti wrote:
> It did rm -rf the directory in MLton 20070826.
>
> It does not with the current SVN sources (r6853).

It does for me (on x86-darwin and amd64-linux).

> ... and I don't know why. There is no explicit < rm -rf gdtoa > call and I don't understand what used to be done exactly in the clean scripts to achieve this.

The <src>/bin/clean script will delete everything listed in the .ignore 
file in a directory.


> ________________________________
> From: Wesley W. Terpstra [mailto:wesley@terpstra.ca]
> Sent: Tuesday, September 16, 2008 2:59 PM
> To: Nicolas Bertolotti
> Cc: Matthew Fluet; mlton@mlton.org
> Subject: Re: [MLton] compiled binary freezes on solaris during LargeReal.fromLargeInt()
>
> On Tue, Sep 16, 2008 at 11:15 AM, Nicolas Bertolotti <Nicolas.Bertolotti@mathworks.fr<mailto:Nicolas.Bertolotti@mathworks.fr>> wrote:
> This was actually a build problem during the construction of my cross compilers. It appears that a "make clean" in the "runtime" directory does not clean the files in "gdtoa" (more precisely the file "arith.h").
>
> Actually, it rm -rf's the entire gdtoa directory ...
>
>

From Nicolas.Bertolotti at mathworks.fr  Tue Sep 16 10:54:03 2008
From: Nicolas.Bertolotti at mathworks.fr (Nicolas Bertolotti)
Date: Tue Sep 16 10:54:41 2008
Subject: [MLton] compiled binary freezes on solaris during
	LargeReal.fromLargeInt()
In-Reply-To: <Pine.LNX.4.64.0809160949420.28710@nagoya.uchicago.edu>
References: <8320D98DA9A5C54C926D397795FE7CEA29F536E061@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809151412000.2305@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA29F536E076@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809151642560.2305@nagoya.uchicago.edu>
	<8320D98DA9A5C54C926D397795FE7CEA29F536E0F8@EXCHANGE-UK.ad.mathworks.com>
	<162de7480809160559t6a667514u92c9faee1cd41216@mail.gmail.com>
	<8320D98DA9A5C54C926D397795FE7CEA29F536E190@EXCHANGE-UK.ad.mathworks.com>
	<Pine.LNX.4.64.0809160949420.28710@nagoya.uchicago.edu>
Message-ID: <8320D98DA9A5C54C926D397795FE7CEA29F536E25E@EXCHANGE-UK.ad.mathworks.com>

OK then, my fault ... I have removed all those files from my source distribution.

Thanks

> -----Original Message-----
> From: Matthew Fluet [mailto:fluet@tti-c.org]
> Sent: Tuesday, September 16, 2008 4:56 PM
> To: Nicolas Bertolotti
> Cc: Wesley W. Terpstra; Matthew Fluet; mlton@mlton.org
> Subject: RE: [MLton] compiled binary freezes on solaris during
> LargeReal.fromLargeInt()
>
> On Tue, 16 Sep 2008, Nicolas Bertolotti wrote:
> > It did rm -rf the directory in MLton 20070826.
> >
> > It does not with the current SVN sources (r6853).
>
> It does for me (on x86-darwin and amd64-linux).
>
> > ... and I don't know why. There is no explicit < rm -rf gdtoa > call and
> I don't understand what used to be done exactly in the clean scripts to
> achieve this.
>
> The <src>/bin/clean script will delete everything listed in the .ignore
> file in a directory.
>
>
> > ________________________________
> > From: Wesley W. Terpstra [mailto:wesley@terpstra.ca]
> > Sent: Tuesday, September 16, 2008 2:59 PM
> > To: Nicolas Bertolotti
> > Cc: Matthew Fluet; mlton@mlton.org
> > Subject: Re: [MLton] compiled binary freezes on solaris during
> LargeReal.fromLargeInt()
> >
> > On Tue, Sep 16, 2008 at 11:15 AM, Nicolas Bertolotti
> <Nicolas.Bertolotti@mathworks.fr<mailto:Nicolas.Bertolotti@mathworks.fr>>
> wrote:
> > This was actually a build problem during the construction of my cross
> compilers. It appears that a "make clean" in the "runtime" directory does
> not clean the files in "gdtoa" (more precisely the file "arith.h").
> >
> > Actually, it rm -rf's the entire gdtoa directory ...
> >
> >

From vesa.a.j.k at gmail.com  Tue Sep 16 21:41:49 2008
From: vesa.a.j.k at gmail.com (Vesa Karvonen)
Date: Tue Sep 16 21:41:54 2008
Subject: [MLton] Running regression tests in parallel
Message-ID: <9e43b9a0809162141o1453e208qcf067dab9768992c@mail.gmail.com>

Attached is an experimental script I cooked up the other night for
running regression tests in parallel.  It is not yet finished.  One
thing missing is code to trap signals. The changes are probably also
incompatible with some other options (like -resume).  It also probably
doesn't work on all platforms.  (I'll fix those issues.) However, on
my 4-core linux machine with -parallel 4, it cuts the time for running
regression tests to about a third (~9 minutes rather than ~27).

This is my first attempt at this kind of parallel programming with
shell scripts. :-)  Basically, I'm using lockfile for simple mutual
exclusion and condition synchronization. If you happen to know some
better/easier ways to do things in parallel in shell scripts, let me
know.  Another completely different approach might be to use make
(rewriting the regression script as a Makefile) and use make with the
-j option.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: regression
Type: application/octet-stream
Size: 10114 bytes
Desc: not available
Url : http://mlton.org/pipermail/mlton/attachments/20080917/88cb613f/regression.obj
From wesley at terpstra.ca  Mon Sep 22 03:30:23 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Mon Sep 22 03:30:39 2008
Subject: [MLton] Bytecode misses function call?
Message-ID: <162de7480809220330l2eca8c32h43c8e5975c1bba20@mail.gmail.com>

In testing library support I've run into what appears to be a bug in the
bytecode codegen. You can see this problem for yourself (only on amd64)
with:
  cd regression/library
  ./library-test -codegen bytecode -debug true -keep g

The check program will segfault. Tracing it with gdb reveals that check.sml
calls 'checkconfirmC'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080922/192ca4d0/attachment.htm
From wesley at terpstra.ca  Mon Sep 22 03:33:35 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Mon Sep 22 03:33:37 2008
Subject: [MLton] Re: Bytecode misses function call?
In-Reply-To: <162de7480809220330l2eca8c32h43c8e5975c1bba20@mail.gmail.com>
References: <162de7480809220330l2eca8c32h43c8e5975c1bba20@mail.gmail.com>
Message-ID: <162de7480809220333p1efa8d1bg41176ab1188002e2@mail.gmail.com>

Sorry for the half-written send!

On Mon, Sep 22, 2008 at 12:30 PM, Wesley W. Terpstra <wesley@terpstra.ca>wrote:

> In testing library support I've run into what appears to be a bug in the
> bytecode codegen. You can see this problem for yourself (only on amd64)
> with:
>   cd regression/library
>   ./library-test -codegen bytecode -debug true -keep g
>

The check program will segfault. Tracing it with gdb reveals that check.sml
calls 'checkconfirmC' (line 31) before it calls 'm5_open' (line 2). It seems
to me that this is an error in the bytecode codegen.

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080922/24a42f51/attachment.html
From wesley at terpstra.ca  Mon Sep 22 04:27:09 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Mon Sep 22 04:27:12 2008
Subject: [MLton] Re: Bytecode misses function call?
In-Reply-To: <162de7480809220333p1efa8d1bg41176ab1188002e2@mail.gmail.com>
References: <162de7480809220330l2eca8c32h43c8e5975c1bba20@mail.gmail.com>
	<162de7480809220333p1efa8d1bg41176ab1188002e2@mail.gmail.com>
Message-ID: <162de7480809220427k243c9680l4be402e77cef86d9@mail.gmail.com>

Skipped content of type multipart/alternative-------------- next part --------------
A non-text attachment was scrubbed...
Name: foo.sml
Type: application/smil
Size: 465 bytes
Desc: not available
Url : http://mlton.org/pipermail/mlton/attachments/20080922/136e9b17/foo.smil
-------------- next part --------------
A non-text attachment was scrubbed...
Name: foo.c
Type: text/x-csrc
Size: 264 bytes
Desc: not available
Url : http://mlton.org/pipermail/mlton/attachments/20080922/136e9b17/foo.c
From wesley at terpstra.ca  Mon Sep 22 05:30:07 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Mon Sep 22 05:30:10 2008
Subject: [MLton] Re: Bytecode misses function call?
In-Reply-To: <162de7480809220427k243c9680l4be402e77cef86d9@mail.gmail.com>
References: <162de7480809220330l2eca8c32h43c8e5975c1bba20@mail.gmail.com>
	<162de7480809220333p1efa8d1bg41176ab1188002e2@mail.gmail.com>
	<162de7480809220427k243c9680l4be402e77cef86d9@mail.gmail.com>
Message-ID: <162de7480809220530n45abd3d6s2b9e40d3a5299aa2@mail.gmail.com>

On Mon, Sep 22, 2008 at 1:27 PM, Wesley W. Terpstra <wesley@terpstra.ca>wrote:

> Attached is a minimal test case. The expected output (as produced by the C
> and native codegens):
> > m5_open called
> > libm3smlFnPublic called
> The output when compiled with the bytecode codegen:
> > libm3smlFnPublic called
> > m5_open called
> libm3smlFnPublic pointers don't match!
> > libm3smlFnPublic called
>
> Compile line:
> mlton -default-ann 'allowFFI true' -codegen bytecode foo.sml foo.c
>
> Problem appears in both svn/HEAD and 20070826.
>

Found and squashed. The problem was that the bytecodegen only looked at a
symbol name for use in its MLton_callC function. This meant it used the same
code to access the address of a symbol and also to call the symbol (if it is
a function).

To fix this, I added an additional field to the key used in the hash table
(symbol : bool) backing MLton_callC.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080922/c9528042/attachment.html
From wesley at terpstra.ca  Mon Sep 22 09:33:43 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Mon Sep 22 09:33:50 2008
Subject: [MLton] getText{Start,End} considered harmful
Message-ID: <162de7480809220933v37365594hcca72ce874159478@mail.gmail.com>

MLton currently uses two functions, GC_getText{Start,End}, to create a giant
lookup table for profiling. These two functions are among the hardest things
to supply when porting MLton. It becomes even harder once shared libraries
are considered, because then one needs to consider which text segment is
intended. After reading the i386 sys v abi specification, I still have no
idea how to do this correctly for shared libraries on elf.

It seems to me that not only are these a portability problem, they are
completely unnecessary. Making a giant lookup table for every address in the
text segment hardly seems like a worthy goal. I imagine the justification is
that it has minimal impact on the performance of the program during handling
of SIGPROF. However, I would contend that a giant lookup table could very
well be more intrusive than just using a binary search of a compressed
(repeated sourceSeqIndexes eliminated) sourceMaps.sourceLabels. Even if the
binary search does have a greater impact, at 1kHZ it will hardly matter.

Of course this leaves open the interpretation of addresses before the first
label and after the last label. As profiling labels are only used for
i386/amd64 native codegens, any address before the first label is simply
unknown. If the native codegens also emitted a label with a no-op at the end
of each assembler file, this would neatly deal with addresses past the last
label: the last label is unreachable anyway, so anything after it is
unknown.

So to sum up: could I just kill these off and use a binary search?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080922/0ff9a12d/attachment.htm
From wesley at terpstra.ca  Mon Sep 22 19:08:51 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Mon Sep 22 19:08:58 2008
Subject: [MLton] Re: getText{Start,End} considered harmful
In-Reply-To: <162de7480809220933v37365594hcca72ce874159478@mail.gmail.com>
References: <162de7480809220933v37365594hcca72ce874159478@mail.gmail.com>
Message-ID: <162de7480809221908t2eba4c5br41279c3434a492fc@mail.gmail.com>

Skipped content of type multipart/alternative-------------- next part --------------
A non-text attachment was scrubbed...
Name: no-text.patch
Type: text/x-patch
Size: 11840 bytes
Desc: not available
Url : http://mlton.org/pipermail/mlton/attachments/20080923/38b0b7f0/no-text-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mlton.mlprof.nopatch.gz
Type: application/x-gzip
Size: 18403 bytes
Desc: not available
Url : http://mlton.org/pipermail/mlton/attachments/20080923/38b0b7f0/mlton.mlprof.nopatch-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mlton.mlprof.patch.gz
Type: application/x-gzip
Size: 18287 bytes
Desc: not available
Url : http://mlton.org/pipermail/mlton/attachments/20080923/38b0b7f0/mlton.mlprof.patch-0001.bin
From ville at laurikari.net  Mon Sep 22 22:58:30 2008
From: ville at laurikari.net (Ville Laurikari)
Date: Mon Sep 22 22:58:35 2008
Subject: [MLton] getText{Start,End} considered harmful
In-Reply-To: <162de7480809220933v37365594hcca72ce874159478@mail.gmail.com>
References: <sfid-H20080922-193407-+040.73-1@spamfilter.osbf.lua>
	<162de7480809220933v37365594hcca72ce874159478@mail.gmail.com>
Message-ID: <20080923055830.GA13838@laurikari.net>

On Mon, Sep 22, 2008 at 06:33:43PM +0200, Wesley W. Terpstra wrote:
> MLton currently uses two functions, GC_getText{Start,End}, to create a giant
> lookup table for profiling. These two functions are among the hardest things
> to supply when porting MLton. It becomes even harder once shared libraries

I can vouch for the difficulty of porting these.   One can always of
course just #define HAS_TIME_PROFILING FALSE and live without
profiling...

It took me quite a while to figure out how to do it on HP-UX (it did
turn out to be quite simple, though).  It works at least for
standalone executables, but I have no idea about shared libraries.

For AIX, I didn't even try.

> So to sum up: could I just kill these off and use a binary search?

Yes, please :)

--
http://www.iki.fi/vl/

From wesley at terpstra.ca  Tue Sep 23 05:27:02 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Tue Sep 23 05:27:06 2008
Subject: [MLton] getText{Start,End} considered harmful
In-Reply-To: <20080923055830.GA13838@laurikari.net>
References: <sfid-H20080922-193407-+040.73-1@spamfilter.osbf.lua>
	<162de7480809220933v37365594hcca72ce874159478@mail.gmail.com>
	<20080923055830.GA13838@laurikari.net>
Message-ID: <162de7480809230527j2eaf2b51g9e52260e6d7c86d1@mail.gmail.com>

On Tue, Sep 23, 2008 at 7:58 AM, Ville Laurikari <ville@laurikari.net>wrote:

> It took me quite a while to figure out how to do it on HP-UX (it did
> turn out to be quite simple, though).  It works at least for
> standalone executables, but I have no idea about shared libraries.


The problem with shared libraries is that etext will be the main program's
text segment. I don't know about __text_start, but _start and _init also
will refer to the main program. The only way I know of is to find the ELF
headers and try to parse them. Of course, I don't know how to find the
loaded ELF header either. Ironically, windows is the only platform that will
currently work, as __image_base__ refers to the loaded start of the current
DSO on win32.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton/attachments/20080923/49830d44/attachment.html
From fluet at tti-c.org  Sun Sep 28 10:01:05 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Sun Sep 28 10:01:27 2008
Subject: [MLton] getText{Start,End} considered harmful
In-Reply-To: <162de7480809220933v37365594hcca72ce874159478@mail.gmail.com>
References: <162de7480809220933v37365594hcca72ce874159478@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0809281150370.30797@nagoya.uchicago.edu>

On Mon, 22 Sep 2008, Wesley W. Terpstra wrote:
> MLton currently uses two functions, GC_getText{Start,End}, to create a giant
> lookup table for profiling. These two functions are among the hardest things
> to supply when porting MLton. It becomes even harder once shared libraries
> are considered, because then one needs to consider which text segment is
> intended. After reading the i386 sys v abi specification, I still have no
> idea how to do this correctly for shared libraries on elf.

My guess is that profiling with shared libraries is a bit much to ask. 
For one, you've worked to ensure that multiple ML shared libraries can be 
used within the same executable, but what would it mean for both of them 
to be profiling libraries -- they would both want to use SIGPROF for their 
own purposes.

Nonetheless, I appreciate the GC_getText{Start,End} difficulties.
There are/were other issues with the approach to time profiling:
   http://mlton.org/pipermail/mlton/2005-November/028283.html
   http://mlton.org/pipermail/mlton-commit/2005-November/000238.html

> It seems to me that not only are these a portability problem, they are
> completely unnecessary. Making a giant lookup table for every address in the
> text segment hardly seems like a worthy goal. I imagine the justification is
> that it has minimal impact on the performance of the program during handling
> of SIGPROF. However, I would contend that a giant lookup table could very
> well be more intrusive than just using a binary search of a compressed
> (repeated sourceSeqIndexes eliminated) sourceMaps.sourceLabels. Even if the
> binary search does have a greater impact, at 1kHZ it will hardly matter.

Running some actual benchmarks (e.g., with the hamlet benchmark, as is 
reported in the above messages) provides real evidence; "I imagine..." and 
"I would contend..." are good hypotheses, but are not experiments.

> So to sum up: could I just kill these off and use a binary search?

I see that such a patch has already been committed, but it would be nice 
to report the actual runtime (and profiling results) impact of such a 
change.

From fluet at tti-c.org  Sun Sep 28 10:06:50 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Sun Sep 28 10:07:07 2008
Subject: [MLton] getText{Start,End} considered harmful
In-Reply-To: <Pine.LNX.4.64.0809281150370.30797@nagoya.uchicago.edu>
References: <162de7480809220933v37365594hcca72ce874159478@mail.gmail.com>
	<Pine.LNX.4.64.0809281150370.30797@nagoya.uchicago.edu>
Message-ID: <Pine.LNX.4.64.0809281205450.1134@nagoya.uchicago.edu>

On Sun, 28 Sep 2008, Matthew Fluet wrote:
> I see that such a patch has already been committed, but it would be nice to 
> report the actual runtime (and profiling results) impact of such a change.

Sorry, my fault for reading/responding to messages in sequential order. 
Your later results with the self-compile seem to support the argument that 
a binary search suffices.


From wesley at terpstra.ca  Sun Sep 28 14:27:58 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Sun Sep 28 14:28:03 2008
Subject: [MLton] getText{Start,End} considered harmful
In-Reply-To: <Pine.LNX.4.64.0809281150370.30797@nagoya.uchicago.edu>
References: <162de7480809220933v37365594hcca72ce874159478@mail.gmail.com>
	<Pine.LNX.4.64.0809281150370.30797@nagoya.uchicago.edu>
Message-ID: <162de7480809281427v7a3e9613gf52652030efa0031@mail.gmail.com>

On Sun, Sep 28, 2008 at 7:01 PM, Matthew Fluet <fluet@tti-c.org> wrote:
> My guess is that profiling with shared libraries is a bit much to ask. For
> one, you've worked to ensure that multiple ML shared libraries can be used
> within the same executable, but what would it mean for both of them to be
> profiling libraries -- they would both want to use SIGPROF for their own
> purposes.

Yes, this is clear. However, it seems reasonable to expect to profile
one library. The GC_getText{Start,End} were causing segfaults in this
case.

> There are/were other issues with the approach to time profiling:
>  http://mlton.org/pipermail/mlton/2005-November/028283.html
>  http://mlton.org/pipermail/mlton-commit/2005-November/000238.html

Those emails propose to use code-modification for the C codegen and
the labels for the native codegen. I didn't intend to change this.
AFAIK, the C-codegen profiling path is completely unaffected.

> Running some actual benchmarks (e.g., with the hamlet benchmark, as is
> reported in the above messages) provides real evidence; "I imagine..." and
> "I would contend..." are good hypotheses, but are not experiments.

I should probably run some more experiments, yes.

From fluet at tti-c.org  Tue Sep 30 12:15:39 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Tue Sep 30 12:16:06 2008
Subject: [MLton] getText{Start,End} considered harmful
In-Reply-To: <162de7480809281427v7a3e9613gf52652030efa0031@mail.gmail.com>
References: <162de7480809220933v37365594hcca72ce874159478@mail.gmail.com>
	<Pine.LNX.4.64.0809281150370.30797@nagoya.uchicago.edu>
	<162de7480809281427v7a3e9613gf52652030efa0031@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0809301212440.16687@nagoya.uchicago.edu>

On Sun, 28 Sep 2008, Wesley W. Terpstra wrote:
> On Sun, Sep 28, 2008 at 7:01 PM, Matthew Fluet <fluet@tti-c.org> wrote:
>> There are/were other issues with the approach to time profiling:
>>  http://mlton.org/pipermail/mlton/2005-November/028283.html
>>  http://mlton.org/pipermail/mlton-commit/2005-November/000238.html
>
> Those emails propose to use code-modification for the C codegen and
> the labels for the native codegen. I didn't intend to change this.
> AFAIK, the C-codegen profiling path is completely unaffected.

Agreed that the C-codegen profiling path is unaffected by your changes. 
But, I brought up that old thread because it lays out the arguments for 
(and against) keeping the time overhead of profiling low.  Something not 
brought up in that thread is that, while using code labels in the native 
codegens works, it does complicate the native codegens somewhat, because 
everytime a new assembly basic block is introduced, it needs to conjure up 
the right profiling information to associate with the new profiling label. 
Also, it is unlikely that we could easily use labels in a reusable codegen 
framework (e.g., MLRISC, C--, LLVM).

Just as you noted that the sourceLabels could be compressed by eliminating 
duplicates, the 'field' approach to time profiling in the C-codegen (and 
available on the native codegens) could probably be improved by avoiding 
idempotent assignments to the curSourceSeqsIndex.

Anyways, your approach is certainly expedient for the problem at hand. 
And the self-compile didn't suffer from the binary search.  So, consider 
the above just general comments.

From fluet at tti-c.org  Tue Sep 30 12:19:51 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Tue Sep 30 12:20:22 2008
Subject: [MLton] getText{Start,End} considered harmful
In-Reply-To: <20080923055830.GA13838@laurikari.net>
References: <sfid-H20080922-193407-+040.73-1@spamfilter.osbf.lua>
	<162de7480809220933v37365594hcca72ce874159478@mail.gmail.com>
	<20080923055830.GA13838@laurikari.net>
Message-ID: <Pine.LNX.4.64.0809301415570.16687@nagoya.uchicago.edu>

On Tue, 23 Sep 2008, Ville Laurikari wrote:
> On Mon, Sep 22, 2008 at 06:33:43PM +0200, Wesley W. Terpstra wrote:
>> MLton currently uses two functions, GC_getText{Start,End}, to create a giant
>> lookup table for profiling. These two functions are among the hardest things
>> to supply when porting MLton. It becomes even harder once shared libraries
>
> I can vouch for the difficulty of porting these.   One can always of
> course just #define HAS_TIME_PROFILING FALSE and live without
> profiling...

On an architecture that doesn't support native codegen (i.e., that must 
use the C-codegen), then one could probably have defined the 
GC_getText{Start,End} functions to return NULL (or to even 'die'), since 
the C-codegen uses the 'time-field' approach to time profiling (rather 
than the 'time-label' approach).

But, a moot point after Wesley's commit.

From fluet at tti-c.org  Tue Sep 30 14:17:58 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Tue Sep 30 14:18:23 2008
Subject: [MLton] Re: [MLton-commit] r6887
In-Reply-To: <E1KiJku-0005Bu-6i@mlton.org>
References: <E1KiJku-0005Bu-6i@mlton.org>
Message-ID: <Pine.LNX.4.64.0809301608210.16687@nagoya.uchicago.edu>

On Tue, 23 Sep 2008, Wesley Terpstra wrote:
> feround.c and IEEEReal.c carried duplicated fesetround code.
> The IEEEReal i386 version also works on x86_64.

This is not technically true, as it only sets the control word on the x87 
unit and not on the SSE unit.  For amd64, we use the SSE instructions 
exclusively for floating-point computation, so changing the x87 control 
word has no effect.  See
   http://sources.redhat.com/cgi-bin/cvsweb.cgi/~checkout~/libc/sysdeps/x86_64/fpu/fesetround.c?rev=1.2&cvsroot=glibc

I doubt that there is any amd64 platform that does not provide 
fe{get,set}round in libm, so I don't think it is necesary to provide 
assembly for amd64.


From fluet at tti-c.org  Tue Sep 30 14:28:13 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Tue Sep 30 14:28:38 2008
Subject: [MLton] Re: [MLton-commit] r6887
In-Reply-To: <Pine.LNX.4.64.0809301608210.16687@nagoya.uchicago.edu>
References: <E1KiJku-0005Bu-6i@mlton.org>
	<Pine.LNX.4.64.0809301608210.16687@nagoya.uchicago.edu>
Message-ID: <Pine.LNX.4.64.0809301625190.16687@nagoya.uchicago.edu>

On Tue, 30 Sep 2008, Matthew Fluet wrote:
> On Tue, 23 Sep 2008, Wesley Terpstra wrote:
>>  feround.c and IEEEReal.c carried duplicated fesetround code.
>>  The IEEEReal i386 version also works on x86_64.
>
> This is not technically true, as it only sets the control word on the x87 
> unit and not on the SSE unit.  For amd64, we use the SSE instructions 
> exclusively for floating-point computation, so changing the x87 control word 
> has no effect.  See
>   http://sources.redhat.com/cgi-bin/cvsweb.cgi/~checkout~/libc/sysdeps/x86_64/fpu/fesetround.c?rev=1.2&cvsroot=glibc
>
> I doubt that there is any amd64 platform that does not provide 
> fe{get,set}round in libm, so I don't think it is necesary to provide assembly 
> for amd64.

Hmm, I got confused looking at just the patch in the commit log.  I guess 
the SSE control word changes were added some time ago.  And, looking at 
that commit entry (r6693), it seems that MinGW on Win64 does not provide 
(working) fe{get,set}round in libm.

From fluet at tti-c.org  Tue Sep 30 14:31:13 2008
From: fluet at tti-c.org (Matthew Fluet)
Date: Tue Sep 30 14:31:39 2008
Subject: [MLton] Re: [MLton-commit] r6883
In-Reply-To: <E1Ki9NN-0007po-Dl@mlton.org>
References: <E1Ki9NN-0007po-Dl@mlton.org>
Message-ID: <Pine.LNX.4.64.0809301619380.16687@nagoya.uchicago.edu>

On Tue, 23 Sep 2008, Wesley Terpstra wrote:
> When building a library, assume -default-ann "allowFFI true".

> +      (* It doesn't make sense to have a library without FFI *)
> +      val () =
> +         case !format of
> +            Executable => ()
> +          | _ => ignore (Control.Elaborate.processDefault "allowFFI true")
> +

While I agree that it doesn't make sense to build a library that doesn't 
export any functions (and, thus, uses FFI), I don't think that the default 
annotation should be 'allowFFI true'.  Yes, it is a little more concise 
for a one-file *.sml library, but it is an abstraction violation for a 
well-designed, modular, multi-file *.mlb project.  In that setting, you 
would expect a project like:

   foo.mlb (* 'pure' SML *)
   bar.mlb (* 'pure' SML *)
   baz.sml (* 'pure' SML *)
   qux.sml (* 'pure' SML *)
   ann "allowFFI true" in
     export.sml (* export functions *)
   end

Not defaulting to "allowFFI true" ensures that no FFI is used in the 
'pure' SML portions of the library.

From wesley at terpstra.ca  Tue Sep 30 15:24:58 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Tue Sep 30 15:25:25 2008
Subject: [MLton] Re: [MLton-commit] r6887
In-Reply-To: <Pine.LNX.4.64.0809301608210.16687@nagoya.uchicago.edu>
References: <E1KiJku-0005Bu-6i@mlton.org>
	<Pine.LNX.4.64.0809301608210.16687@nagoya.uchicago.edu>
Message-ID: <162de7480809301524h68d482b1mc5d7fe60680dcc64@mail.gmail.com>

On Tue, Sep 30, 2008 at 11:17 PM, Matthew Fluet <fluet@tti-c.org> wrote:
> On Tue, 23 Sep 2008, Wesley Terpstra wrote:
>> feround.c and IEEEReal.c carried duplicated fesetround code.
>> The IEEEReal i386 version also works on x86_64.
>
> This is not technically true, as it only sets the control word on the x87
> unit and not on the SSE unit.

An earlier commit by me added the SSE instructions to update the SSE
unit at the same time (check IEEEReal.c -- I added 64 bit assembly) as
the x87 unit. At the time I didn't realize there were two copies so
only updated the one. The commit you saw merged them and removed the
old version.

> For amd64, we use the SSE instructions exclusively for floating-point computation, so changing the x87 control word has no effect.
> I doubt that there is any amd64 platform that does not provide
> fe{get,set}round in libm, so I don't think it is necesary to provide
> assembly for amd64.

Not true. I added that code because it was needed on MinGW/win64. The
"libc" there also uses x87 ops, so we need to set both since MLton
native codegen uses the other. I assume it's supposed to affect
cos/sin/etc? At least it passes the regressions as it is now.

From wesley at terpstra.ca  Tue Sep 30 15:31:07 2008
From: wesley at terpstra.ca (Wesley W. Terpstra)
Date: Tue Sep 30 15:31:16 2008
Subject: [MLton] Re: [MLton-commit] r6883
In-Reply-To: <Pine.LNX.4.64.0809301619380.16687@nagoya.uchicago.edu>
References: <E1Ki9NN-0007po-Dl@mlton.org>
	<Pine.LNX.4.64.0809301619380.16687@nagoya.uchicago.edu>
Message-ID: <162de7480809301531j7922276ag65d6076457b7b93a@mail.gmail.com>

On Tue, Sep 30, 2008 at 11:31 PM, Matthew Fluet <fluet@tti-c.org> wrote:
> On Tue, 23 Sep 2008, Wesley Terpstra wrote:
>>
>> When building a library, assume -default-ann "allowFFI true".
>
>> +      (* It doesn't make sense to have a library without FFI *)
>> +      val () =
>> +         case !format of
>> +            Executable => ()
>> +          | _ => ignore (Control.Elaborate.processDefault "allowFFI
>> true")
>> +
>
> While I agree that it doesn't make sense to build a library that doesn't
> export any functions (and, thus, uses FFI), I don't think that the default
> annotation should be 'allowFFI true'.  Yes, it is a little more concise for
> a one-file *.sml library, but it is an abstraction violation for a
> well-designed, modular, multi-file *.mlb project.  In that setting, you
> would expect a project like:
>
>  foo.mlb (* 'pure' SML *)
>  bar.mlb (* 'pure' SML *)
>  baz.sml (* 'pure' SML *)
>  qux.sml (* 'pure' SML *)
>  ann "allowFFI true" in
>    export.sml (* export functions *)
>  end
>
> Not defaulting to "allowFFI true" ensures that no FFI is used in the 'pure'
> SML portions of the library.

Fair enough. It might make sense to enable it for just sml files then,
just as MLton.* is available for sml files but not mlb files by
default. On the other hand, I don't really care that much. =)

Feel free to revert it and update regresson/library/library-test.