From wesley at terpstra.ca Tue Oct 6 21:50:29 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Tue Oct 6 21:51:07 2009 Subject: [MLton] Re: Optimization pass: eliminate-replace In-Reply-To: <162de7480909251326i53f09027n9f9989a7bfbcff10@mail.gmail.com> References: <162de7480909251326i53f09027n9f9989a7bfbcff10@mail.gmail.com> Message-ID: <162de7480910062150h3e0b2609o3584ed0a8758fc47@mail.gmail.com> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: duplicate-store.patch Type: text/x-patch Size: 8840 bytes Desc: not available Url : http://mlton.org/pipermail/mlton/attachments/20091007/40aaae52/duplicate-store.bin From henry.cejtin at sbcglobal.net Thu Oct 8 17:23:29 2009 From: henry.cejtin at sbcglobal.net (Henry Cejtin) Date: Thu Oct 8 17:24:02 2009 Subject: [MLton] latest MLton segfault in gmp Message-ID: <509955.22208.qm@web82404.mail.mud.yahoo.com> Running under Debian testing, I grabbed the latest MLton from SVN and compiled it (using the last official MLton: 20070826) on an AMD-64 machine. The make couldn't finish because the resulting mlton-compile segfaults compiling mllex.mlb. In fact, that mlton-compile always segfaults, even compiling a hello-world.sml. The gdb traceback shows that it died in __gmpz_mul_2exp(). The version of gmp that I have is 2:4.3.1+dfsg-3. Any ideas or suggestions? From mtf at cs.rit.edu Thu Oct 8 19:21:29 2009 From: mtf at cs.rit.edu (Matthew Fluet) Date: Thu Oct 8 19:21:35 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: <509955.22208.qm@web82404.mail.mud.yahoo.com> References: <509955.22208.qm@web82404.mail.mud.yahoo.com> Message-ID: On Thu, 8 Oct 2009, Henry Cejtin wrote: > Running under Debian testing, I grabbed the latest MLton from SVN and > compiled it (using the last official MLton: 20070826) on an AMD-64 > machine. > > The make couldn't finish because the resulting mlton-compile segfaults > compiling mllex.mlb. In fact, that mlton-compile always segfaults, even > compiling a hello-world.sml. The gdb traceback shows that it died in > __gmpz_mul_2exp(). > > The version of gmp that I have is 2:4.3.1+dfsg-3. > > Any ideas or suggestions? Can you run through the regression suite with mlton-20070826? That would determine if it is a pervasive problem with gmp on all programs with IntInf. There was a bug with the bytes-needed calculation for IntInf.~>>, identified and fixed by Wesley in r7083,r7084,r7085: --- a/basis-library/integer/int-inf0.sml +++ b/basis-library/integer/int-inf0.sml @@ -1203,7 +1203,7 @@ structure IntInf = if shift = 0wx0 then arg else Prim.~>> (arg, shift, - reserve (S.max (1, S.- (numLimbs arg, shiftSize shift)), 0)) + reserve (S.max (0, S.- (numLimbs arg, shiftSize shift)), 1)) end fun mkBigCvt {base: Int32.int, Perhaps there is a similar bug for IntInf.<<, though it looks o.k. to me. From wesley at terpstra.ca Fri Oct 9 07:56:18 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Fri Oct 9 07:56:55 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: References: <509955.22208.qm@web82404.mail.mud.yahoo.com> Message-ID: <162de7480910090756g695634c0u3ac445966fbb7176@mail.gmail.com> On Fri, Oct 9, 2009 at 4:21 AM, Matthew Fluet wrote: > On Thu, 8 Oct 2009, Henry Cejtin wrote: > >> The make couldn't finish because the resulting mlton-compile segfaults >> compiling mllex.mlb. In fact, that mlton-compile always segfaults, even >> compiling a hello-world.sml. The gdb traceback shows that it died in >> __gmpz_mul_2exp(). >> >> The version of gmp that I have is 2:4.3.1+dfsg-3. >> > I have reproduced this problem. Then I copied my svn/HEAD compiler from sarge to squeeze and it segfaulted as well. Something in squeeze has changed. The MinGW32 port self-compiles using 4.3.1 and debian hasn't patched gmp in any relevant way. Can you run through the regression suite with mlton-20070826? > Here are the highlights: testing flexrecord.2: Type error: actual and formal not of same type actual: ('a_4068 * nat) * (nat * nat) formal: (nat * nat) * (nat * nat) expression: ZZZ_f x_0 unhandled exception: TypeError compilation of flexrecord.2 failed with -type-check true real-algsimp (which appears to be a fix added around r6241). 1a2 > true 3,4c4 < false < false --- > true ... another new regression Error: test-create.sml 42.53. Function applied to incorrect argument. expects: [Unix.exit_status] but got: [?.PosixProcess.exit_status] in: statusToString status ... also a fix since 2007. testing thread-switch-share 1,2c1,2 < size1 >= size2 = true < sum1 = sum2 = true --- > ./bin/regression: line 28: 13263 Segmentation fault "./$f" > Nonzero exit status. testing thread-switch-size 1c1,2 < !rs > 0 = true --- > ./bin/regression: line 28: 13301 Segmentation fault "./$f" > Nonzero exit status. ... more bugs fixed since 2007. testing weak.2 DeepFlatten.replaceVar global_0 compilation of weak.2 failed with -type-check true ... not sure about this one? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091009/75fa20e5/attachment.htm From mtf at cs.rit.edu Fri Oct 9 08:16:02 2009 From: mtf at cs.rit.edu (Matthew Fluet) Date: Fri Oct 9 08:16:16 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: <162de7480910090756g695634c0u3ac445966fbb7176@mail.gmail.com> References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910090756g695634c0u3ac445966fbb7176@mail.gmail.com> Message-ID: On Fri, 9 Oct 2009, Wesley W. Terpstra wrote: > On Fri, Oct 9, 2009 at 4:21 AM, Matthew Fluet wrote: > >> On Thu, 8 Oct 2009, Henry Cejtin wrote: >> >>> The make couldn't finish because the resulting mlton-compile segfaults >>> compiling mllex.mlb. In fact, that mlton-compile always segfaults, even >>> compiling a hello-world.sml. The gdb traceback shows that it died in >>> __gmpz_mul_2exp(). >>> >>> The version of gmp that I have is 2:4.3.1+dfsg-3. >>> >> > I have reproduced this problem. Then I copied my svn/HEAD compiler from > sarge to squeeze and it segfaulted as well. Something in squeeze has > changed. The MinGW32 port self-compiles using 4.3.1 and debian hasn't > patched gmp in any relevant way. My x86-darwin (sing MacPorts gmp) also has 4.3.1, without problems. > Can you run through the regression suite with mlton-20070826? >> > > Here are the highlights: > > testing flexrecord.2: > Type error: actual and formal not of same type > actual: ('a_4068 * nat) * (nat * nat) > formal: (nat * nat) * (nat * nat) > expression: ZZZ_f x_0 > unhandled exception: TypeError > compilation of flexrecord.2 failed with -type-check true Test introduced by r6216 (20071127) and subsequently fixed. > testing weak.2 > DeepFlatten.replaceVar global_0 > compilation of weak.2 failed with -type-check true Test introduced by r6189 (20071120) and subsequently fixed. So, no additional gmp bugs. But, gmp reallocation bugs aren't always so obvious, because we always give it to the end of the heap (even if we limit checked for a smaller amount), so there is no error unless we really are running into the end of the heap. Could you try the regressions again with '-debug true'? That should throw an assertion if there really isn't enough room in the heap. You could also try to compile HEAD with 20070826 and '-debug true', which might be more revealing, since the resulting mlton-compile always segfaults. Of course, I'm surprised that you and Henry see the same behavior. If it is a bad limit check (due to a miscalculation of needed bytes for an IntInf result), then it is usually so dependent upon the exact sequence of GCs, which in turn is very sensitive to available memory and the exact contents of the heap (which includes things like the string representation of the path to the current executable, likely to be of different sizes on different machines), that it is nearly impossible to recreate on another machine. From henry.cejtin at sbcglobal.net Fri Oct 9 10:12:52 2009 From: henry.cejtin at sbcglobal.net (Henry Cejtin) Date: Fri Oct 9 10:13:25 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: References: <509955.22208.qm@web82404.mail.mud.yahoo.com> Message-ID: <707333.71199.qm@web82408.mail.mud.yahoo.com> I tried the stock AMD-64 version of MLton 20070826 on all the current int- inf.* things and got no errors. I'll compile the latest SVN with -debug true and see if that is any more revealing. From wesley at terpstra.ca Sat Oct 10 13:27:49 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Sat Oct 10 13:28:27 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910090756g695634c0u3ac445966fbb7176@mail.gmail.com> Message-ID: <162de7480910101327i553e2adfkb9fed6b6398f242e@mail.gmail.com> On Fri, Oct 9, 2009 at 5:16 PM, Matthew Fluet wrote: > But, gmp reallocation bugs aren't always so obvious It's not a gmp reallocation bug. Here's what I know so far: * The input argument is '1' and the shift is by 128 * We have 67 reserve bytes indicated * There is enough room for 33037511 more limbs, so not at the heap end. * The argument is allocated on the stack * The same parameters work several times before the segfault * gdb shows that the target IntInf has been filled correctly * /proc/*/maps show the memory address is in a valid range * It is dying on the MPN_ZERO line in mpz/mul_2exp.c * The memory is only 4-byte aligned at the point of failure I've tried compiling with -align 8 and then it works... I'm not sure this is a solution, though; it may have just masked the problem. Can you see if adding -align 8 to mlton/Makefile fixes it for you as well, Henry? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091010/c2506101/attachment.html From wesley at terpstra.ca Sat Oct 10 14:10:22 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Sat Oct 10 14:10:27 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: <162de7480910101327i553e2adfkb9fed6b6398f242e@mail.gmail.com> References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910090756g695634c0u3ac445966fbb7176@mail.gmail.com> <162de7480910101327i553e2adfkb9fed6b6398f242e@mail.gmail.com> Message-ID: <162de7480910101410j12a9f36fu79d5551684457151@mail.gmail.com> On Sat, Oct 10, 2009 at 10:27 PM, Wesley W. Terpstra wrote: > I've tried compiling with -align 8 and then it works... I'm not sure this > is a solution, though; it may have just masked the problem. > Found the smoking gun! Debian builds gmp with -O3 whereas I used -O2 for MinGW32. If you look at the assembler output of mpz/mul_exp.c with the two options you will notice a difference... the introduction of a 'movdqa' instruction, which is an SSE2 instruction that expects 16-byte alignment. >From what I've read, an array of 64-bit words should be 64-bit aligned. MLton IntInfs are such arrays and must thus be 8-byte aligned. They aren't. Here's the problem vectorized assembler from gcc with -O3 (I've marked the problem code): .LVL16: andl $15, %eax shrq $3, %rax ^^^^^^^^^^^ This ignores the 4-byte alignment of the array, only caring about it's 8-byte alignment before it moves on to doing 16-byte aligned moves. cmpq %r12, %rax cmova %r12, %rax testq %rax, %rax je .L10 .LBB2: cmpq %rax, %r12 movq $0, (%r14) leaq 8(%r14), %rdi leaq -1(%r12), %rsi je .L8 .L10: movq %r12, %rbx subq %rax, %rbx movq %rbx, %rcx shrq %rcx movq %rcx, %r9 addq %r9, %r9 je .L16 pxor %xmm0, %xmm0 leaq (%r14,%rax,8), %r8 xorl %edx, %edx .p2align 4,,10 .p2align 3 .L12: .loc 1 64 0 movq %rdx, %rax addq $1, %rdx salq $4, %rax cmpq %rcx, %rdx movdqa %xmm0, (%r8,%rax) ^^^^^^^^^^^^^^^^^^^^^^^^^ At this point the memory MUST be 16-byte aligned, but isn't if the input is 4-byte aligned +8 -> 12!=0 mod 16. This causes our segfault. jb .L12 subq %r9, %rsi cmpq %r9, %rbx leaq (%rdi,%r9,8), %rdi je .L8 What's the plan going forward? align(AMD64) == 8? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091010/903bdf7d/attachment.htm From mtf at cs.rit.edu Sun Oct 11 06:59:14 2009 From: mtf at cs.rit.edu (Matthew Fluet) Date: Tue Oct 13 04:52:45 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: <162de7480910101410j12a9f36fu79d5551684457151@mail.gmail.com> References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910090756g695634c0u3ac445966fbb7176@mail.gmail.com> <162de7480910101327i553e2adfkb9fed6b6398f242e@mail.gmail.com> <162de7480910101410j12a9f36fu79d5551684457151@mail.gmail.com> Message-ID: On Sat, 10 Oct 2009, Wesley W. Terpstra wrote: > On Sat, Oct 10, 2009 at 10:27 PM, Wesley W. Terpstra wrote: > >> I've tried compiling with -align 8 and then it works... I'm not sure this >> is a solution, though; it may have just masked the problem. > > Found the smoking gun! Debian builds gmp with -O3 whereas I used -O2 for > MinGW32. If you look at the assembler output of mpz/mul_exp.c with the two > options you will notice a difference... the introduction of a 'movdqa' > instruction, which is an SSE2 instruction that expects 16-byte alignment. ... > What's the plan going forward? align(AMD64) == 8? Nice detective work. However, from what you have described, it doesn't seem as though defaulting to '-align 8' on amd64{-linux?} is sufficient. That will guarantee that all objects are aligned on 8-byte boundaries, but it won't guarantee that IntInf arrays are aligned on 16-byte boundaries. Going to '-align 16' would waste quite a bit more space. On the other hand, I don't see how the gmp.h header guarantees that a mp_limb_t* is 16-byte aligned. It is simply a pointer to a 64-bit integer, so it seems that gcc can only assume that the allocated object pointed to by a mp_limb_t* is 8-byte aligned. Am I misunderstanding the assembly? Is it dynamically checking the alignment and using the SSE2 instructions only if it happens to be 16-byte aligned? In which case, then '-align 8' would be reasonable. From wesley at terpstra.ca Sun Oct 11 07:08:56 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Tue Oct 13 04:52:47 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910090756g695634c0u3ac445966fbb7176@mail.gmail.com> <162de7480910101327i553e2adfkb9fed6b6398f242e@mail.gmail.com> <162de7480910101410j12a9f36fu79d5551684457151@mail.gmail.com> Message-ID: <162de7480910110708s1ea2ea90t5a038fcdac6ae129@mail.gmail.com> On Sun, Oct 11, 2009 at 3:59 PM, Matthew Fluet wrote: > On Sat, 10 Oct 2009, Wesley W. Terpstra wrote: > >> On Sat, Oct 10, 2009 at 10:27 PM, Wesley W. Terpstra > >wrote: >> >> I've tried compiling with -align 8 and then it works... I'm not sure this >>> is a solution, though; it may have just masked the problem. >>> >> >> Found the smoking gun! Debian builds gmp with -O3 whereas I used -O2 for >> MinGW32. If you look at the assembler output of mpz/mul_exp.c with the two >> options you will notice a difference... the introduction of a 'movdqa' >> instruction, which is an SSE2 instruction that expects 16-byte alignment. >> > ... > >> What's the plan going forward? align(AMD64) == 8? >> > > However, from what you have described, it doesn't seem as though defaulting > to '-align 8' on amd64{-linux?} is sufficient. No, 8 alignment is sufficient. That assembler from gcc assumes it was 8 byte aligned and then promotes it to 16-byte alignment by testing the 3rd bit and optionally doing a single 8-byte operation before moving on to 16-byte operations. > ... it won't guarantee that IntInf arrays are aligned on 16-byte boundaries Unnecessary. On the other hand, I don't see how the gmp.h header guarantees that a > mp_limb_t* is 16-byte aligned. It doesn't. > It is simply a pointer to a 64-bit integer, so it seems that gcc can only > assume that the allocated object pointed to by a mp_limb_t* is 8-byte > aligned. That's exactly what it assumes. > Am I misunderstanding the assembly? Is it dynamically checking the > alignment and using the SSE2 instructions only if it happens to be 16-byte > aligned? > It dynamically checks the alignment and does a single 8-byte operation if necessary to move forward onto 16-byte aligned values. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091011/41ee349b/attachment.html From wesley at terpstra.ca Sun Oct 11 07:15:31 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Tue Oct 13 04:52:59 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: <162de7480910110708s1ea2ea90t5a038fcdac6ae129@mail.gmail.com> References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910090756g695634c0u3ac445966fbb7176@mail.gmail.com> <162de7480910101327i553e2adfkb9fed6b6398f242e@mail.gmail.com> <162de7480910101410j12a9f36fu79d5551684457151@mail.gmail.com> <162de7480910110708s1ea2ea90t5a038fcdac6ae129@mail.gmail.com> Message-ID: <162de7480910110715y49b3b1a3v7d478986935755c4@mail.gmail.com> Before we change MLton's default alignment, I think there is another question that needs to be answered: is gcc's behaviour correct? I assume you (Matthew) read the amd64 ABI. What exactly are the rules for alignment of pointers/word64s? I assumed that they can be 4-byte aligned on amd64, but perhaps being inside an array requires they be 8-byte aligned? If gcc is correct to assume that arrays containing word64s should be 8-byte aligned, then we should definitely move to -align 8 on all amd64 targets. However, if gcc is mistaken, then we should probably file a bug against gcc and leave -align 4 as is. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091011/55b191f6/attachment.html From wesley at terpstra.ca Tue Oct 13 01:17:33 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Tue Oct 13 04:54:53 2009 Subject: [MLton] In hope of a release ... Message-ID: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> Just a heads-up: Since summer has come to a close a new release of MLton should be happening soon. To that end I'm going to start pushing svn builds of MLton to debian/experimental and try to get it built on more of the debian architectures. With the restructured c-types.ml system from Matthew I think it should be relatively easy to get it going on most of the debian targets using the C codegen... I hope. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091013/b95edbba/attachment.htm From wesley at terpstra.ca Tue Oct 13 01:20:45 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Tue Oct 13 04:54:55 2009 Subject: [MLton] Real32... optional? Message-ID: <162de7480910130120i6c49786cra57977332fd646f5@mail.gmail.com> The s390 architecture only has 64-bit IEEE registers. Should I remove the Real32 structure and update the regressions appropriately? Or does MLton guarantee the presence of this optional structure? Otherwise the port was a breeze (all other regressions pass). -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091013/a7350c06/attachment.html From wesley at terpstra.ca Tue Oct 13 02:34:42 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Tue Oct 13 04:55:00 2009 Subject: [MLton] Re: Real32... optional? In-Reply-To: <162de7480910130120i6c49786cra57977332fd646f5@mail.gmail.com> References: <162de7480910130120i6c49786cra57977332fd646f5@mail.gmail.com> Message-ID: <162de7480910130234r29c180au8698472bfae5e185@mail.gmail.com> On Tue, Oct 13, 2009 at 10:20 AM, Wesley W. Terpstra wrote: > The s390 architecture only has 64-bit IEEE registers. Should I remove the > Real32 structure and update the regressions appropriately? Or does MLton > guarantee the presence of this optional structure? > Oh, nevermind. It has Real32... just not IEEE floats... nor IEEE doubles. The table at < http://de.wikipedia.org/wiki/Gleitkommazahl#Interne_Darstellung> shows they have more mantissa bits than they should have (and less exponent bits). Blah. Fortunately it seems gcc has software floating point options for this target, which I will enable. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091013/dfdad435/attachment.htm From mtf at cs.rit.edu Tue Oct 13 10:58:14 2009 From: mtf at cs.rit.edu (Matthew Fluet) Date: Tue Oct 13 10:58:29 2009 Subject: [MLton] Re: [MLton-commit] r7248 -- IA64 changes In-Reply-To: References: Message-ID: Ville ought to vette these changes (to the gcc flags passed on an IA64 architecture), since he made the initial port (for ia64-hpux). Is '-mlp64' the default mode (and, so the flag was redundant)? On Tue, 13 Oct 2009, Wesley Terpstra wrote: > God that port was easy. IA64 works. :) > > ---------------------------------------------------------------------- > > U mlton/trunk/bin/mlton-script > U mlton/trunk/package/debian/control > U mlton/trunk/runtime/Makefile > U mlton/trunk/runtime/platform/linux.c > > ---------------------------------------------------------------------- > > Modified: mlton/trunk/bin/mlton-script > =================================================================== > --- mlton/trunk/bin/mlton-script 2009-10-13 13:12:41 UTC (rev 7247) > +++ mlton/trunk/bin/mlton-script 2009-10-13 13:54:00 UTC (rev 7248) > @@ -109,7 +109,7 @@ > -target-cc-opt netbsd '-I/usr/pkg/include' \ > -target-cc-opt openbsd '-I/usr/local/include' \ > -target-cc-opt aix '-maix64' \ > - -target-cc-opt ia64 '-mlp64 -mtune=itanium2' \ > + -target-cc-opt ia64 '-mtune=itanium2' \ > -target-cc-opt sparc '-m32 -mcpu=v8 -Wa,-xarch=v8plusa' \ > -target-cc-opt x86 \ > '-m32 > > Modified: mlton/trunk/package/debian/control > =================================================================== > --- mlton/trunk/package/debian/control 2009-10-13 13:12:41 UTC (rev 7247) > +++ mlton/trunk/package/debian/control 2009-10-13 13:54:00 UTC (rev 7248) > @@ -6,7 +6,7 @@ > Standards-Version: 3.7.2 > > Package: mlton > -Architecture: amd64 hppa i386 powerpc s390 sparc > +Architecture: amd64 hppa i386 ia64 powerpc s390 sparc > Depends: ${shlibs:Depends}, gcc, libc6-dev, libgmp3-dev (>= 4.0.1) > Description: Optimizing compiler for Standard ML > MLton (mlton.org) is a whole-program optimizing > > Modified: mlton/trunk/runtime/Makefile > =================================================================== > --- mlton/trunk/runtime/Makefile 2009-10-13 13:12:41 UTC (rev 7247) > +++ mlton/trunk/runtime/Makefile 2009-10-13 13:54:00 UTC (rev 7248) > @@ -69,7 +69,7 @@ > endif > > ifeq ($(TARGET_ARCH), ia64) > -FLAGS += -mlp64 -mtune=itanium2 > +FLAGS += -mtune=itanium2 > endif > > ifeq ($(TARGET_OS)-$(TARGET_ARCH), aix-powerpc64) > > Modified: mlton/trunk/runtime/platform/linux.c > =================================================================== > --- mlton/trunk/runtime/platform/linux.c 2009-10-13 13:12:41 UTC (rev 7247) > +++ mlton/trunk/runtime/platform/linux.c 2009-10-13 13:54:00 UTC (rev 7248) > @@ -15,7 +15,6 @@ > > /* potentially correct for other archs: > * alpha: ucp->m_context.sc_pc > - * ia64: ucp->m_context.sc_ip & ~0x3UL > */ > static void catcher (__attribute__ ((unused)) int sig, > __attribute__ ((unused)) siginfo_t* sip, > @@ -30,6 +29,9 @@ > #elif (defined (__hppa__)) > ucontext_t* ucp = (ucontext_t*)mystery; > GC_handleSigProf ((code_pointer) (ucp->uc_mcontext.sc_iaoq[0] & ~0x3UL)); > +#elif (defined(__ia64__)) > + ucontext_t* ucp = (ucontext_t*)mystery; > + GC_handleSigProf ((code_pointer) ucp->_u._mc.sc_ip); > #elif (defined (__ppc__)) || (defined (__powerpc__)) > ucontext_t* ucp = (ucontext_t*)mystery; > GC_handleSigProf ((code_pointer) ucp->uc_mcontext.regs->nip); > > > _______________________________________________ > MLton-commit mailing list > MLton-commit@mlton.org > http://mlton.org/mailman/listinfo/mlton-commit > From wesley at terpstra.ca Tue Oct 13 11:17:44 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Tue Oct 13 11:18:20 2009 Subject: [MLton] Card/cross map alignment broken? Message-ID: <162de7480910131117k585a8282xb2fadef2fbf448f4@mail.gmail.com> I've noticed that the alpha port sometimes 'hangs'. The 'hang' is caused by a sudden burst of unaligned memory accesses which bring the kernel to its knees. I suppose if I waited long enough it would work, so it isn't really a hang. Obviously, I already use -align 8. Due to the nature of the hang I have a strong suspicion that the GC is responsible. Things run smoothly until suddenly the program 'hangs', but changing GC parameters will move the location of the 'hang'. Looking through the GC code I strongly suspect that the crossMap is ending up unaligned. Placing it after the cardMap (a 1-byte aligned array) seems quite risky, but I lack the expertise in this area of the code to be certain this is a problem. Can someone familiar with this code please check that s->generationalMaps.crossMap = (GC_crossMap) (s->heap.start + s->heap.size + cardMapSize); indeed must be >8 bytes aligned? Thank you! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091013/8f7bf83f/attachment-0001.html From fw at deneb.enyo.de Tue Oct 13 11:20:33 2009 From: fw at deneb.enyo.de (Florian Weimer) Date: Tue Oct 13 11:20:43 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: <162de7480910110715y49b3b1a3v7d478986935755c4@mail.gmail.com> (Wesley W. Terpstra's message of "Sun, 11 Oct 2009 16:15:31 +0200") References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910090756g695634c0u3ac445966fbb7176@mail.gmail.com> <162de7480910101327i553e2adfkb9fed6b6398f242e@mail.gmail.com> <162de7480910101410j12a9f36fu79d5551684457151@mail.gmail.com> <162de7480910110708s1ea2ea90t5a038fcdac6ae129@mail.gmail.com> <162de7480910110715y49b3b1a3v7d478986935755c4@mail.gmail.com> Message-ID: <87vdijnojy.fsf@mid.deneb.enyo.de> * Wesley W. Terpstra: > Before we change MLton's default alignment, I think there is another > question that needs to be answered: is gcc's behaviour correct? I assume you > (Matthew) read the amd64 ABI. What exactly are the rules for alignment of > pointers/word64s? I assumed that they can be 4-byte aligned on amd64, but > perhaps being inside an array requires they be 8-byte aligned? 8-byte alignment is preferable for performance reasons (so GCC follows that), but it's not enforced by the hardware. From mtf at cs.rit.edu Tue Oct 13 12:14:53 2009 From: mtf at cs.rit.edu (Matthew Fluet) Date: Tue Oct 13 12:14:58 2009 Subject: [MLton] Card/cross map alignment broken? In-Reply-To: <162de7480910131117k585a8282xb2fadef2fbf448f4@mail.gmail.com> References: <162de7480910131117k585a8282xb2fadef2fbf448f4@mail.gmail.com> Message-ID: On Tue, 13 Oct 2009, Wesley W. Terpstra wrote: > Due to the nature of the hang I have a strong suspicion that the GC is > responsible. Things run smoothly until suddenly the program 'hangs', but > changing GC parameters will move the location of the 'hang'. Looking through > the GC code I strongly suspect that the crossMap is ending up unaligned. > Placing it after the cardMap (a 1-byte aligned array) seems quite risky, but > I lack the expertise in this area of the code to be certain this is a > problem. > > Can someone familiar with this code please check that > s->generationalMaps.crossMap = > (GC_crossMap) (s->heap.start + s->heap.size + cardMapSize); > indeed must be >8 bytes aligned? All of heap.size, cardMapSize, and crossMapSize are aligned according to the system page size (s->sysvals.pagesize). See sizeofCardMap and sizeofCrossMap in /runtime/gc/generational.c; under the current parameters, they should always be the same size. And heap.start is a large mmap with a page aligned size, so should be page aligned. If you run with '@MLton gc-messages', you'll see the address at which the heap is allocated and the size of the card/cross map. From wesley at terpstra.ca Tue Oct 13 11:12:57 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Tue Oct 13 12:15:19 2009 Subject: [MLton] Re: [MLton-commit] r7248 -- IA64 changes In-Reply-To: References: Message-ID: <162de7480910131112m1bd9e6c7l8b8464abc98385ea@mail.gmail.com> On Tue, Oct 13, 2009 at 7:58 PM, Matthew Fluet wrote: > Ville ought to vette these changes (to the gcc flags passed on an IA64 > architecture), since he made the initial port (for ia64-hpux). Is '-mlp64' > the default mode (and, so the flag was redundant)? > We've been in discussion about this already. HP-UX needs it and Linux can't have it. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091013/48bfde56/attachment.htm From mtf at cs.rit.edu Tue Oct 13 12:21:53 2009 From: mtf at cs.rit.edu (Matthew Fluet) Date: Tue Oct 13 12:22:07 2009 Subject: [MLton] Real32... optional? In-Reply-To: <162de7480910130120i6c49786cra57977332fd646f5@mail.gmail.com> References: <162de7480910130120i6c49786cra57977332fd646f5@mail.gmail.com> Message-ID: On Tue, 13 Oct 2009, Wesley W. Terpstra wrote: > The s390 architecture only has 64-bit IEEE registers. Should I remove the > Real32 structure and update the regressions appropriately? Or does MLton > guarantee the presence of this optional structure? > > Otherwise the port was a breeze (all other regressions pass). Real32 is definitely desirable on x86 and amd64. Reconfiguring the basis library to only sometimes compile and export Real32 would be a major undertaking. The platform configurability of the basis library is only on a very local level. Using the software floating-point support seems best. From wesley at terpstra.ca Tue Oct 13 13:54:12 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Tue Oct 13 13:54:49 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: <87vdijnojy.fsf@mid.deneb.enyo.de> References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910090756g695634c0u3ac445966fbb7176@mail.gmail.com> <162de7480910101327i553e2adfkb9fed6b6398f242e@mail.gmail.com> <162de7480910101410j12a9f36fu79d5551684457151@mail.gmail.com> <162de7480910110708s1ea2ea90t5a038fcdac6ae129@mail.gmail.com> <162de7480910110715y49b3b1a3v7d478986935755c4@mail.gmail.com> <87vdijnojy.fsf@mid.deneb.enyo.de> Message-ID: <162de7480910131354r244fd724tbb66e633f9664759@mail.gmail.com> On Tue, Oct 13, 2009 at 8:20 PM, Florian Weimer wrote: > 8-byte alignment is preferable for performance reasons (so GCC follows > that) Sure, that's clear. gcc is welcome to emit aligned output. The question is if it is justified to assume that all other modules likewise align their output. That's what it did and that's why we crash. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091013/2100f524/attachment.html From wesley at terpstra.ca Tue Oct 13 17:10:13 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Tue Oct 13 17:10:49 2009 Subject: [MLton] Card/cross map alignment broken? In-Reply-To: References: <162de7480910131117k585a8282xb2fadef2fbf448f4@mail.gmail.com> Message-ID: <162de7480910131710x5b40b5e7w7fa86fd1fb4a20f8@mail.gmail.com> On Tue, Oct 13, 2009 at 9:14 PM, Matthew Fluet wrote: > All of heap.size, cardMapSize, and crossMapSize are aligned according to > the system page size (s->sysvals.pagesize). Ok, but it is definitely updateCrossMap that is doing the unaligned accesses. The moment the GC switches to generational behaviour, the performance of the system dies. I'm trying to narrow the problem down further, but it's painful... 30 minutes per binary search step. :p gdb says the address of the trap is... (gdb) break *(void*)0x0000000120d843ac Breakpoint 1 at 0x120d843ac: file gc/generational.c, line 373. => s->generationalMaps.crossMap[cardIndex] = (GC_crossMapElem)offset; ... but we all know how reliable gcc is in this regard.. o.O -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091014/b425fe26/attachment.htm From wesley at terpstra.ca Tue Oct 13 18:43:05 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Tue Oct 13 18:43:40 2009 Subject: [MLton] Card/cross map alignment broken? In-Reply-To: <162de7480910131710x5b40b5e7w7fa86fd1fb4a20f8@mail.gmail.com> References: <162de7480910131117k585a8282xb2fadef2fbf448f4@mail.gmail.com> <162de7480910131710x5b40b5e7w7fa86fd1fb4a20f8@mail.gmail.com> Message-ID: <162de7480910131843q45011596gcf053afdff1644f5@mail.gmail.com> On Wed, Oct 14, 2009 at 2:10 AM, Wesley W. Terpstra wrote: > Ok, but it is definitely updateCrossMap that is doing the unaligned > accesses. > > gdb says the address of the trap is... > (gdb) break *(void*)0x0000000120d843ac > Breakpoint 1 at 0x120d843ac: file gc/generational.c, line 373. > => s->generationalMaps.crossMap[cardIndex] = (GC_crossMapElem)offset; > ... shockingly, gdb was correct. It took me some time to wrap my mind around this, but there is indeed a bug in that line. Or rather, that line became buggy when r4683 tried to quell a warning. The warning was harmless, but the "fix" wasn't. Here's what is going on: typedef uint8_t GC_crossMapStart __attribute__ ((aligned(4)); defines a type that is a uint8_t, which happens to be aligned on a 4-byte boundary. That's all fine and dandy. Make a pointer type to the object, well that pointer is 4-byte aligned. Swell. Now deference that pointer at [1]. Uhm. Well, the object type pointed to is aligned(4), so the (pointer+1) must be 4-byte aligned. I'm an alpha and loading a byte is expensive, but no problem, it's word aligned! Let's just load it using a word32 memory access. BAM! unaligned access. Morale of the story? Don't mess around with __attribute__ ((aligned))! Two of the three places it is used in MLton are wrong and I'm not 100% sure the remaining use (pointer) is safe either. I assume that "pointer" is used only to refer to objects on the heap. That means that on an alpha it is actually 8-byte aligned. Telling the alpha it's less aligned than it actually is will either degrade performance or have no effect at all. When you cast pointers you might get a warning, but gcc is also doing the 'right thing' for MLton's purposes. If I cast a char* to an int* gcc assumes that I know what I'm doing and that the char* pointer was really aligned to an int after all. That's why it issues a warning, to let you know it just increased its alignment assumptions. Telling gcc that the char* was 4-byte aligned quells the warning (on a 32-bit system) but doesn't improve the assembler. All it does is introduce potential bugs if you ever derefence that char* pointer at say [1] [2] or [3]. So there is absolutely nothing to gain (performance-wise) by declaring these types as aligned and everything to lose (correctness-wise). As for the warnings, they aren't even warnings that are enabled by default in C! We turned them on with -Wcast-align. They are off by default because lots of correct C code (including ours) do casts like this. The align(4) hack also doesn't help on 64-bit ports... I get flooded with these warnings on ia64 and alpha. I am going to unilaterally remove all __attribute__ ((align))s and quell this class of warning by removing -Wcast-align. After my rant I assume no one dares to object. :) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091014/c02a42e9/attachment.html From fw at deneb.enyo.de Tue Oct 13 22:23:13 2009 From: fw at deneb.enyo.de (Florian Weimer) Date: Tue Oct 13 22:23:21 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: <162de7480910131354r244fd724tbb66e633f9664759@mail.gmail.com> (Wesley W. Terpstra's message of "Tue, 13 Oct 2009 22:54:12 +0200") References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910090756g695634c0u3ac445966fbb7176@mail.gmail.com> <162de7480910101327i553e2adfkb9fed6b6398f242e@mail.gmail.com> <162de7480910101410j12a9f36fu79d5551684457151@mail.gmail.com> <162de7480910110708s1ea2ea90t5a038fcdac6ae129@mail.gmail.com> <162de7480910110715y49b3b1a3v7d478986935755c4@mail.gmail.com> <87vdijnojy.fsf@mid.deneb.enyo.de> <162de7480910131354r244fd724tbb66e633f9664759@mail.gmail.com> Message-ID: <87skdm35xa.fsf@mid.deneb.enyo.de> * Wesley W. Terpstra: > Sure, that's clear. gcc is welcome to emit aligned output. The question is > if it is justified to assume that all other modules likewise align their > output. That's what it did and that's why we crash. The AMD64 psABI supplement requires 8-byte alignment. And the end of the input area on the stack must be aligned to a 16 byte boundary upon function entry. See . From wesley at terpstra.ca Wed Oct 14 02:57:19 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Wed Oct 14 02:57:55 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: <87skdm35xa.fsf@mid.deneb.enyo.de> References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910101327i553e2adfkb9fed6b6398f242e@mail.gmail.com> <162de7480910101410j12a9f36fu79d5551684457151@mail.gmail.com> <162de7480910110708s1ea2ea90t5a038fcdac6ae129@mail.gmail.com> <162de7480910110715y49b3b1a3v7d478986935755c4@mail.gmail.com> <87vdijnojy.fsf@mid.deneb.enyo.de> <162de7480910131354r244fd724tbb66e633f9664759@mail.gmail.com> <87skdm35xa.fsf@mid.deneb.enyo.de> Message-ID: <162de7480910140257g7e6bfa46lfb3ce618ce62a5d3@mail.gmail.com> On Wed, Oct 14, 2009 at 7:23 AM, Florian Weimer wrote: > The AMD64 psABI supplement requires 8-byte alignment. And the end of > the input area on the stack must be aligned to a 16 byte boundary upon > function entry. See . > The stack was actually already aligned, just not the GMP limb array in MLton's heap. Still, if requiring this 8-byte alignment is sanctioned, we should just go with -align 8 and be safe. Since Matthew is the AMD64 expert I'm mostly waiting to hear his opinion before I commit a fix. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091014/a1763248/attachment-0001.htm From ville at laurikari.net Wed Oct 14 02:59:44 2009 From: ville at laurikari.net (Ville Laurikari) Date: Wed Oct 14 05:21:58 2009 Subject: [MLton] Re: [MLton-commit] r7248 -- IA64 changes In-Reply-To: <162de7480910131112m1bd9e6c7l8b8464abc98385ea@mail.gmail.com> References: <162de7480910131112m1bd9e6c7l8b8464abc98385ea@mail.gmail.com> Message-ID: <20091014095944.GA16582@laurikari.net> On Tue, Oct 13, 2009 at 08:12:57PM +0200, Wesley W. Terpstra wrote: > On Tue, Oct 13, 2009 at 7:58 PM, Matthew Fluet wrote: > > Ville ought to vette these changes (to the gcc flags passed on an IA64 > We've been in discussion about this already. HP-UX needs it and Linux can't > have it. Yep. The latest version is OK. -- http://hashedbits.com/ From matthew.fluet at gmail.com Wed Oct 14 14:57:11 2009 From: matthew.fluet at gmail.com (Matthew Fluet) Date: Wed Oct 14 14:57:45 2009 Subject: [MLton] In hope of a release ... In-Reply-To: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> Message-ID: It is fortunate that the ports have been going so smoothly. I would have cautioned against it ... introducing new, and mostly untested, platforms shortly before a release is not a recipe for stability. There are a few other items that I would like to finish before releasing. One is the performance bugs identified by Reactive Systems. Another is a regression failure with signals on Mac OS X 10.5 (Leopard), which I believe is due to a broken sigaltstack, but have had difficulty isolating. On Tue, Oct 13, 2009 at 4:17 AM, Wesley W. Terpstra wrote: > Just a heads-up: > > Since summer has come to a close a new release of MLton should be happening > soon. To that end I'm going to start pushing svn builds of MLton to > debian/experimental and try to get it built on more of the debian > architectures. With the restructured c-types.ml system from Matthew I think > it should be relatively easy to get it going on most of the debian targets > using the C codegen... I hope. > > _______________________________________________ > MLton mailing list > MLton@mlton.org > http://mlton.org/mailman/listinfo/mlton > From matthew.fluet at gmail.com Wed Oct 14 14:57:33 2009 From: matthew.fluet at gmail.com (Matthew Fluet) Date: Wed Oct 14 14:58:15 2009 Subject: [MLton] Card/cross map alignment broken? In-Reply-To: <162de7480910131843q45011596gcf053afdff1644f5@mail.gmail.com> References: <162de7480910131117k585a8282xb2fadef2fbf448f4@mail.gmail.com> <162de7480910131710x5b40b5e7w7fa86fd1fb4a20f8@mail.gmail.com> <162de7480910131843q45011596gcf053afdff1644f5@mail.gmail.com> Message-ID: Thanks for the explanation. The use of aligning the "pointer" type was meant for heap objects that are (at least) 4 byte aligned. On Tue, Oct 13, 2009 at 9:43 PM, Wesley W. Terpstra wrote: > On Wed, Oct 14, 2009 at 2:10 AM, Wesley W. Terpstra > wrote: >> >> Ok, but it is definitely updateCrossMap that is doing the unaligned >> accesses. >> gdb says the address of the trap is... >> (gdb) break *(void*)0x0000000120d843ac >> Breakpoint 1 at 0x120d843ac: file gc/generational.c, line 373. >> => s->generationalMaps.crossMap[cardIndex] = (GC_crossMapElem)offset; > > ... shockingly, gdb was correct. It took me some time to wrap my mind around > this, but there is indeed a bug in that line. Or rather, that line became > buggy when r4683 tried to quell a warning. The warning was harmless, but the > "fix" wasn't. > > Here's what is going on: > ? typedef uint8_t GC_crossMapStart __attribute__ ((aligned(4)); > defines a type that is a uint8_t, which happens to be aligned on a 4-byte > boundary. That's all fine and dandy. Make a pointer type to the object, well > that pointer is 4-byte aligned. Swell. > > Now deference that pointer at [1]. > > Uhm. > > Well, the object type pointed to is aligned(4), so the (pointer+1) must be > 4-byte aligned. I'm an alpha and loading a byte is expensive, but no > problem, it's word aligned! Let's just load it using a word32 memory access. > BAM! unaligned access. > > Morale of the story? Don't mess around with __attribute__ ((aligned))! Two > of the three places it is used in MLton are wrong and I'm not 100% sure the > remaining use (pointer) is safe either. I assume that "pointer" is used only > to refer to objects on the heap. That means that on an alpha it is actually > 8-byte aligned. Telling the alpha it's less aligned than it actually is will > either degrade performance or have no effect at all. > > When you cast pointers you might get a warning, but gcc is also doing the > 'right thing' for MLton's purposes. If I cast a char* to an int* gcc assumes > that I know what I'm doing and that the char* pointer was really aligned to > an int after all. That's why it issues a warning, to let you know it just > increased its alignment assumptions. Telling gcc that the char* was 4-byte > aligned quells the warning (on a 32-bit system) but doesn't improve the > assembler. All it does is introduce potential bugs if you ever derefence > that char* pointer at say [1] [2] or [3]. So there is absolutely nothing to > gain (performance-wise) by declaring these types as aligned and everything > to lose (correctness-wise). > > As for the warnings, they aren't even warnings that are enabled by default > in C! We turned them on with -Wcast-align. They are off by default because > lots of correct C code (including ours) do casts like this. The align(4) > hack also doesn't help on 64-bit ports...? I get flooded with these warnings > on ia64 and alpha. > > I am going to unilaterally remove all __attribute__ ((align))s and quell > this class of warning by removing -Wcast-align. After my rant I assume no > one dares to object. :) > > > _______________________________________________ > MLton mailing list > MLton@mlton.org > http://mlton.org/mailman/listinfo/mlton > From matthew.fluet at gmail.com Wed Oct 14 14:58:20 2009 From: matthew.fluet at gmail.com (Matthew Fluet) Date: Wed Oct 14 14:58:53 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: <162de7480910140257g7e6bfa46lfb3ce618ce62a5d3@mail.gmail.com> References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910101327i553e2adfkb9fed6b6398f242e@mail.gmail.com> <162de7480910101410j12a9f36fu79d5551684457151@mail.gmail.com> <162de7480910110708s1ea2ea90t5a038fcdac6ae129@mail.gmail.com> <162de7480910110715y49b3b1a3v7d478986935755c4@mail.gmail.com> <87vdijnojy.fsf@mid.deneb.enyo.de> <162de7480910131354r244fd724tbb66e633f9664759@mail.gmail.com> <87skdm35xa.fsf@mid.deneb.enyo.de> <162de7480910140257g7e6bfa46lfb3ce618ce62a5d3@mail.gmail.com> Message-ID: I'm hardly an expert. I used the www.x86-64.org document to implement the C calling convention in the native codegen, but didn't peruse it much otherwise. Searching for "align" in the document, though, reveals that on page 12, it declares that {,signed,unsigned} {,long} long all have 8-byte alignment. However, on the next page it states: Like the Intel386 architecture, the AMD64 architecture in general does not re- quire all data accesses to be properly aligned. Misaligned data accesses are slower than aligned accesses but otherwise behave identically. The only exceptions are that __m128 and __m256 must always be aligned properly. So, it isn't clear to me that one really needs to 8-align 64-bit integers. In the next subsection (p. 13), on aggregates and unions, it states: An array uses the same alignment as its elements, except that a local or global array variable of length at least 16 bytes or a C99 variable-length array variable always has alignment of at least 16 bytes. This seems to suggest that gcc is within its rights to assume that an array of {,long} long-s is 16-byte aligned (if it has at least 2 elements). MLton would have a hard time supporting that short of implementing '-align 16'. I suggest that going with '-align 8' (as Wesley committed earlier today) is a reasonable way to go. I did want to also point out that there is a legacy issue, I would assume, on Debian. Since mlton-20070826 is dynamically linked against libgmp, isn't it just an incredible luck of the draw that a self-compile with mlton-20070826 didn't happen to produce a non-16-byte aligned IntInf array. On Wed, Oct 14, 2009 at 5:57 AM, Wesley W. Terpstra wrote: > On Wed, Oct 14, 2009 at 7:23 AM, Florian Weimer wrote: >> >> The AMD64 psABI supplement requires 8-byte alignment. ?And the end of >> the input area on the stack must be aligned to a 16 byte boundary upon >> function entry. See . > > The stack was actually already aligned, just not the GMP limb array in > MLton's heap. > > Still, if requiring this 8-byte alignment is sanctioned, we should just go > with -align 8 and be safe. > > Since Matthew is the AMD64 expert I'm mostly waiting to hear his opinion > before I commit a fix. > > > _______________________________________________ > MLton mailing list > MLton@mlton.org > http://mlton.org/mailman/listinfo/mlton > From wesley at terpstra.ca Wed Oct 14 15:30:33 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Wed Oct 14 15:31:08 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910101410j12a9f36fu79d5551684457151@mail.gmail.com> <162de7480910110708s1ea2ea90t5a038fcdac6ae129@mail.gmail.com> <162de7480910110715y49b3b1a3v7d478986935755c4@mail.gmail.com> <87vdijnojy.fsf@mid.deneb.enyo.de> <162de7480910131354r244fd724tbb66e633f9664759@mail.gmail.com> <87skdm35xa.fsf@mid.deneb.enyo.de> <162de7480910140257g7e6bfa46lfb3ce618ce62a5d3@mail.gmail.com> Message-ID: <162de7480910141530n56865d40r1336e77e9397e3f7@mail.gmail.com> On Wed, Oct 14, 2009 at 11:56 PM, Matthew Fluet wrote: > I'm hardly an expert. I used the www.x86-64.org document to implement > the C calling convention in the native codegen, but didn't peruse it > much otherwise. > Nice link, thanks. > Searching for "align" in the document, though, reveals that on page > 12, it declares that {,signed,unsigned} {,long} long all have 8-byte > alignment. Ok, that table is pretty clear. The ABI defines that Word64s must be 8-byte aligned. Therefore gcc was within it's rights to assume that the pointer was 8-byte aligned and the bug was ours. > However, on the next page it states: > Like the Intel386 architecture, the AMD64 architecture in general > does not re- > quire all data accesses to be properly aligned. Misaligned data > accesses are slower > than aligned accesses but otherwise behave identically. The only > exceptions are > that __m128 and __m256 must always be aligned properly. > This is not a contradiction. Architecture != ABI. The machine can do it, but the ABI forbids it. So, it isn't clear to me that one really needs to 8-align 64-bit integers. > If we want to link with any other application code ... libc, libgmp, ffi, .... then it's 100% clear we need to do 8-byte alignment. We have just been lucky that no other software actually made use of the 8-byte alignment guarantee until now (since few architectural limitations actually trip over an ABI violation). > In the next subsection (p. 13), on aggregates and unions, it states: > > An array uses the same alignment as its elements, except that a > local or global array variable of length at least 16 bytes or a C99 > variable-length > array variable always has alignment of at least 16 bytes. > I think by global/local arrays they mean arrays not in the heap but the data segment. (local = static int64_t foo[4];, global = extern int64_t foo[4];) At any rate, this sounds like we don't need to worry because MLton only passes arrays as pointers (both FFI and GMP limb structure). I did want to also point out that there is a legacy issue, I would > assume, on Debian. Since mlton-20070826 is dynamically linked against > libgmp, isn't it just an incredible luck of the draw that a > self-compile with mlton-20070826 didn't happen to produce a > non-16-byte aligned IntInf array. > Yes, I was surprised too. However there are a couple reasons this worked out. First, the only code gcc managed to vectorize in the gmp C is the MPN_ZERO method. Second, the only place MPN_ZERO gets called (for us) is to clear the low bits of a left-shifted intinf. Third, it won't use 16-byte writes unless there are 16-bytes to write, so it had to be a >=128-bit left shift. I wonder if these maybe didn't happen in 20070826? I imagine that as gcc gets smarter, vectorizing more code, this will become a more serious legacy issue. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091015/20b367c3/attachment.html From wesley at terpstra.ca Wed Oct 14 15:47:24 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Wed Oct 14 15:47:58 2009 Subject: [MLton] In hope of a release ... In-Reply-To: References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> Message-ID: <162de7480910141547p33853035t4e8922b2220901bf@mail.gmail.com> On Wed, Oct 14, 2009 at 11:57 PM, Matthew Fluet wrote: > It is fortunate that the ports have been going so smoothly. I would > have cautioned against it ... introducing new, and mostly untested, > platforms shortly before a release is not a recipe for stability. > Well, I'd argue that porting an otherwise portable project to new platforms usually helps turn up hidden bugs that can bite you later. Those two alignment issues, for example. There's also a segfault on kfreebsd-amd64, which I'll wager also stems from an actual bug in MLton (supposing I can find it). Of the remaining debian targets: mips(el) -- known to work because I've cross-compiled it to build embedded stuff for ages, but no porter machine is available with debian that I can bootstrap it on. hoping to find someone to loan me shell access. arm/m68k -- insufficient memory, so not included hurd-i386 -- adding hurd support would require actual new code, so no sense trying to get this in There are a few other items that I would like to finish before > releasing. One is the performance bugs identified by Reactive > Systems. If it's reproducible I can take a whack at it. Which issue is this? link? > Another is a regression failure with signals on Mac OS X > 10.5 (Leopard), which I believe is due to a broken sigaltstack, but > have had difficulty isolating. > What's the failure mode? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mlton.org/pipermail/mlton/attachments/20091015/c0c848be/attachment.htm From matthew.fluet at gmail.com Thu Oct 15 06:56:38 2009 From: matthew.fluet at gmail.com (Matthew Fluet) Date: Thu Oct 15 06:57:12 2009 Subject: [MLton] latest MLton segfault in gmp In-Reply-To: <162de7480910141530n56865d40r1336e77e9397e3f7@mail.gmail.com> References: <509955.22208.qm@web82404.mail.mud.yahoo.com> <162de7480910110708s1ea2ea90t5a038fcdac6ae129@mail.gmail.com> <162de7480910110715y49b3b1a3v7d478986935755c4@mail.gmail.com> <87vdijnojy.fsf@mid.deneb.enyo.de> <162de7480910131354r244fd724tbb66e633f9664759@mail.gmail.com> <87skdm35xa.fsf@mid.deneb.enyo.de> <162de7480910140257g7e6bfa46lfb3ce618ce62a5d3@mail.gmail.com> <162de7480910141530n56865d40r1336e77e9397e3f7@mail.gmail.com> Message-ID: On Wed, Oct 14, 2009 at 6:30 PM, Wesley W. Terpstra wrote: > On Wed, Oct 14, 2009 at 11:56 PM, Matthew Fluet > wrote: >> In the next subsection (p. 13), on aggregates and unions, it states: >> >> ?An array uses the same alignment as its elements, except that a >> local or global array variable of length at least 16 bytes or a C99 >> variable-length >> array variable always has alignment of at least 16 bytes. > > I think by global/local arrays they mean arrays not in the heap but the data > segment. (local = static int64_t foo[4];, global = extern int64_t foo[4];) > > At any rate, this sounds like we don't need to worry because MLton only > passes arrays as pointers (both FFI and GMP limb structure). Agreed. I missed that it was "array *variable* of length at least 16 bytes", not any array (memory object). >> I did want to also point out that there is a legacy issue, I would >> assume, on Debian. ?Since mlton-20070826 is dynamically linked against >> libgmp, isn't it just an incredible luck of the draw that a >> self-compile with mlton-20070826 didn't happen to produce a >> non-16-byte aligned IntInf array. > > Yes, I was surprised too. However there are a couple reasons this worked > out. First, the only code gcc managed to vectorize in the gmp C is the > MPN_ZERO method. Second, the only place MPN_ZERO gets called (for us) is to > clear the low bits of a left-shifted intinf. Third, it won't use 16-byte > writes unless there are 16-bytes to write, so it had to be a >=128-bit left > shift. I wonder if these maybe didn't happen in 20070826? I'll bet it is r6253 (20071209): http://mlton.org/cgi-bin/viewsvn.cgi?rev=6253&view=rev That limits constant folding to IntInfs in the range -2^128 -- 2^128, and introduces an explicit calculation of 2^128 as IntInf.<<(1,0w128). From wesley at terpstra.ca Wed Oct 21 17:51:03 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Wed Oct 21 17:51:38 2009 Subject: [MLton] Re: In hope of a release ... In-Reply-To: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> Message-ID: <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> On Tue, Oct 13, 2009 at 10:17 AM, Wesley W. Terpstra wrote: > To that end I'm going to start pushing svn builds of MLton to > debian/experimental and try to get it built on more of the debian > architectures. I'm done. The status and build logs (including regressions) can be found at: https://buildd.debian.org/~luk/status/package.php?p=mlton The only architecture MLton isn't working on is arm[el]. These systems haven't enough memory and their floating point isn't rounded by a control word, so I don't see MLton in their future. Regressions pass completely on all targets except hurd-i386 where http://bugs.debian.org/551470 prevents some regressions from completing. From henry.cejtin at sbcglobal.net Wed Oct 21 21:50:18 2009 From: henry.cejtin at sbcglobal.net (Henry Cejtin) Date: Wed Oct 21 21:50:53 2009 Subject: [MLton] Re: In hope of a release ... In-Reply-To: <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> Message-ID: <997498.18962.qm@web82405.mail.mud.yahoo.com> Truly excellent. I can't wait to get updated MLtons on my Debian testing machine via a simple apt-get. Thanks for all the work. ----- Original Message ---- From: Wesley W. Terpstra To: mlton@mlton.org Sent: Wednesday, October 21, 2009 7:51:03 PM Subject: [MLton] Re: In hope of a release ... On Tue, Oct 13, 2009 at 10:17 AM, Wesley W. Terpstra wrote: > To that end I'm going to start pushing svn builds of MLton to > debian/experimental and try to get it built on more of the debian > architectures. I'm done. The status and build logs (including regressions) can be found at: https://buildd.debian.org/~luk/status/package.php?p=mlton The only architecture MLton isn't working on is arm[el]. These systems haven't enough memory and their floating point isn't rounded by a control word, so I don't see MLton in their future. Regressions pass completely on all targets except hurd-i386 where http://bugs.debian.org/551470 prevents some regressions from completing. _______________________________________________ MLton mailing list MLton@mlton.org http://mlton.org/mailman/listinfo/mlton From henry.cejtin at sbcglobal.net Thu Oct 22 14:41:46 2009 From: henry.cejtin at sbcglobal.net (Henry Cejtin) Date: Thu Oct 22 14:42:42 2009 Subject: [MLton] Is this really legal to allow? Message-ID: <269148.75277.qm@web82404.mail.mud.yahoo.com> I see that MLton (both the 20070826 version and the current SVN version) consider the following to be legal: val x: char vector = "Is this legal?" I know, of course, that in MLton CharVector.vector is just the same as char vector, but is it legal to make that visible? The argument against this is that the resulting code certainly won't be compiled by most ML compilers. From adam at spicenitz.org Thu Oct 22 18:46:41 2009 From: adam at spicenitz.org (Adam Goode) Date: Thu Oct 22 18:46:48 2009 Subject: [MLton] Re: In hope of a release ... In-Reply-To: <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> Message-ID: <4AE10B01.9050402@spicenitz.org> On 10/21/2009 08:51 PM, Wesley W. Terpstra wrote: > On Tue, Oct 13, 2009 at 10:17 AM, Wesley W. Terpstra wrote: >> To that end I'm going to start pushing svn builds of MLton to >> debian/experimental and try to get it built on more of the debian >> architectures. > > I'm done. The status and build logs (including regressions) can be found at: > https://buildd.debian.org/~luk/status/package.php?p=mlton > > The only architecture MLton isn't working on is arm[el]. These systems > haven't enough memory and their floating point isn't rounded by a > control word, so I don't see MLton in their future. > Hmm, the Fedora builders (running on Marvell hardware) definitely have enough memory (1GHz machines, 2GB RAM). I agree about the floating point though, I put a patch into MLton to throw an exception if setting the rounding mode is not available, but due to limitations in glibc on some ARM hardware, this can silently fail to set. But it was definitely fun getting it to build: http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=12752 And only a factor of 5 slowdown in build time! Adam -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature Url : http://mlton.org/pipermail/mlton/attachments/20091022/c921601c/signature.pgp From wesley at terpstra.ca Thu Oct 22 19:11:48 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Thu Oct 22 19:12:22 2009 Subject: [MLton] Re: In hope of a release ... In-Reply-To: <4AE10B01.9050402@spicenitz.org> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> <4AE10B01.9050402@spicenitz.org> Message-ID: <162de7480910221911s5ccb17cbrdcab6835a7d20d38@mail.gmail.com> On Fri, Oct 23, 2009 at 3:46 AM, Adam Goode wrote: >> The only architecture MLton isn't working on is arm[el]. These systems >> haven't enough memory and their floating point isn't rounded by a >> control word, so I don't see MLton in their future. > > Hmm, the Fedora builders (running on Marvell hardware) definitely have > enough memory (1GHz machines, 2GB RAM). The debian arm buildd have 512MB. If they had 1GB I would've probably ported it to them anyway. ;) > I put a patch into MLton to throw an exception if setting the > rounding mode is not available, but due to limitations in glibc on some > ARM hardware, this can silently fail to set. I was under the impression that arm assembler instructions embedded the rounding mode into their opcode. ie: mul_round_to_zero %r0,4. Are you saying some arm processors have 'normal' IEEE operations that consult a control word? Why didn't you submit the patches needed by your port for inclusion? > http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=12752 How many archs does fedora have buildds for? From adam at spicenitz.org Thu Oct 22 19:44:41 2009 From: adam at spicenitz.org (Adam Goode) Date: Thu Oct 22 19:44:48 2009 Subject: [MLton] Re: In hope of a release ... In-Reply-To: <162de7480910221911s5ccb17cbrdcab6835a7d20d38@mail.gmail.com> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> <4AE10B01.9050402@spicenitz.org> <162de7480910221911s5ccb17cbrdcab6835a7d20d38@mail.gmail.com> Message-ID: <4AE11899.6050302@spicenitz.org> On 10/22/2009 10:11 PM, Wesley W. Terpstra wrote: > On Fri, Oct 23, 2009 at 3:46 AM, Adam Goode wrote: > The debian arm buildd have 512MB. If they had 1GB I would've probably > ported it to them anyway. ;) You should see if you can get Marvell to donate some machines to Debian. They run the Fedora ARM builders currently. > I was under the impression that arm assembler instructions embedded > the rounding mode into their opcode. ie: mul_round_to_zero %r0,4. Are > you saying some arm processors have 'normal' IEEE operations that > consult a control word? Well, there are 2 things. First, in software floating point (the default), rounding mode is optional. Only round to zero is required. With hardware VFP, there is a control word. But as an optimization, you get 2 sets of instructions: ones that honor the control word, and ones that always to round to zero. The round to zero instructions are only used in the cases where C requires it (for casting), so that you don't need to save/restore the rounding mode for casting. Otherwise, the rounding mode should be as in IEEE. We use fesetround from C99, but there is a bug in how glibc handles this on ARM, depending on how things were compiled and what the hardware is: Behavior of fesetround with non-default rounding: Compiled for softfloat, no VFP hardware: returns failure (good!) Compiled for VFP, no VFP hardware: fails to run (good!) Compiled for VFP, VFP hardware present: sets rounding mode (good!) Compiled for softfloat, VFP hardware present: silent failure (bad!) glibc will always set the VFP if it is present, even if floating point is software. This can be fixed with making libgcc dynamically linked, and picking the correct floating point library at runtime instead of compile time, but this is not done yet. More background here: http://sourceware.org/ml/libc-ports/2009-04/msg00020.html > > Why didn't you submit the patches needed by your port for inclusion? > They're in there: r7081, r7082. It broke some minor things that were fixed up later by others. >> http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=12752 > > How many archs does fedora have buildds for? > http://fedoraproject.org/wiki/Architectures Primary Architectures x86 (32-bit) x86_64 (64-bit) ppc (32-bit) ppc64 (64-bit) Secondary Architectures ARM IA64 (inactive) S390 SPARC Parisc (inactive?) As of Fedora 13, ppc/ppc64 is going to be a secondary arch too. Releases don't get delayed by problems in secondary arches. As a related note, your work will be great for the secondary arches in Fedora, especially S390 which I started to look at but I didn't have access to hardware. I would love to have a new release! Adam -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature Url : http://mlton.org/pipermail/mlton/attachments/20091022/3cc8dd85/signature.pgp From wesley at terpstra.ca Thu Oct 22 23:35:59 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Thu Oct 22 23:36:34 2009 Subject: [MLton] Re: In hope of a release ... In-Reply-To: <4AE11899.6050302@spicenitz.org> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> <4AE10B01.9050402@spicenitz.org> <162de7480910221911s5ccb17cbrdcab6835a7d20d38@mail.gmail.com> <4AE11899.6050302@spicenitz.org> Message-ID: <162de7480910222335o6bd8c45dpa2fc221eadf06e34@mail.gmail.com> On Fri, Oct 23, 2009 at 4:44 AM, Adam Goode wrote: > Well, there are 2 things. First, in software floating point (the > default), rounding mode is optional. Only round to zero is required. > With hardware VFP, there is a control word. But as an optimization, you > get 2 sets of instructions: ones that honor the control word, and ones > that always to round to zero. The round to zero instructions are only > used in the cases where C requires it (for casting), so that you don't > need to save/restore the rounding mode for casting. Otherwise, the > rounding mode should be as in IEEE. This sounds much bettter than I'd thought. The software version sounds like it would be non-conforming re: the basis. The hardware version can support a full MLton port. > We use fesetround from C99, but there is a bug in how glibc handles this > on ARM, depending on how things were compiled and what the hardware is: > > ?Behavior of fesetround with non-default rounding: > > ? Compiled for softfloat, no VFP hardware: returns failure (good!) > ? Compiled for VFP, no VFP hardware: fails to run (good!) > ? Compiled for VFP, VFP hardware present: sets rounding mode (good!) > ? Compiled for softfloat, VFP hardware present: silent failure (bad!) So if a program is compiled to use hardware floating point there is no problem aside from excluding the weaker machines? Is the presence of VFP correlated with the presence of >=256MB of RAM? If yes, then could we just assume any machine using MLton has VFP hardware? > glibc will always set the VFP if it is present, even if floating point > is software. This can be fixed with making libgcc dynamically linked, > and picking the correct floating point library at runtime instead of > compile time, but this is not done yet. So if you specified software float and there is hardware available, programs transparently start using hardware? The fesetround then fails b/c it doesn't set the hardware control word only the flag the software floating point sees..? Another solution might be to check for VFP hardware in fesetround and if present set the control word. fesetround() always needs inline assembler anyway. > http://sourceware.org/ml/libc-ports/2009-04/msg00020.html > They're in there: r7081, r7082. Of course they are. -.- > Primary Architectures > ?ppc64 (64-bit) Has MLton ever been compiled on ppc64? > As a related note, your work will be great for the secondary arches in > Fedora, especially S390 which I started to look at but I didn't have > access to hardware. I would love to have a new release! Debian has s390 porter boxes. You can probably request access from the relevant admin if you wanted it. Incidentally, s390 is *fast*. It compiles MLton faster than kfreebsd-i386 despite lacking a native codegen. From ville at laurikari.net Thu Oct 22 23:38:15 2009 From: ville at laurikari.net (Ville Laurikari) Date: Thu Oct 22 23:38:37 2009 Subject: [MLton] Re: In hope of a release ... In-Reply-To: <162de7480910222335o6bd8c45dpa2fc221eadf06e34@mail.gmail.com> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> <4AE10B01.9050402@spicenitz.org> <162de7480910221911s5ccb17cbrdcab6835a7d20d38@mail.gmail.com> <4AE11899.6050302@spicenitz.org> <162de7480910222335o6bd8c45dpa2fc221eadf06e34@mail.gmail.com> Message-ID: <20091023063815.GB30280@laurikari.net> On Fri, Oct 23, 2009 at 08:35:59AM +0200, Wesley W. Terpstra wrote: > Has MLton ever been compiled on ppc64? On ppc64 AIX, yes. Don't know about Linux. -- http://hashedbits.com/ From matthew.fluet at gmail.com Sat Oct 24 11:45:41 2009 From: matthew.fluet at gmail.com (Matthew Fluet) Date: Sat Oct 24 11:46:13 2009 Subject: [MLton] Is this really legal to allow? In-Reply-To: <269148.75277.qm@web82404.mail.mud.yahoo.com> References: <269148.75277.qm@web82404.mail.mud.yahoo.com> Message-ID: I don't know that it is illegal. Most SML compilers expose the fact that LargeInt == IntInf, though, arguably, the Basis Library asserts that they are both opaquely sealed. In any case, I don't think the fix is easy in MLton. String constants are overloaded and handled by the compiler, and so must elaborate to a type known by the compiler, which only knows about the primitive types 'char' and 'vector'. There isn't an easy way to get the abstract String.string type (as would arise if we didn't include enough sharing/where-type constraints in the signature match that exports the basis library) "into" the compiler as the type for string constant elaboration. On Thu, Oct 22, 2009 at 5:41 PM, Henry Cejtin wrote: > I ? see ? that ? MLton ? (both ?the 20070826 version and the current SVN > version) consider the following to be legal: > > ? ?val x: char vector = "Is this legal?" > > I know, of course, that in MLton CharVector.vector is just the same ? as > char vector, but is it legal to make that visible? > > The ? argument ? against ? this ? is ?that ?the resulting code certainly > won't be compiled by most ML compilers. > > _______________________________________________ > MLton mailing list > MLton@mlton.org > http://mlton.org/mailman/listinfo/mlton > From adam at spicenitz.org Mon Oct 26 18:02:36 2009 From: adam at spicenitz.org (Adam Goode) Date: Mon Oct 26 18:02:46 2009 Subject: [MLton] Re: In hope of a release ... In-Reply-To: <162de7480910222335o6bd8c45dpa2fc221eadf06e34@mail.gmail.com> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> <4AE10B01.9050402@spicenitz.org> <162de7480910221911s5ccb17cbrdcab6835a7d20d38@mail.gmail.com> <4AE11899.6050302@spicenitz.org> <162de7480910222335o6bd8c45dpa2fc221eadf06e34@mail.gmail.com> Message-ID: <4AE654BC.1060707@spicenitz.org> On 10/23/2009 02:35 AM, Wesley W. Terpstra wrote: > This sounds much bettter than I'd thought. The software version sounds > like it would be non-conforming re: the basis. The hardware version > can support a full MLton port. > Well, I would say it could be barely conforming, since the basis appears to tolerate implementations to throw an exception rather than implement rounding modes. > So if a program is compiled to use hardware floating point there is no > problem aside from excluding the weaker machines? Is the presence of > VFP correlated with the presence of >=256MB of RAM? If yes, then could > we just assume any machine using MLton has VFP hardware? > Unfortunately, VFP is still very much an optional feature. In fact, if you have an XScale, I don't think it is even available. > So if you specified software float and there is hardware available, > programs transparently start using hardware? The fesetround then fails > b/c it doesn't set the hardware control word only the flag the > software floating point sees..? There is no transparent use yet, but this is possible when glibc is compiled in multilib mode. There are a few issues to work out, but it could be done. I looked at it months ago, but nothing recently. This would solve the problem with rounding modes: if you have hardware, you'll automatically use it, if not, fesetround will fail and MLton will throw the exception. Right now if you have hardware, fesetround will succeed but the rounding mode will be effectively ignored. > > Another solution might be to check for VFP hardware in fesetround and > if present set the control word. fesetround() always needs inline > assembler anyway. > Yes, that is what fesetround does. Unfortunately, it can't know if you're actually going to use the VFP from then on! By default, you don't, so the rounding mode is set in the hardware you're not using. > > Has MLton ever been compiled on ppc64? > I have not compiled it there. I suppose I could try to get an account somewhere, but haven't gotten around to it. > Debian has s390 porter boxes. You can probably request access from the > relevant admin if you wanted it. Incidentally, s390 is *fast*. It > compiles MLton faster than kfreebsd-i386 despite lacking a native > codegen. Nice! I should check to see if Fedora offers these things. The emulator is SLOW. Adam -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature Url : http://mlton.org/pipermail/mlton/attachments/20091026/734f058c/signature.pgp From wesley at terpstra.ca Mon Oct 26 23:02:04 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Mon Oct 26 23:02:09 2009 Subject: [MLton] Re: In hope of a release ... In-Reply-To: <4AE654BC.1060707@spicenitz.org> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> <4AE10B01.9050402@spicenitz.org> <162de7480910221911s5ccb17cbrdcab6835a7d20d38@mail.gmail.com> <4AE11899.6050302@spicenitz.org> <162de7480910222335o6bd8c45dpa2fc221eadf06e34@mail.gmail.com> <4AE654BC.1060707@spicenitz.org> Message-ID: <162de7480910270002o5b8a2ee7i2f64ac9f1adbe414@mail.gmail.com> On Tue, Oct 27, 2009 at 4:02 AM, Adam Goode wrote: >> So if you specified software float and there is hardware available, >> programs transparently start using hardware? The fesetround then fails >> b/c it doesn't set the hardware control word only the flag the >> software floating point sees..? > > There is no transparent use yet, but this is possible when glibc is > compiled in multilib mode. There are a few issues to work out, but it > could be done. I looked at it months ago, but nothing recently. This > would solve the problem with rounding modes: if you have hardware, > you'll automatically use it, if not, fesetround will fail and MLton will > throw the exception. Right now if you have hardware, fesetround will > succeed but the rounding mode will be effectively ignored. I'm still puzzled as to why the software floating point doesn't support fesetround. The software emulation could test its own 'control word' (aka global variable) as easily as (easier than?) hardware. From adam at spicenitz.org Tue Oct 27 17:02:30 2009 From: adam at spicenitz.org (Adam Goode) Date: Tue Oct 27 17:02:39 2009 Subject: [MLton] Re: In hope of a release ... In-Reply-To: <162de7480910270002o5b8a2ee7i2f64ac9f1adbe414@mail.gmail.com> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> <4AE10B01.9050402@spicenitz.org> <162de7480910221911s5ccb17cbrdcab6835a7d20d38@mail.gmail.com> <4AE11899.6050302@spicenitz.org> <162de7480910222335o6bd8c45dpa2fc221eadf06e34@mail.gmail.com> <4AE654BC.1060707@spicenitz.org> <162de7480910270002o5b8a2ee7i2f64ac9f1adbe414@mail.gmail.com> Message-ID: <4AE79826.6090204@spicenitz.org> On 10/27/2009 03:02 AM, Wesley W. Terpstra wrote: > On Tue, Oct 27, 2009 at 4:02 AM, Adam Goode wrote: >>> So if you specified software float and there is hardware available, >>> programs transparently start using hardware? The fesetround then fails >>> b/c it doesn't set the hardware control word only the flag the >>> software floating point sees..? >> >> There is no transparent use yet, but this is possible when glibc is >> compiled in multilib mode. There are a few issues to work out, but it >> could be done. I looked at it months ago, but nothing recently. This >> would solve the problem with rounding modes: if you have hardware, >> you'll automatically use it, if not, fesetround will fail and MLton will >> throw the exception. Right now if you have hardware, fesetround will >> succeed but the rounding mode will be effectively ignored. > > I'm still puzzled as to why the software floating point doesn't > support fesetround. The software emulation could test its own 'control > word' (aka global variable) as easily as (easier than?) hardware. The software floating point only supports default rounding, since that is all that is required for ARM EABI compliance. For gcc, it is the implementation here, once described to me as "magic": http://gcc.gnu.org/viewcvs/trunk/gcc/config/arm/ieee754-sf.S?revision=148210&view=markup http://gcc.gnu.org/viewcvs/trunk/gcc/config/arm/ieee754-df.S?revision=148210&view=markup Adam -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature Url : http://mlton.org/pipermail/mlton/attachments/20091027/0f31f522/signature.pgp From adam at spicenitz.org Tue Oct 27 17:03:34 2009 From: adam at spicenitz.org (Adam Goode) Date: Tue Oct 27 17:03:39 2009 Subject: [MLton] Re: In hope of a release ... In-Reply-To: <162de7480910270002o5b8a2ee7i2f64ac9f1adbe414@mail.gmail.com> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> <4AE10B01.9050402@spicenitz.org> <162de7480910221911s5ccb17cbrdcab6835a7d20d38@mail.gmail.com> <4AE11899.6050302@spicenitz.org> <162de7480910222335o6bd8c45dpa2fc221eadf06e34@mail.gmail.com> <4AE654BC.1060707@spicenitz.org> <162de7480910270002o5b8a2ee7i2f64ac9f1adbe414@mail.gmail.com> Message-ID: <4AE79866.3010808@spicenitz.org> On 10/27/2009 03:02 AM, Wesley W. Terpstra wrote: > On Tue, Oct 27, 2009 at 4:02 AM, Adam Goode wrote: >>> So if you specified software float and there is hardware available, >>> programs transparently start using hardware? The fesetround then fails >>> b/c it doesn't set the hardware control word only the flag the >>> software floating point sees..? >> >> There is no transparent use yet, but this is possible when glibc is >> compiled in multilib mode. There are a few issues to work out, but it >> could be done. I looked at it months ago, but nothing recently. This >> would solve the problem with rounding modes: if you have hardware, >> you'll automatically use it, if not, fesetround will fail and MLton will >> throw the exception. Right now if you have hardware, fesetround will >> succeed but the rounding mode will be effectively ignored. > > I'm still puzzled as to why the software floating point doesn't > support fesetround. The software emulation could test its own 'control > word' (aka global variable) as easily as (easier than?) hardware. And remember, "Only the default rounding mode is intended for best performances" Adam -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature Url : http://mlton.org/pipermail/mlton/attachments/20091027/596ca261/signature.pgp From wesley at terpstra.ca Wed Oct 28 04:44:33 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Wed Oct 28 04:45:13 2009 Subject: [MLton] Re: In hope of a release ... In-Reply-To: <4AE79826.6090204@spicenitz.org> References: <162de7480910130117g22581c48td965baa1889ef229@mail.gmail.com> <162de7480910211751m6e57e92bl75dd17e058d9719f@mail.gmail.com> <4AE10B01.9050402@spicenitz.org> <162de7480910221911s5ccb17cbrdcab6835a7d20d38@mail.gmail.com> <4AE11899.6050302@spicenitz.org> <162de7480910222335o6bd8c45dpa2fc221eadf06e34@mail.gmail.com> <4AE654BC.1060707@spicenitz.org> <162de7480910270002o5b8a2ee7i2f64ac9f1adbe414@mail.gmail.com> <4AE79826.6090204@spicenitz.org> Message-ID: <162de7480910280544p772ba26bwcbce5d55081c6022@mail.gmail.com> On Wed, Oct 28, 2009 at 2:02 AM, Adam Goode wrote: > The software floating point only supports default rounding, since that > is all that is required for ARM EABI compliance. For gcc, it > is the implementation here, once described to me as "magic": > > http://gcc.gnu.org/viewcvs/trunk/gcc/config/arm/ieee754-sf.S?revision=148210&view=markup > http://gcc.gnu.org/viewcvs/trunk/gcc/config/arm/ieee754-df.S?revision=148210&view=markup I can see that they tried to heavily optimize it. The SoftFloat package [1] manages to do things in C for all rounding modes. I guess they wanted faster FP. [1] http://www.jhauser.us/arithmetic/SoftFloat.html From wesley at terpstra.ca Sat Oct 31 09:54:14 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Sat Oct 31 09:54:51 2009 Subject: [MLton] {to/from}Poly Bug? Message-ID: <162de7480910311054s4e0e623dj85f77427450dafa8@mail.gmail.com> Maybe I've misunderstood the intention of these methods? terpstra@orange:~$ cat bug.sml val x = Word8Array.array (10, 0w0) val y : Word8.word array = MLton.Word8Array.toPoly x terpstra@orange:~$ mlton bug.sml Error: bug.sml 2.5. Pattern and expression disagree. pattern: [Word8.word] array expression: [MLton.Word8Array.elem] array in: (y): Word8.word array = MLton.Word8Array.toPoly x compilation aborted: parseAndElaborate reported errors From matthew.fluet at gmail.com Sat Oct 31 12:07:46 2009 From: matthew.fluet at gmail.com (Matthew Fluet) Date: Sat Oct 31 12:08:20 2009 Subject: [MLton] {to/from}Poly Bug? In-Reply-To: <162de7480910311054s4e0e623dj85f77427450dafa8@mail.gmail.com> References: <162de7480910311054s4e0e623dj85f77427450dafa8@mail.gmail.com> Message-ID: No misunderstanding. There seems to be some missing sharing constraints in the signature that defines the basis library. The following is untested, but it or something like it should do the trick: Index: basis-library/libs/basis-extra/top-level/basis.sig =================================================================== --- basis-library/libs/basis-extra/top-level/basis.sig (revision 7269) +++ basis-library/libs/basis-extra/top-level/basis.sig (working copy) @@ -717,9 +717,15 @@ sharing type Word64Array2.vector = Word64Vector.vector sharing type MLton.BinIO.instream = BinIO.instream sharing type MLton.BinIO.outstream = BinIO.outstream + sharing type MLton.CharArray.elem = CharArray.elem + sharing type MLton.CharArray.t = CharArray.array + sharing type MLton.CharVector.elem = CharVector.elem + sharing type MLton.CharVector.t = CharVector.vector sharing type MLton.TextIO.instream = TextIO.instream sharing type MLton.TextIO.outstream = TextIO.outstream + sharing type MLton.Word8Array.elem = Word8Array.elem sharing type MLton.Word8Array.t = Word8Array.array + sharing type MLton.Word8Vector.elem = Word8Vector.elem sharing type MLton.Word8Vector.t = Word8Vector.vector end On Sat, Oct 31, 2009 at 1:54 PM, Wesley W. Terpstra wrote: > Maybe I've misunderstood the intention of these methods? > > terpstra@orange:~$ cat bug.sml > val x = Word8Array.array (10, 0w0) > val y : Word8.word array = MLton.Word8Array.toPoly x > > terpstra@orange:~$ mlton bug.sml > Error: bug.sml 2.5. > ?Pattern and expression disagree. > ? ?pattern: ? ?[Word8.word] array > ? ?expression: [MLton.Word8Array.elem] array > ? ?in: (y): Word8.word array = MLton.Word8Array.toPoly x > compilation aborted: parseAndElaborate reported errors > > _______________________________________________ > MLton mailing list > MLton@mlton.org > http://mlton.org/mailman/listinfo/mlton > From hansel at reactive-systems.com Sat Oct 31 18:43:08 2009 From: hansel at reactive-systems.com (David Hansel) Date: Sat Oct 31 18:43:43 2009 Subject: [MLton] Possible bug in MLton-7230.msi for MinGW and 64 bit questions Message-ID: <4AECF5BC.6050400@reactive-systems.com> Hi all, We recently tried compiling our code with the MLton-7230 MinGW experimental release from your web site. Our code had been working fine when compiled using the "MLton MLTONVERSION (built Tue Feb 03 16:33:30 2009 on orange)" build. It compiled fine with the 7230 release but the generated executable always crashed immediately after starting. We found that the crashes only occurred when using the FFI interface and that they always happened during calls to "dlopen". (line 142 in MLton\lib\mlton\sml\mlnlffi-lib\memory\linkage-dlopen.sml) After some more digging we found a change between the two versions regarding the RTLD_* constants used in file MLton\lib\mlton\sml\mlnlffi-lib\memory\linkage-libdl.sml For the older version, the constants defined in that file were val RTLD_LAZY = 0wx00001 val RTLD_NOW = 0wx00002 val RTLD_GLOBAL = 0wx00100 val RTLD_LOCAL = 0wx00000 whereas with the 7230 version, the constants from MLton\lib\mlton\sml\mlnlffi-lib\memory\platform\rtld-flags.mingw.sml are used which are defined as val RTLD_GLOBAL = 0wx2 val RTLD_LAZY = 0wx0 val RTLD_LOCAL = 0wx4 val RTLD_NOW = 0wx0 After changing the constants in rtld-flags.mingw.sml to the values from the previous version, the crashes went away. Our code calls DynLinkage.open_lib with parameters "global=true" and "lazy=true", so the values of the constants definitely come into play. Should we be using different parameters or is this a bug in MLton? Also, from the web site it sounds like the MLton-7230.msi version supports generating Windows 64 bit executables, is that true? If so, how do we tell MLton to generate a 64 bit executable? If not, can you tell us what the current state of Windows 64 bit support for MLton is? Thank you very much and best regards, David -- ---------------------------------------------------------- David Hansel http://www.reactive-systems.com/ OpenPGP (GnuPG) public key file: http://www.reactive-systems.com/~hansel/pgp_public_key.txt ---------------------------------------------------------- From wesley at terpstra.ca Sat Oct 31 18:59:05 2009 From: wesley at terpstra.ca (Wesley W. Terpstra) Date: Sat Oct 31 18:59:41 2009 Subject: [MLton] Possible bug in MLton-7230.msi for MinGW and 64 bit questions In-Reply-To: <4AECF5BC.6050400@reactive-systems.com> References: <4AECF5BC.6050400@reactive-systems.com> Message-ID: <162de7480910311959v774a675buc7824df2bdb9479e@mail.gmail.com> On Sun, Nov 1, 2009 at 3:43 AM, David Hansel wrote: > After changing the constants in rtld-flags.mingw.sml to the values from the previous > version, ?the crashes went away. This sounds to be a bug in the libdl we are getting from google. I was planning to build a new msi this weekend anyway so I'll take a look tomorrow. > Also, ?from the web site it sounds like the MLton-7230.msi version supports > generating Windows 64 bit executables, ?is that true? ?If so, ?how do we tell > MLton to generate a 64 bit executable? mlton -target x86_64-w64-mingw32 your-mlb-file-here.mlb