[MLton] Crashes with 64-bit native code generator on Windows

[David Hansel]

Hi again,

I have a few more observations regarding the crashes that might ring a
bell for some of the MLton developers involved in the native code-generators
and/or FFI interface:

1) In my previous post I included a disassembly of the location where the
crash happens.  With some creative grep-ing I was able to find the location
of that code within the assembly code that MLton produces for our program:

_L_176132:
	movq (_c_stackP+0x0)(%rip),%rsp
	movq 0x40(%rbp),%r14
	movq 0x0(%r14),%r13
	movq 0x0(%r13),%r11
	movl %r15d,%r9d
	xorq %r8,%r8
	movl $0x0,%r15d
	movl %r15d,%edx
	xorq %rcx,%rcx
	subq $0x20,%rsp
	addq $0x40,%rbp
	leaq (_L_176133+0x0)(%rip),%r15
	movq %r15,0xFFFFFFFFFFFFFFF8(%rbp)
	movq %rbp,(_gcState+0x10)(%rip)
	movq %r12,(_gcState+0x0)(%rip)
	call *(_applyFFTempFun+0x0)(%rip)   <<------- CRASH
	addq $0x20,%rsp
	movq (_gcState+0x0)(%rip),%r12
	movq (_gcState+0x10)(%rip),%rbp
	jmp _L_176133

I was able to reproduce this with several examples for which the crash
occurs (all of which unfortunately include a large part of our code so
I can not make them available here).  The crash always occurs in the
"*applyFFTempFun" call and always because applyFFTempFun is NULL.

I am reasonably sure that this actually is the location of the crash and
not just some similar looking code because if I comment out the "call"
statement in the assembly code and then compile the executable from the
assembly code, the crash goes away (the target of the FFI call in question
is an empty function that does not do anything so it is not surprising
that skipping the call does not produce other problems).  Also, if I
introduce an infinite loop right before the call,  the compiled program
hangs right before where I would usually observe the crash.

2) As I mentioned before,  if I compile the program from the SML
code and just insert a "print" statement in function "get" within
MLton/lib/mlton/sml/mlnlffi-lib/memory/linkage-libdl.sml,  the crash
also does not occur.  Interestingly,  the MLton-produced assembly code
for that version (only change is the "print" statement) does not contain
ANY calls to "applyFFTempFun".

3) Looking at the MLton source code (amd64-generate-transfers.fun),  I can see
that calls to "applyFFTempFun" seem to be inserted for "Indirect" FFI calls.
I do not know enough about the code generator or the FFI interface to make
much sense out of this.
However,  I can see that the MLTon-produced code with the crash only contains
a call to "applyFFTempFun" (which I assume is created in line 1566 of file
amd64-generate-transfers.fun) but never any code that would set the value
of "applyFFTempFun" (which I assume should be created in line 1183 of file
amd64-generate-transfers.fun).

Given these observations,  does anyone have any suggestions about MLton
debugging options or other ways to shed more light on what might be going
wrong here?


Thanks,

David