[MLton-user] Re: Compiling ML/C Program to dll / Writing a Java GUI for an ML program

[Max Lehn]

Hi,

I am working with Christof on the BubbleStorm project. I'll try to
sketch the way we make our SML-to-Java bindings.

The bindings basically consist of three 'layers' of (wrapper) code:
1. SML wrapper code exporting the SML API code via FFI. The code is
compiled to a dynamic library ("mlton -export-header <header>.h -format
library ...").
2. C code using the exported SML functions and implementing the native
part for JNI. The C code is compiled to a second dynamic (JNI) library
(depending on the first; I am not sure if the two libraries could be
merged).
3. Java code importing the native functions and wrapping them into
nice-to-use Java classes. The Java application imports the two dynamic
libraries.

'Objects' between SML and Java: Java programmers expect a nice
object-oriented API. Although SML is not OOP, an interface using
structures in an 'class-like' fashion allows for a quite good mapping to
Java classes. (Following our convention, a 'class' in SML is a structure
with a type t and functions with an instance of type t as the first
parameter (i.e, a 'method').) Since we cannot move objects between SML
and Java, all SML 'object instances' are passed via handles. For this,
we define one handle-to-object map per 'class' in the SML wrapper part.
(The map is actually a custom implementation based on an array for
efficient access.) When calling a method of an object or passing an
object as a method parameter, we always use its handle.

The second big thing to solve is the thread synchronization for Java.
All typical Java applications use threading, so we have to translate
between our single-threaded, event-based programming model in SML to
Java. (We have active networking code in SML; if your SML parts are just
passive, it may be somewhat easier. The very basic thing, is to prevent
calls to SML from multiple threads at the same time.). Therefore, we
start our SML event loop in a separate thread. Whenever an SML function
is called from Java, the call information is stored in a queue, the SML
part is notified to pull the data and call the function, and the results
are in turn passed back through a queue. A similarly complex
synchronization is necessary for callback functions implemented in Java
and called from SML.
This sounds rather expensive, and probably is. So I wouldn't recommend
it for high performance applications. But we've used the concept already
for some applications which are more than proof-of-concept implementations.

One big drawback of our current solution is the need for a lot of
hand-written wrapper code. So it's really annoying (and also
error-prone) to port complex interfaces. We discussed automatic code
generation, but from my experience there are too many special cases to
make auto-generation really useful.
Furthermore, error handling is sometimes nasty. We built an exception
passing mechanism which throws Java exceptions instead of crashing the
VM on unhandled SML exceptions. But a Java-C-SML stack trace is hardly
readable.
And there are several other tiny things to solve, but so far we have
been able to do (almost) everything we needed.

To conclude, it works, but it is messy in some places. This was the
high-level overview and a disclaimer; if you're still interested, I can
try to provide you the relevant extracts and example code from our project.

Max


Matthew Fluet wrote:
> On Fri, Jul 29, 2011 at 3:57 PM, Steffen J. Smolka
> <steffen.smolka at in.tum.de> wrote:
>> Hi,
>> I have been wondering for a while now if there's a good way to make ML code
>> and Java code work together.
>> What I would love to do is write a program in ML and wrap it with a GUI
>> written in Java/Scala.
>> What I have found so far is that it's possible to make Java work with C
>> (JNI/JNA) and C work with ML (with MLton's Foreign Function Interface).
>> Combining this, I would hope to get Java working with ML (via an interface
>> written in C).
>> However, it seems that JNI/JNA only works with dynamic libraries, that is
>> dlls in Windows.
>> The problem is that i can't get MLton to compile my ML/C program to a
>> dll. Compiling the program to an executable works just fine.
>> Is it just not possible to compile ML programs to dlls with MLton or am I
>> doing something wrong?
>> Or is there possibly a better way to make ML code and Java code work
>> together?
>> I know it's possible to compile the ML programm to a standalone executable
>> and simply call that executable from within Java.
>> The problem is that this would limit data transfer beetween the programs to
>> strings which might require lots of encoding and decoding and doesn't seem
>> like a good solution.
> You could certainly send binary data back and forth to the SML program
> (rather than just text data).  That still requires some amount of
> serialization of complex data.  But, unless you are simply passing
> flat arrays of 32-bit integers (or something similar), you would still
> need some amount of serialization, since Java doesn't know MLton's
> native object format and MLton doesn't know Java's native object
> format.
>
> As for compiling to a DLL, there is some support for compiling a SML
> program to a library.  See "http://mlton.org/LibrarySupport"; I'm not
> sure that it supports compiling to a DLL, since there isn't a native
> Windows port of MLton; on Windows, we use either Cygwin or MinGW.
>
> _______________________________________________
> MLton-user mailing list
> MLton-user at mlton.org
> http://mlton.org/mailman/listinfo/mlton-user


-- 
Max Lehn
Databases and Distributed Systems
Technische Universität Darmstadt
Hochschulstr. 10, 64289 Darmstadt, Germany
Tel.: +49 6151 167425
http://www.dvs.tu-darmstadt.de/