[MLton] QuickCheck and xUnit in SML (MLton library project)

[Vesa Karvonen]

Quoting Gergely Buday <gbuday at gmail.com>:
> >    - Arbitrary (generate random values of arbitrary SML types; inspired
> >      by the QuickCheck library by Koen Claessen and John Hughes),
> 
> See
> 
> http://contrapunctus.net/league/haques/qcheck/

I've been aware of QCheck/SML.  At the time when I first encountered it,
I wasn't quite satisfied with the interface it presents to the user, but I
also couldn't see any way to do significantly better in SML.  Compared to
the QuickCheck/Haskell library, QCheck/SML seems considerably more verbose
to use.  For example, if you want QCheck/SML to show counter examples, you
need to separately provide a pretty printing function.  Also, there is
nothing like the Arbitrary type class allowing one to obtain a default
test data generator almost for free (by just specifying the type).  Rather,
the specification of test data generators is done using the lower level Gen
interface.

These problems can be alleviated by using a type-indexed approach.  The
necessary show function (roughly 'a -> string) and the default generator
(roughly rng -> 'a) can be specified simultaneously as a single combined
type-index.  This is implemented using techniques similar to what I
describe in my note posted to the MLton-user list:

   http://mlton.org/pipermail/mlton-user/2006-August/000907.html

(I have since developed the technique further.  The STRUCTURAL_TYPE
signature used in my above post has one bug (namely regExn does not allow
to reconstruct exceptions).  I have also devised some other improvements.)

Here is a sketch of how the familiar rev properties could be specified
using the UnitTest framework:

local
   open Type UnitTest
in
   val () =
       unitTests
          (title "rev")

          (check int
                 |< pred (fn x => rev [x] = [x]))

          (check (list int * list int)
                 |< trivial (fn xs & ys => null xs orelse null ys)
                 |< pred (fn xs & ys => rev (xs @ ys) = rev ys @ rev xs))

          (check (list int)
                 |< classify (fn xs =>
                                 case length xs of
                                    0 => SOME "length 0"
                                  | 1 => SOME "length 1-2"
                                  | 2 => SOME "length 1-2"
                                  | 3 => SOME "length 3-5"
                                  | 4 => SOME "length 3-5"
                                  | 5 => SOME "length 3-5"
                                  | _ => NONE)
                 |< pred (fn xs => rev (rev xs) = xs))

          $
end

A few words about the above.  The unitTest -function begins a fold.  The
expression (title "rev") specifies the title used for subsequent tests
(both ad hoc and randomized) - there is no need to specify a distinct
title for every test.  Three properties are checked above.  The first
parameter to the check -function is a type-index and the second is the
property to be checked.  |< is just the right (associative) pipe function.
>From the above specification, the framework would already be able to show
counter examples in nicely formatted SML syntax.  The type-indexed arbitrary
function is actually used through the combined type-index by the check
function.  (The trivial, classify, and pred fns are copied from the
QCheck/SML example code.)

The UnitTest module also supports xUnit-style ad hoc testing.  For example,
a simple equality test can be written as:

local
   open Type UnitTest
in
   val () =
       unitTests
          (title "Foo")

          (testEq (list int)
                  (fn () =>
                      {expect = [1, 2, 3],
                       actual = [3, 2, 1]}))

          $
end

(Note that one can build local auxiliary functions on top of testEq and other
testing functions, so multiple similar tests can be written more concisely.)

>From the above test case, the UnitTest framework generates the output:

1. Foo test failed.
   Failure: expected [1, 2, 3], but got [3, 2, 1].
   No exception history available.

In the above, the testEq function gets the necessary equality predicate and
the show function through the combined type-index given as the first parameter.

-Vesa Karvonen