[MLton-user] *** glibc detected *** realloc(): invalid pointer

[Nicolas Bertolotti]

I faced the exact same issue with MLton 2005.

This is actually not a bug in GMP but an overflow in the IntInf implementation.

The broken piece of code is located in the file int-inf.c :
<
static inline void initRes (__mpz_struct *mpzp, uint bytes) {
        ...
        mpzp->_mp_alloc = (gcState.limitPlusSlop - (pointer)bp->limbs) / 4;
        ...
}
<

The overflow occurs on machines with a large amount of RAM (it caused some crashes in our product on some recent 64 bits machines while running the product in 32 bits mode) when the "-" operation overflows and leads to a negative result.

In the latest version of MLton (I think the change is related to the 64 bits port), this piece of code has been replaced with :
<
res->_mp_alloc = (s->limitPlusSlop - (pointer)bp->obj.body.limbs) / (sizeof(mp_limb_t));
<
Here, the overflow may still occur. Anyway, as the result is then divided by "sizeof(mp_limb_t)" which is an unsigned value, the final result is correct.

A simple fix for you is to replace < 4 > with < 4U > in your version.

... but a fix that would completely eliminate the overflow would be :
        mpzp->_mp_alloc = ((uintptr_t) gcState.limitPlusSlop - (uintptr_t)bp->limbs) / 4U;

Hope this helps.

Nicolas


________________________________
From: mlton-user-bounces at mlton.org [mailto:mlton-user-bounces at mlton.org] On Behalf Of Wesley W. Terpstra
Sent: Tuesday, August 19, 2008 11:48 PM
To: mlton-user at mlton.org
Subject: [MLton-user] *** glibc detected *** realloc(): invalid pointer

Has anyone else noticed that gmp seems to be broken as of today? My system hasn't changed, and I downgraded to MLton 20061107 on my completely normal etch machine. However, when I try to execute MLton-generated executables now, I often get the error:
*** glibc detected *** realloc(): invalid pointer: 0x6e8ecdc8 ***
which seems to be coming from gmp:
Program received signal SIGABRT, Aborted.
0xffffe405 in __kernel_vsyscall ()
(gdb) bt
#0  0xffffe405 in __kernel_vsyscall ()
#1  0x55609811 in raise () from /lib/tls/i686/cmov/libc.so.6
#2  0x5560afb9 in abort () from /lib/tls/i686/cmov/libc.so.6
#3  0x5563edfa in __fsetlocking () from /lib/tls/i686/cmov/libc.so.6
#4  0x55649063 in realloc () from /lib/tls/i686/cmov/libc.so.6
#5  0x556490b1 in realloc () from /lib/tls/i686/cmov/libc.so.6
#6  0x55648dcc in realloc () from /lib/tls/i686/cmov/libc.so.6
#7  0x55581559 in __gmp_default_reallocate () from /usr/lib/libgmp.so.3
#8  0x55596916 in __gmpz_realloc () from /usr/lib/libgmp.so.3
#9  0x55593965 in __gmpz_mul () from /usr/lib/libgmp.so.3
#10 0x0899c3af in ?? ()
...

What I find particularly worrying is that AFAIK, when MLton calls gmpz_mul, gmp should NEVER need to realloc the integer..? Is this some sort of Y2008 bug in gmp? Has anyone else started experiencing it as of today? It doesn't always happen to me, but trying to build MLton seems a sure-fire way to trigger it.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mlton.org/pipermail/mlton-user/attachments/20080819/7aaf20a7/attachment.html