[MLton] Re: [MLton-commit] r6699

[Matthew Fluet]

On Wed, 17 Jun 2009, Wesley W. Terpstra wrote:
> On Wed, Jun 17, 2009 at 2:25 PM, Matthew Fluet<fluet at tti-c.org> wrote:
>> It seems that there are multiple, independent(?) issues.  Certainly one is
>> that a Cygwin program doesn't like to be started via CreateProcess with a
>> non-NULL lpEnvironment argument; I can't find any documentation for this,
>> but I can definitely demonstrate it with some short C programs.
>
> ... great. Is this a cygwin bug? This seems a pretty serious issue.
>
> At any rate, the solution seems clear except that lpEnvironment =>
> cygwin implosion. One possible work-around would be to temporarily
> change the calling process' environment before invoking CreateProcess.
> I don't find anyone else reporting this problem on google, though. Are
> you sure?

Here's my testing:

[fluet at winxp-cygwin tmp]$ cat child.c
#include <stdlib.h>
#include <stdio.h>

int main(int argc, const char* argv[]) {
   fprintf(stderr, "HW! [stderr]\n");
   fprintf(stdout, "HW! [stdout]\n");
   exit (5);
}
[fluet at winxp-cygwin tmp]$ gcc -Wall -o child.exe child.c

[fluet at winxp-cygwin tmp]$ cat parent.c
#include <windows.h>
#include <stdio.h>

#define BUFSIZE 4096

int main (int argc, const char* argv[]) {
   LPCTSTR modnamep = NULL;
   TCHAR modname[BUFSIZE];
   if (argc > 1) {
     lstrcpy(modname, argv[1]);
     printf("modname = %s\n", modname);
     modnamep = modname;
   } else { return 0; }
   LPTSTR cmdlinep = NULL;
   TCHAR cmdline[BUFSIZE];
   if (argc > 2) {
     lstrcpy(cmdline, argv[2]);
     printf("cmdline = %s\n", cmdline);
     cmdlinep = cmdline;
   }

   LPTSTR envp = NULL;
   TCHAR env[BUFSIZE];
   if (argc > 3) {
     envp = env;
     int i = 3;
     LPSTR curp = env;
     while (argc > i) {
       lstrcpy (curp, argv[i]);
       printf("env[%i] = %s\n", i - 3, argv[i]);
       curp += lstrlen(curp) + 1;
       i++;
     }
     *curp = (TCHAR)0;
   }

   STARTUPINFO siStartInfo;
   ZeroMemory(&siStartInfo, sizeof(STARTUPINFO));
   siStartInfo.cb = sizeof(STARTUPINFO);

   PROCESS_INFORMATION piProcInfo;
   ZeroMemory(&piProcInfo, sizeof(PROCESS_INFORMATION));

   BOOL bSuccess = FALSE;
   printf("CreateProcess\n");
   bSuccess = CreateProcess(modnamep,      // module name
                            cmdlinep,      // command line
                            NULL,          // process security attributes
                            NULL,          // primary thread security attributes
                            TRUE,          // handles are inherited
                            0,             // creation flags
                            envp,          // use parent's environment
                            NULL,          // use parent's current directory
                            &siStartInfo,  // STARTUPINFO pointer
                            &piProcInfo);  // receives PROCESS_INFORMATION
   if ( ! bSuccess ) {
     printf("CreateProcess failed (%ld).\n", (long int)(GetLastError()));
     return 0;
   }
   printf("piProcInfo.hProcess = %ld\n", (long int)(piProcInfo.hProcess));

   DWORD status = 0;
   WaitForSingleObject(piProcInfo.hProcess, INFINITE);
   GetExitCodeProcess(piProcInfo.hProcess, &status);
   printf("status: %ld\n", status);

   CloseHandle(piProcInfo.hProcess);
   CloseHandle(piProcInfo.hThread);
   return 1;
}
[fluet at winxp-cygwin tmp]$ gcc -Wall -o parent.exe parent.c

[fluet at winxp-cygwin tmp]$ ./parent.exe 'C:\WINDOWS\system32\finger.exe' 'C:\WINDOWS\system32\finger.exe -l fluet at mlton.org'
modname = C:\WINDOWS\system32\finger.exe
cmdline = C:\WINDOWS\system32\finger.exe -l fluet at mlton.org
CreateProcess
piProcInfo.hProcess = 1856
[mlton.org]

> Finger: connect::Connection refused
status: 0
[fluet at winxp-cygwin tmp]$ ./parent.exe 'C:\WINDOWS\system32\finger.exe' 'C:\WINDOWS\system32\finger.exe -l fluet at mlton.org' 'FOO=bar'
modname = C:\WINDOWS\system32\finger.exe
cmdline = C:\WINDOWS\system32\finger.exe -l fluet at mlton.org
env[0] = FOO=bar
CreateProcess
piProcInfo.hProcess = 1856
Unknown host: mlton.org

status: 0

So, finger.exe is unhappy about not having the full environment (or, 
rather, the DNS resolver is unhappy), but the command runs as expected.

[fluet at winxp-cygwin tmp]$ ./parent.exe 'C:\WINDOWS\system32\cmd.exe' 'C:\WINDOWS\system32\cmd.exe /c set'
modname = C:\WINDOWS\system32\cmd.exe
cmdline = C:\WINDOWS\system32\cmd.exe /c set
CreateProcess
piProcInfo.hProcess = 1856
COMSPEC=C:\WINDOWS\system32\cmd.exe
PATH=C:\cygwin\home\fluet\bin;...elided...;c:\WINDOWS\system32\WindowsPowerShell\v1.0;C:\cygwin\bin
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.JS;.WS
PROMPT=$P$G
SYSTEMDRIVE=C:
SYSTEMROOT=C:\WINDOWS
WINDIR=C:\WINDOWS
status: 0
[fluet at winxp-cygwin tmp]$ ./parent.exe 'C:\WINDOWS\system32\cmd.exe' 'C:\WINDOWS\system32\cmd.exe /c set' 'FOO=bar'
modname = C:\WINDOWS\system32\cmd.exe
cmdline = C:\WINDOWS\system32\cmd.exe /c set
env[0] = FOO=bar
CreateProcess
piProcInfo.hProcess = 1856
COMSPEC=C:\WINDOWS\system32\cmd.exe
FOO=bar
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.JS;.WS
PROMPT=$P$G
status: 0

So, cmd.exe sees the variables named in an explicit lpEnvironment (and 
loses the environment of the parent; I suspect that COMSPEC, PATHEXT, and 
PROPMT are baked into cmd.exe).

On the other hand:

[fluet at winxp-cygwin tmp]$ ./parent.exe 'C:\cygwin\home\fluet\tmp\child.exe' 'C:\cygwin\home\fluet\tmp\child.exe 1 2 3'
modname = C:\cygwin\home\fluet\tmp\child.exe
cmdline = C:\cygwin\home\fluet\tmp\child.exe 1 2 3
CreateProcess
piProcInfo.hProcess = 1848
HW! [stderr]
HW! [stdout]
status: 5
[fluet at winxp-cygwin tmp]$ ./parent.exe 'C:\cygwin\home\fluet\tmp\child.exe' 'C:\cygwin\home\fluet\tmp\child.exe 1 2 3' 'FOO=bar'
modname = C:\cygwin\home\fluet\tmp\child.exe
cmdline = C:\cygwin\home\fluet\tmp\child.exe 1 2 3
env[0] = FOO=bar
CreateProcess
piProcInfo.hProcess = 1848
status: -1073741515

A little more investigation reveals that a cygwin program is unhappy 
unless C:\cygwin\bin is in the PATH environment variable:

[fluet at winxp-cygwin tmp]$ ./parent.exe 'C:\cygwin\home\fluet\tmp\child.exe' 'C:\cygwin\home\fluet\tmp\child.exe 1 2 3' 'PATH=C:\cygwin\bin'
modname = C:\cygwin\home\fluet\tmp\child.exe
cmdline = C:\cygwin\home\fluet\tmp\child.exe 1 2 3
env[0] = PATH=C:\cygwin\bin
CreateProcess
piProcInfo.hProcess = 1844
HW! [stderr]
HW! [stdout]
status: 5
[fluet at winxp-cygwin tmp]$ ./parent.exe 'C:\cygwin\home\fluet\tmp\child.exe' 'C:\cygwin\home\fluet\tmp\child.exe 1 2 3' 'PATH=C:\WINDOWS\system32'
modname = C:\cygwin\home\fluet\tmp\child.exe
cmdline = C:\cygwin\home\fluet\tmp\child.exe 1 2 3
env[0] = PATH=C:\WINDOWS\system32
CreateProcess
piProcInfo.hProcess = 1848
status: -1073741515

And, with the exception of needing C:\cygwin\bin in the PATH, we are able 
to manage the environment:

[fluet at winxp-cygwin tmp]$ cat environ.sml
val () =
    List.app
    (fn s => (print (String.toString s); print "\n"))
    (Posix.ProcEnv.environ ())
[fluet at winxp-cygwin tmp]$ mlton -output environ.exe environ.sml
[fluet at winxp-cygwin tmp]$ ./parent.exe 'C:\cygwin\home\fluet\tmp\environ.exe' 'C:\cygwin\home\fluet\tmp\environ.exe'
modname = C:\cygwin\home\fluet\tmp\environ.exe
cmdline = C:\cygwin\home\fluet\tmp\environ.exe
CreateProcess
piProcInfo.hProcess = 1848
PATH=/home/fluet/bin:...elided...:/cygdrive/c/WINDOWS/system32/WindowsPowerShell/v1.0:/usr/bin
SYSTEMDRIVE=C:
SYSTEMROOT=C:\\WINDOWS
WINDIR=C:\\WINDOWS
TERM=cygwin
HOME=/home/fluet
status: 0
[fluet at winxp-cygwin tmp]$ ./parent.exe 'C:\cygwin\home\fluet\tmp\environ.exe' 'C:\cygwin\home\fluet\tmp\environ.exe' 'PATH=C:\cygwin\bin' 'FOO=bar' 'BAR=foo'
modname = C:\cygwin\home\fluet\tmp\environ.exe
cmdline = C:\cygwin\home\fluet\tmp\environ.exe
env[0] = PATH=C:\cygwin\bin
env[1] = FOO=bar
env[2] = BAR=foo
CreateProcess
piProcInfo.hProcess = 1848
PATH=/usr/bin
FOO=bar
BAR=foo
TERM=cygwin
HOME=/home/fluet
status: 0

This also explains why the current implementation of create, that copies 
the whole environment via Posix.ProcEnv.environ (which would appear to 
have a PATH entry with the appropriate directory) fails.  The 'problem' is 
that the PATH environment variable returned by Posix.ProcEnv.environ has 
been cygwinified: it has '/usr/bin', not 'C:\cygwin\bin'.  Of course, 
'/usr/bin' means nothing to CreateProcess/Windows, which I'm guessing is 
using the PATH to find cygwin1.dll (in C:\cygwin\bin) for a cygwin 
program.  So, I suspect that the exit status -1073741515 corresponds to a 
dll load failure.  finger.exe and cmd.exe are only dependent upon core 
system dlls, which presumably can be found even in the absence of a PATH.

All in all, I think the way forward is clear:

  * create {env = NONE, ...} should result in calling CreateProcess with a
    NULL lpEnvironment; this is, by far, the most common case I would
    think.

  * create {env = SOME [...], ...} should behave as currently implemented;
    the user is responsible for establishing the correct environment (it is
    just the case that the correct environment for a Cygwin program is
    subtle)