0005465: Increase stack size of ocamlopt.opt for windows

(0006611)
lefessan (developer)
2012-01-06 15:39

Have you tried to use "editbin /STACK:size" on ocamlopt.opt to try to fix the problem ?

I don't know in which Microsoft package "editbin" is available, so it might not be generic solution.

(0006615)
shadinger (reporter)
2012-01-06 16:48

I guess this should work but it would imply to install Visual Studio which is an overkill. I want to stick to Mingw64. I don't know any mingw64 equivalent to EDITBIN.

(0006697)
xleroy (administrator)
2012-01-17 15:48

We should be able to pass this option for all executables produced by ocamlopt. Need to check what we do already for the MSVC ports and transfer this to the MinGW ports.

(0006702)
lefessan (developer)
2012-01-17 21:24

The way it is done for MSVC:

MKEXE=$(FLEXLINK) -exe -link /STACK:16777216

For MSVC64, the stack size is set to 33554432 instead of 16777216.
The MKEXE line should be modified accordingly in config/Makefile.mingw and config/Makefile.mingw64. For Cygwin, it must be done in the configure script, probably by using Unix-like options, by modifying the "gcc*,*-*-cygwin*)" case
when mkexe is defined.

Right now, I have no Windows system to boot on, so a patch tested by a windows-user would be welcome !

(0006723)
frisch (developer)
2012-01-18 11:41

I've implemented a new option -stack in flexlink. It allows to set the stack reserve in the resulting image, in a way independent of the toolchain (it patches the image directly).

In config/Makefile.msvc, we could thus replace:

MKEXE=$(FLEXLINK) -exe -link /STACK:16777216

with

MKEXE=$(FLEXLINK) -exe -stack 16777216

And do the same for the other ports (and in the configure script for Cygwin).

(Or should I use this value as a default in flexlink?)

(0006728)
lefessan (developer)
2012-01-19 13:36

Could you use two different values for 32 bits and 64 bits ?

(0006729)
frisch (developer)
2012-01-19 13:52

> Could you use two different values for 32 bits and 64 bits ?

If I define a default value in flexlink, yes of course. Otherwise, it is just a matter of deciding which number to put in the config/Makefile.*.

(0006742)
shadinger (reporter)
2012-01-19 22:25

Alain, great work.

Is there a way to use flexlink to change a posteriori the stack size of an existing EXE like EDITBIN does? Your ocaml code seems to be able to do it, but I don't know if it is possible from the flexlink CLI.

(0006743)
frisch (developer)
2012-01-19 22:49

> but I don't know if it is possible from the flexlink CLI.

Currently not, but this is a trivial addition. Will do it.

(0007124)
xleroy (administrator)
2012-03-21 15:48

I see a description of a complete fix in the comments to this PR, but I don't see any change in config/Makefile.mingw*. Could someone who cares about the Windows/MinGW do what needs to be done, then resolve this PR? Thanks.

(0007126)
frisch (developer)
2012-03-22 06:25

Xavier: I'll do it. I need to release a new version of flexlink, with the new -stack option, first.

(0007135)
frisch (developer)
2012-03-23 16:20

I've released flexdll 0.28, which adds a -stack option (to set the stack reserve on the target image). One can also do:

flexlink -patch -stack 123456 foo.exe

to set the stack size on an existing image.

I'll update OCaml config Makefiles accordingly.

(0007136)
frisch (developer)
2012-03-23 16:23
edited on: 2012-03-23 16:23

The trunk has been updated (commits 12263, 12264).

shadinger: Can you try it? If you confirm everything is ok, I'll push the change to the 4.00 branch.

(0007137)
shadinger (reporter)
2012-03-23 17:00

Great. I will try beginning of next week.

(0007174)
frisch (developer)
2012-03-26 19:36

Pushed to version/4.00 (commit 12277).

(0007181)
shadinger (reporter)
2012-03-27 08:52

Alain, I have a flexlink fatal error while compiling under Mingw64 very early in the process: compilation of 'byterun' directory ends with:

x86_64-w64-mingw32-gcc -DOCAML_STDLIB_DIR='"C:/ocamlmgw64/lib"' -I"c:\flexdll" -
O -mms-bitfields -Wall -Wno-unused -c prims.c
flexlink -chain mingw64 -exe -o ocamlrun.exe prims.o -lws2_32 libcamlrun.a
** Fatal error: hd
make[1]: *** [ocamlrun.exe] Error 2
make[1]: Leaving directory `/home/Papa/ocaml-win64/byterun'
make: *** [coldstart] Error 2

I tried recompiling flexlink with my own compiler chain, it produces the same error.

I don't have a clue about this 'hd' Fatal error.

(0007183)
frisch (developer)
2012-03-27 09:40

If you recompile flexlink with -g, you should be able to get a stacktrace (with OCAMLRUNPARAM=b=1).

I expect the exception to be raised by Reloc.get_output1. Can you instrument it and see which commands returns an empty result? (maybe "cygpath -S" ?)

(0007184)
shadinger (reporter)
2012-03-27 10:11

I changed Reloc.get_output1 to the following:

let get_output1 ?use_bash cmd =
  Printf.eprintf "** Before %s\n" cmd;
  let res = List.hd (get_output ?use_bash cmd) in
  Printf.eprintf "** After\n";
  res

I get the following result:
** Before x86_64-w64-mingw32-gcc -print-sysroot
** Fatal error: hd

Meanwhile, "x86_64-w64-mingw32-gcc -print-sysroot" or "gcc -print-sysroot" produces an empty result.

(0007185)
shadinger (reporter)
2012-03-27 10:16

Additional info:

The last fully working version for me is producing the following:
** Before x86_64-w64-mingw32-gcc -print-libgcc-file-name
** After

And "x86_64-w64-mingw32-gcc -print-libgcc-file-name" get the following output:
c:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.6.1/libgcc.a

(0007195)
frisch (developer)
2012-03-27 13:18

Which version of mingw64 do you use? Has it been installed as a Cygwin package?

(0007202)
shadinger (reporter)
2012-03-27 18:09

I am using MinGW-w64 tdm64-gcc-4.6.1 with MSYS. It is not a cygwin package. MSYS does a better job than cygwin to solve the directory mess between windows and cygwin when compiling native 64 bits apps (not using the cygwin DLLs).

All the steps are described here:
https://github.com/shadinger/opalang/wiki/Compiling-ocaml-MinGW64-w64 [^]

(0007214)
frisch (developer)
2012-03-28 16:12

I'm not familiar with msys. In particular, I don't know what I should use as a replacement for `gcc -print-sysroot` on this system.

Note that the official toolchain supported by OCaml is mingw64 packaged by Cygwin, not the msys one. If you can contribute a patch to fix flexdll for this system (without breaking the officially supported one), I'll happily integrate it, but unfortunately, I don't have time to work on it myself.

Notes
(0006611) lefessan (developer) 2012-01-06 15:39	Have you tried to use "editbin /STACK:size" on ocamlopt.opt to try to fix the problem ? I don't know in which Microsoft package "editbin" is available, so it might not be generic solution.

(0006615) shadinger (reporter) 2012-01-06 16:48	I guess this should work but it would imply to install Visual Studio which is an overkill. I want to stick to Mingw64. I don't know any mingw64 equivalent to EDITBIN.

(0006697) xleroy (administrator) 2012-01-17 15:48	We should be able to pass this option for all executables produced by ocamlopt. Need to check what we do already for the MSVC ports and transfer this to the MinGW ports.

(0006702) lefessan (developer) 2012-01-17 21:24	The way it is done for MSVC: MKEXE=$(FLEXLINK) -exe -link /STACK:16777216 For MSVC64, the stack size is set to 33554432 instead of 16777216. The MKEXE line should be modified accordingly in config/Makefile.mingw and config/Makefile.mingw64. For Cygwin, it must be done in the configure script, probably by using Unix-like options, by modifying the "gcc,--cygwin)" case when mkexe is defined. Right now, I have no Windows system to boot on, so a patch tested by a windows-user would be welcome !

(0006723) frisch (developer) 2012-01-18 11:41	I've implemented a new option -stack in flexlink. It allows to set the stack reserve in the resulting image, in a way independent of the toolchain (it patches the image directly). In config/Makefile.msvc, we could thus replace: MKEXE=$(FLEXLINK) -exe -link /STACK:16777216 with MKEXE=$(FLEXLINK) -exe -stack 16777216 And do the same for the other ports (and in the configure script for Cygwin). (Or should I use this value as a default in flexlink?)

(0006728) lefessan (developer) 2012-01-19 13:36	Could you use two different values for 32 bits and 64 bits ?

(0006729) frisch (developer) 2012-01-19 13:52	> Could you use two different values for 32 bits and 64 bits ? If I define a default value in flexlink, yes of course. Otherwise, it is just a matter of deciding which number to put in the config/Makefile.*.

(0006742) shadinger (reporter) 2012-01-19 22:25	Alain, great work. Is there a way to use flexlink to change a posteriori the stack size of an existing EXE like EDITBIN does? Your ocaml code seems to be able to do it, but I don't know if it is possible from the flexlink CLI.

(0006743) frisch (developer) 2012-01-19 22:49	> but I don't know if it is possible from the flexlink CLI. Currently not, but this is a trivial addition. Will do it.

(0007124) xleroy (administrator) 2012-03-21 15:48	I see a description of a complete fix in the comments to this PR, but I don't see any change in config/Makefile.mingw*. Could someone who cares about the Windows/MinGW do what needs to be done, then resolve this PR? Thanks.

(0007126) frisch (developer) 2012-03-22 06:25	Xavier: I'll do it. I need to release a new version of flexlink, with the new -stack option, first.

(0007135) frisch (developer) 2012-03-23 16:20	I've released flexdll 0.28, which adds a -stack option (to set the stack reserve on the target image). One can also do: flexlink -patch -stack 123456 foo.exe to set the stack size on an existing image. I'll update OCaml config Makefiles accordingly.

(0007136) frisch (developer) 2012-03-23 16:23 edited on: 2012-03-23 16:23	The trunk has been updated (commits 12263, 12264). shadinger: Can you try it? If you confirm everything is ok, I'll push the change to the 4.00 branch.

(0007137) shadinger (reporter) 2012-03-23 17:00	Great. I will try beginning of next week.

(0007174) frisch (developer) 2012-03-26 19:36	Pushed to version/4.00 (commit 12277).

(0007181) shadinger (reporter) 2012-03-27 08:52	Alain, I have a flexlink fatal error while compiling under Mingw64 very early in the process: compilation of 'byterun' directory ends with: x86_64-w64-mingw32-gcc -DOCAML_STDLIB_DIR='"C:/ocamlmgw64/lib"' -I"c:\flexdll" - O -mms-bitfields -Wall -Wno-unused -c prims.c flexlink -chain mingw64 -exe -o ocamlrun.exe prims.o -lws2_32 libcamlrun.a Fatal error: hd make[1]: * [ocamlrun.exe] Error 2 make[1]: Leaving directory `/home/Papa/ocaml-win64/byterun' make: *** [coldstart] Error 2 I tried recompiling flexlink with my own compiler chain, it produces the same error. I don't have a clue about this 'hd' Fatal error.

(0007183) frisch (developer) 2012-03-27 09:40	If you recompile flexlink with -g, you should be able to get a stacktrace (with OCAMLRUNPARAM=b=1). I expect the exception to be raised by Reloc.get_output1. Can you instrument it and see which commands returns an empty result? (maybe "cygpath -S" ?)

(0007184) shadinger (reporter) 2012-03-27 10:11	I changed Reloc.get_output1 to the following: let get_output1 ?use_bash cmd = Printf.eprintf " Before %s\n" cmd; let res = List.hd (get_output ?use_bash cmd) in Printf.eprintf " After\n"; res I get the following result: Before x86_64-w64-mingw32-gcc -print-sysroot Fatal error: hd Meanwhile, "x86_64-w64-mingw32-gcc -print-sysroot" or "gcc -print-sysroot" produces an empty result.

(0007185) shadinger (reporter) 2012-03-27 10:16	Additional info: The last fully working version for me is producing the following: Before x86_64-w64-mingw32-gcc -print-libgcc-file-name After And "x86_64-w64-mingw32-gcc -print-libgcc-file-name" get the following output: c:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.6.1/libgcc.a

(0007195) frisch (developer) 2012-03-27 13:18	Which version of mingw64 do you use? Has it been installed as a Cygwin package?

(0007202) shadinger (reporter) 2012-03-27 18:09	I am using MinGW-w64 tdm64-gcc-4.6.1 with MSYS. It is not a cygwin package. MSYS does a better job than cygwin to solve the directory mess between windows and cygwin when compiling native 64 bits apps (not using the cygwin DLLs). All the steps are described here: https://github.com/shadinger/opalang/wiki/Compiling-ocaml-MinGW64-w64 [^]

(0007214) frisch (developer) 2012-03-28 16:12	I'm not familiar with msys. In particular, I don't know what I should use as a replacement for `gcc -print-sysroot` on this system. Note that the official toolchain supported by OCaml is mingw64 packaged by Cygwin, not the msys one. If you can contribute a patch to fix flexdll for this system (without breaking the officially supported one), I'll happily integrate it, but unfortunately, I don't have time to work on it myself.

Issue History
Date Modified	Username	Field	Change
2012-01-04 16:30	shadinger	New Issue
2012-01-06 15:39	lefessan	Note Added: 0006611
2012-01-06 16:48	shadinger	Note Added: 0006615
2012-01-17 15:48	xleroy	Note Added: 0006697
2012-01-17 15:48	xleroy	Status	new => confirmed
2012-01-17 21:24	lefessan	Note Added: 0006702
2012-01-18 11:41	frisch	Note Added: 0006723
2012-01-19 13:36	lefessan	Note Added: 0006728
2012-01-19 13:52	frisch	Note Added: 0006729
2012-01-19 22:25	shadinger	Note Added: 0006742
2012-01-19 22:49	frisch	Note Added: 0006743
2012-03-21 15:48	xleroy	Note Added: 0007124
2012-03-22 06:24	frisch	Assigned To	=> frisch
2012-03-22 06:24	frisch	Status	confirmed => assigned
2012-03-22 06:25	frisch	Note Added: 0007126
2012-03-23 16:20	frisch	Note Added: 0007135
2012-03-23 16:23	frisch	Note Added: 0007136
2012-03-23 16:23	frisch	Note Edited: 0007136	View Revisions
2012-03-23 17:00	shadinger	Note Added: 0007137
2012-03-26 19:36	frisch	Note Added: 0007174
2012-03-26 19:37	frisch	Status	assigned => resolved
2012-03-26 19:37	frisch	Resolution	open => fixed
2012-03-27 08:52	shadinger	Note Added: 0007181
2012-03-27 09:40	frisch	Note Added: 0007183
2012-03-27 10:11	shadinger	Note Added: 0007184
2012-03-27 10:16	shadinger	Note Added: 0007185
2012-03-27 13:18	frisch	Note Added: 0007195
2012-03-27 18:09	shadinger	Note Added: 0007202
2012-03-28 16:12	frisch	Note Added: 0007214

Relationships