Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005179OCamlOCaml generalpublic2010-11-24 09:142013-08-31 12:49
Reportercullmann 
Assigned To 
PrioritynormalSeverityfeatureReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version3.12.1+dev 
Target VersionFixed in Version3.13.0+dev 
Summary0005179: Port OCaml to mingw-w64 to have 64bit OCaml without MSVC Toolchain
DescriptionHi,

I tried to compile OCaml 3.12 on Windows with mingw-w64, the 64bit successor of the old mingw toolchain (which is based on gcc 4.5 and higher).

The 32bit version of mingw-w64 compiles ocaml okay (even if using msys/mingw-64, as now they support @reponsefile and co.)

The 64bit version lacks the generation of 64bit gasm and flexlink support.
I patched flexlink to work (at least it seems so) and patched a bit the asm stuff to have a linking variant, still got segfaults, guess I mixed it up somewhere.

Anyone willing to help me in the porting effort?
Our company will test the port extensively after being done by using it for some dev version of Astree.

Attached current diffs to 3.12-SVN.

Btw., mingw-w64 project: http://mingw-w64.sourceforge.net/ [^]
TagsNo tags attached.
Attached Filesdiff file icon ocaml.3.12.diff [^] (53,021 bytes) 2010-11-24 09:14 [Show Content]
diff file icon flexdll.0.26.diff [^] (4,661 bytes) 2010-11-24 09:15 [Show Content]
patch file icon flexdll-dev-r195+mingw64.patch [^] (9,618 bytes) 2011-06-11 18:04 [Show Content]
patch file icon ocaml-3.12-r11067+mingw64.patch [^] (66,074 bytes) 2011-06-11 18:05 [Show Content]
patch file icon amd64.S.patch [^] (2,182 bytes) 2011-12-22 09:27 [Show Content]
patch file icon Makefile.several.patch [^] (607 bytes) 2011-12-22 10:04 [Show Content]

- Relationships
duplicate of 0005478closedfrisch ocamlopt assumes ar command exists 

-  Notes
(0006004)
Antoine Mine (reporter)
2011-06-11 18:34

Hi,

For what it's worth, I've managed to compile a recent OCaml 3.12 dev branch (svn revision r11067) that generates MinGW64 code, with the patches I just added.

It requires a recent Cygwin with the mingw64-x86_64 gcc packages, a patch to flexdll (against svn revision r195), a working 32-bit OCaml to compile the patched flexdll, and a 64-bit version of ActiveTcl. The compilation is then performed as usual, using the new config/Makeifle.mingw64 instead of config/Makefile.mingw. By the way, the patch also updates the MinGW (non-64) port to use the new mingw64-i686 32-bit gcc packages.

Unfortunately, I will not be able to give much support to a MinGW64 OCaml as I don't intend to use it personally. Furthermore, there seems to be no more interest in maintaining a MinGW64 version of Astrée. I'm releasing this as-is in case it might be useful for others, and will gladly answer any question regarding the submitted patches

-
Antoine
(0006234)
frisch (developer)
2011-12-12 09:38

Antoine: thanks for this patch.

Concerning the 32-bit "mingw" package: do you have any reason to prefer the i686-w64-mingw32-gcc toolchain (from the mingw64 project) over the i686-pc-mingw32-gcc one (from the mingw project)? Both are available under Windows, and it's not clear to me which one should be preferred.
(0006235)
frisch (developer)
2011-12-12 10:57
edited on: 2011-12-12 10:57

Antoine, concerning your changes to win32caml/: what is the reason for changing GetWindowLongPtr to GetWindowLong? According to:

http://msdn.microsoft.com/en-us/library/windows/desktop/ms633584%28v=vs.85%29.aspx [^]

<<
Note: If you are retrieving a pointer or a handle, this function has been superseded by the GetWindowLongPtr function. (Pointers and handles are 32 bits on 32-bit Windows and 64 bits on 64-bit Windows.) To write code that is compatible with both 32-bit and 64-bit versions of Windows, use GetWindowLongPtr.
>>

(0006236)
frisch (developer)
2011-12-12 12:41

flexdll 0.27 is now based on the mingw-w64 project (for its mingw and mingw64 toolchains).

Commit 11300: the offical (32-bit) mingw port of OCaml is now based on this compiler. The 64-bit mingw port has not been committed yet.
(0006237)
Camarade_Tux (reporter)
2011-12-12 12:53

I think I've already stated this but it's worth mentionning it again: I've found mingw-w64 to be _much_ more responsive than mingw.org. Actually, when I tried compiling ocaml with mingw-w64 for the first time, I stumbled on 3 bugs in mingw-w-4 in the process; all have been resolved in less than a day (the first one being solved in a matter of minutes).
(0006246)
Antoine Mine (reporter)
2011-12-12 18:53

Hi Alain,

That's nice to see this patch used!

I had no real preference on i686-w64-mingw32-gcc over i686-pc-mingw32-gcc. I guess I picked the first one out of a sense of symmetry! There are so many toolchains now in mingw that I sometimes get confused...

Strangely enough, I had crashes of ocamlwin until I changed the GetWindowLongPtr into GetWindowLong, not knowing it was bad to do it (but it worked better!). If there are still crashes using the right thing (i.e. GetWindowLongPtr), then maybe the problem is elsewhere (variable types, casts, or strange magic in inria.h)?
(0006425)
shadinger (reporter)
2011-12-20 22:55

Hi all,

With the help of Antoine's patches and the new version 0.27 of flexdll I was able to compile ocaml 3.12.1 on mingw64-x64.

The detailed instructions are posted here: https://github.com/shadinger/opalang/wiki/Compiling-ocaml-MinGW64-w64 [^]

This is required for the windows port of opalang which I am trying to compile on mingw64.

I had problems with Antoine's first patch as it raised an assertion failure on some code in emit.ml (line 577 - assert(i.arg.(0).loc <> Reg 9); (* not %r11 *)). I tried several combinations and ended up with Antoine's code for amd64.S, and emil.ml code from ocaml 3.13.0-devel8 which partially support mingw64.

The patched ocaml-win64 is here in github: https://github.com/shadinger/ocaml-win64 [^]

Thanks again to Alain and Antoine.

@Alain: I had an error while using flexlink 0.27 from the binary package (Fatal Error:hd). I had to recompile it with mingw64. I didn't check further.
(0006469)
xleroy (administrator)
2011-12-21 17:43

This afternoon, in a fit of boredom, I rebooted my home PC under Windows 7, resisted the urge to play Portal 2, installed Cygwin, then Emacs, then FlexDLL, then felt a little less lonely, then merged together the various OCaml/mingw64 patches floating around, and added a bit of house dressing on it.

The result is now in the SVN trunk, commit 11927. It mostly works, but not 100% yet:
+ can compile a working ocamlopt.opt
+ runs correctly a number of classic tests, e.g. KB
- miscompiles tests/misc/sorts.ml (ocamlopt-generated executable crashes after a while)
- miscompiles tests/misc/weaktest.ml (ocamlopt-generated executable does an Invalid_argument)
- does not complete "make opt" because of a crash while compiling camlp4 (IIRC) - does not complete "make opt.opt" because of a crash while compiling ocamlbuild (IIRC).

In the last two cases, it could be a stack overflow or something more sinister. I'm unable to investigate since Cygwin's gdb doesn't work over Win64 executables, and I don't (want to) know of any Windows alternative for an asm-level debugger.

So, now, if this 5th Windows port of OCaml is to see public light, someone else will have to pick it up where I'm leaving it, and commit to maintain it in the future. Enjoy!

(0006474)
shadinger (reporter)
2011-12-21 22:21

Great job Xavier!

I have just corrected a hard to find bug on my repository about calling conventions when calling C code in 64 bits mode with more than 4 arguments. The stack arguments were disaligned.

In "asmrun/amd64.S", I really think you don't need the PREPARE_FOR_C_CALL because the stack alignment is already taken care of in emit.ml. I had some weird crashes on native code while bytecode was working ok. When removing the code reserving additional 32 bytes on the stack, everything went fine.

By the way, gdb does work on Mingw64/MSYS for 64 bits which is useful because windows silently hides the segfaults - WinDBG64 from MS is also free and easy to use.

I will pick up soon where you left.
(0006486)
xleroy (administrator)
2011-12-22 09:54

You're right that the PREPARE_FOR_C_CALL in camll_c_call is wrong because the Caml caller already set up the stack exactly like the C callee expects it. Fix committed in trunk (commit 11931). That could explain some of my mysterious crashes; I'll try to retest this evening.

The other occurrences of PREPARE_FOR_C_CALL are needed, I believe. Esp. the one in caml_call_gc, as explained in PR#5008.
(0006487)
shadinger (reporter)
2011-12-22 10:12
edited on: 2011-12-22 12:10

After removing the PREPARE_FOR_C_CALL, the whole compilation process went ok on Mingw64-w64/MSYS out of the box. Excellent news!

At leat all tests/misc are passing now (sorts.ml and weaktest.ml are ok both in bytecode and native). Yet many other tests do not pass.

I have not tested using Mingw64/cygwin.

Please dismiss my submitted patch for amd64.S, I must have been too brutal in removing completely PREPARE_FOR_C_CALL.

I have posted also a small patch for Makefile.several to ignore windows line ending in test results. The Mingw64 versions output files using Windows convention, and reference files are using Unix conventions.

***edit: You also need to add the "--strip-trailing-cr" option in testsuite/makefiles/Makefile.one (diff command appears twice).

(0006545)
xleroy (administrator)
2011-12-28 11:31

Thanks for the testing. I think we can declare success!

Concerning the test suite, it is safe to say that it isn't ready for use with the Mingw & MSVC ports of OCaml... As a first step, commit 11965 should make it possible to override the DIFF variable with something appropriate for Windows.
(0006590)
monate (reporter)
2012-01-04 14:42

Hi, thanks a lot for your efforts on the windows mess side.

The documentation in Makefile.win32 should state that one has to install the package named mingw64-i686-gcc-core in addition to binutils, gcc and runtime. Otherwise the command gcc was not installed :-(
 
With revision 11984, "make -f Makefile.nt opt" fails for both 32 and 64 bits versions with my settings.
Here the exact error with some debugging stuff added:
=====================
make[1]: Leaving directory `/home/lmonate/ocaml-svn/asmrun'
cd stdlib ; make -f Makefile.nt allopt
make[1]: Entering directory `/home/lmonate/ocaml-svn/stdlib'
../boot/ocamlrun ../ocamlopt -verbose -a -o stdlib.cmxa pervasives.cmx array.cmx list.cmx char.cmx string.cmx sys.cmx hashtbl.cmx sort.cmx marshal.cmx obj.cmx int32.cmx int64.cmx nativeint.cmx lexing.cmx parsing.cmx set.cmx map.cmx stack.cmx queue.cmx camlinternalLazy.cmx lazy.cmx stream.cmx buffer.cmx printf.cmx format.cmx scanf.cmx arg.cmx printexc.cmx gc.cmx digest.cmx random.cmx callback.cmx camlinternalOO.cmx oo.cmx camlinternalMod.cmx genlex.cmx weak.cmx filename.cmx complex.cmx arrayLabels.cmx listLabels.cmx stringLabels.cmx moreLabels.cmx stdLabels.cmx
+ ar rc "stdlib.a" "pervasives.o" "array.o" "list.o" "char.o" "string.o" "sys.o" "hashtbl.o" "sort.o" "marshal.o" "obj.o" "int32.o" "int64.o" "nativeint.o" "lexing.o" "parsing.o" "set.o" "map.o" "stack.o" "queue.o" "camlinternalLazy.o" "lazy.o" "stream.o" "buffer.o" "printf.o" "format.o" "scanf.o" "arg.o" "printexc.o" "gc.o" "digest.o" "random.o" "callback.o" "camlinternalOO.o" "oo.o" "camlinternalMod.o" "genlex.o" "weak.o" "filename.o" "complex.o" "arrayLabels.o" "listLabels.o" "stringLabels.o" "moreLabels.o" "stdLabels.o"
'ar' n'est pas reconnu en tant que commande interne
ou externe, un programme ex?cutable ou un fichier de commandes.
File "_none_", line 1:
Error: Error while creating the library stdlib.a
Makefile.shared:58: recipe for target `stdlib.cmxa' failed
make[1]: *** [stdlib.cmxa] Error 2
make[1]: Leaving directory `/home/lmonate/ocaml-svn/stdlib'
Makefile.nt:528: recipe for target `libraryopt' failed
make: *** [libraryopt] Error 2
=========================

The executable ar is not prefixed by the "standard" x86_64-w64-mingw32/i686_w64-mingw32 stuff.
The first cuplrit that comes to mind is line 77 in utils/ccomp.ml.
Do you need more help to investigate?
(0006679)
xleroy (administrator)
2012-01-14 19:46

Benjamin's "ar" problem also reported in PR#5478. Let's continue the discussion under this other PR.
(0007408)
johnwhitington (reporter)
2012-05-03 16:53

Hi!

Just a quick note to say that I've managed to build a 64 bit mingw ocaml on Windows 7 following shadinger's instructions and using his downloads on github.

I've just sent a new executable of our PDF command line tools to a customer who's trying to merge 15000 PDF files together, and is running into memory limits with the 32 bit executable. We'll see how he gets on!

My preliminary tests here show the executable seems to work.

The only things I found missing from the instructions:

a) User needs to remember to add flexdll to the PATH or ocaml won't build
b) The "set OCAMLLIB=" in the instructions stops the user being able to use ocaml after he has built and installed it: solution is to close the msys command prompt and open a new one.

Thanks for all your hard work.

John Whitington
Coherent Graphics Ltd
(0007409)
shadinger (reporter)
2012-05-03 21:07

John, thanks for the kind note.

You are right, I will add and clarify the compilation instructions. I still need to find time to check 3.13.0.

You may also need to tweak the stack size when compiling your application, especially with high memory usage.

The bad thing with Win7 64 bits, is that a stack overflow generates a silent crash of the app whithout any apparent reason. You need to run it through GDB ou Windbg to really see if it's a stack overflow.

- Issue History
Date Modified Username Field Change
2010-11-24 09:14 cullmann New Issue
2010-11-24 09:14 cullmann File Added: ocaml.3.12.diff
2010-11-24 09:15 cullmann File Added: flexdll.0.26.diff
2011-05-20 14:52 doligez Status new => assigned
2011-05-20 14:52 doligez Assigned To => xleroy
2011-06-11 18:04 Antoine Mine File Added: flexdll-dev-r195+mingw64.patch
2011-06-11 18:05 Antoine Mine File Added: ocaml-3.12-r11067+mingw64.patch
2011-06-11 18:34 Antoine Mine Note Added: 0006004
2011-09-21 13:34 doligez Assigned To xleroy => doligez
2011-12-12 09:38 frisch Note Added: 0006234
2011-12-12 10:49 frisch Assigned To doligez => frisch
2011-12-12 10:57 frisch Note Added: 0006235
2011-12-12 10:57 frisch Note Edited: 0006235 View Revisions
2011-12-12 12:41 frisch Note Added: 0006236
2011-12-12 12:53 Camarade_Tux Note Added: 0006237
2011-12-12 18:53 Antoine Mine Note Added: 0006246
2011-12-17 11:20 frisch Assigned To frisch =>
2011-12-20 22:55 shadinger Note Added: 0006425
2011-12-21 17:43 xleroy Note Added: 0006469
2011-12-21 17:43 xleroy Status assigned => feedback
2011-12-21 22:21 shadinger Note Added: 0006474
2011-12-22 09:27 shadinger File Added: amd64.S.patch
2011-12-22 09:54 xleroy Note Added: 0006486
2011-12-22 10:04 shadinger File Added: Makefile.several.patch
2011-12-22 10:12 shadinger Note Added: 0006487
2011-12-22 12:10 shadinger Note Edited: 0006487 View Revisions
2011-12-28 11:31 xleroy Note Added: 0006545
2011-12-28 11:31 xleroy Status feedback => resolved
2011-12-28 11:31 xleroy Resolution open => fixed
2011-12-28 11:31 xleroy Fixed in Version => 3.13.0+dev
2012-01-04 14:42 monate Note Added: 0006590
2012-01-14 19:45 xleroy Relationship added duplicate of 0005478
2012-01-14 19:46 xleroy Note Added: 0006679
2012-05-03 16:53 johnwhitington Note Added: 0007408
2012-05-03 21:07 shadinger Note Added: 0007409
2013-08-31 12:49 xleroy Status resolved => closed


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker