Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0006411OCamlOCaml windowspublic2014-05-12 06:002014-07-17 16:28
Reporterfuruse 
Assigned Tofrisch 
PrioritynormalSeveritymajorReproducibilityalways
StatusresolvedResolutionopen 
PlatformOSOS Version
Product Version4.01.0 
Target Version4.02.0+devFixed in Version4.02.0+dev 
Summary0006411: Win32 port build with Mingw: -link -static-libgcc is required for flexlink call
DescriptionI built OCaml 4.01.0 with mingw64-i686-gcc 4.8.2-2. The build and installation were successful, but I got a linking problem of bigarray.cma:

$ ocaml bigarray.cma
Cannot load required shared library dllbigarray.
Reason: C:/ocamlmgw/lib/stublibs\dllbigarray.dll: The specified module could not be found.
Tagspatch
Attached Files

- Relationships

-  Notes
(0011427)
furuse (reporter)
2014-05-12 06:02

After investigating what is happening I found that the stublib dllbigarray.dll depends on libgcc_s_sjlj-1.dll but it is missing.

Googling about it I found a fix: we need -static-libgcc linker option at the creation of dllbigarray.dll. I do not know exactly which mingw64-i686-gcc versions require this but at least the following patch works fine with 4.8.2-2.

Longer and probably redundant explanation is available at: http://d.hatena.ne.jp/camlspotter/20140502/1399024687 [^]

-----------------

*** ocaml-4.01.0.old/config/Makefile Fri May 2 17:25:36 2014
--- ocaml-4.01.0/config/Makefile Fri May 2 18:17:36 2014
***************
*** 100,106 ****
  CPP=$(BYTECC) -E
  
  ### Flexlink
! FLEXLINK=flexlink -chain mingw -stack 16777216
  FLEXDIR=$(shell $(FLEXLINK) -where)
  IFLEXDIR=-I"$(FLEXDIR)"
  MKDLL=$(FLEXLINK)
--- 100,106 ----
  CPP=$(BYTECC) -E
  
  ### Flexlink
! FLEXLINK=flexlink -chain mingw -stack 16777216 -link -static-libgcc
  FLEXDIR=$(shell $(FLEXLINK) -where)
  IFLEXDIR=-I"$(FLEXDIR)"
  MKDLL=$(FLEXLINK)
(0011437)
protz (manager)
2014-05-13 15:14

Right, but -static-libgcc is not the only way to fix that: you can just dynamically load libgcc_s_sjlj-1.dll by making sure the directory where the dll lives is in the PATH.

jonathan@farquaad:~/Code/ocaml (trunk) $ export PATH=/usr/i686-w64-mingw32/sys-root/mingw/bin/:$PATH

jonathan@farquaad:~/Code/ocaml (trunk) $ ocaml bigarray.cma
                             OCaml version 4.03.0+dev0-2014-05-12

# exit 1;;


I kind of think that the motto these days is that static linking _should_ be avoided, but I don't have enough background to determine whether having Alain add -static-libgcc to the flexlink invocations or having users of OCaml on Windows tweak their path is the right thing to do. Alain, do you have an opinion on that?
(0011439)
protz (manager)
2014-05-13 16:00

After some investigations, it turns out that mmap_win32.c contains an int64 division (line 89) and an int64 modulo (line 109). Both of these lines, when compiled by gcc (http://stackoverflow.com/questions/18448343/divdi3-division-used-for-long-long-by-gcc-on-x86 [^]), generate calls to external library functions (___divi3 and ___modi3), hence requiring libgcc_s_sjlj-1.dll. As far as Damien and I could tell, these are the only places which generate a dependency on the runtime library.

We could probably get away with replacing the division and the modulo by bit-shifting and masking respectively, since the divisor is a power of two in both cases.
(0011454)
frisch (developer)
2014-05-14 12:23

If we can manage to get rid of the dependency to the external library, this seems good to me.

I don't think we can tell users to add a non-default directory to their PATH. Cannot we instruct the linker to add the directory to the dynamic load path when building executables?
(0011456)
furuse (reporter)
2014-05-14 13:53
edited on: 2014-05-14 13:55

If we can remove the dependency completely it is the best.

If we dynamically link it then users of binary executables by OCaml+MinGW may need to install the library, which is not good.

I am not sure about the license of the library but libgcc_s_sjlj-1.dll sounds pretty GNU. Not good for whom want to build closed source OCaml apps.

(0011459)
protz (manager)
2014-05-14 16:37

I believe we all agree that the dependency ought to be removed. The only question is whether we should do it by rewriting the two operations on long long's by hand, or by static-linking libgcc. We would have to figure out the license implications of the latter solution, though. After a quick search, it seems to be fine (http://www.gnu.org/licenses/gcc-exception.html [^]).
(0011497)
doligez (administrator)
2014-05-16 17:49

I vote for rewriting: inelegant, but it's only a few lines.
(0011506)
Camarade_Tux (reporter)
2014-05-16 22:28

> libgcc_s_sjlj-1.dll but it is missing

Your toolchain sounds broken too.
(0011535)
xleroy (administrator)
2014-05-21 17:21

On line 89, "array_size" (the divisor) is definitely not a power of 2.

I probably miss the obvious, but aren't those helper functions __divd3 already used by the runtime system, and if so why aren't they available for DLLs thanks to FlexDLL magic?
(0011602)
shinwell (developer)
2014-05-30 15:02

Alain, are you able to help with this?
(0011613)
frisch (developer)
2014-05-31 18:34

I did not look at the issue concretely, and there might indeed be an issue with flexlink, but if the runtime system depends on those functions and they are provided by a dll which is not in the default dll load path, how is a standalone program (either ocamlrun or a bytecode program compiled with -custom) supposed to work out of the box on a machine without cygwin?
(0011636)
protz (manager)
2014-06-03 14:26

I believe the two options are as follows.
1) flexlink should now pass the -static-libgcc flag at link-time by default. Functions that used to be statically linked with the executable are now in a separate DLL; the -static-libgcc flag reverts to the former behavior. This means that generated programs linked with this flag _will not_ depend on the external DLL.
2) We should rewrite the two operations on long long's "by hand" (seems complicated) so as to not depend on the functions from the now-external DLL.

Alain, do you think option 1) would be ok for you?
(0011638)
frisch (developer)
2014-06-03 14:39

Why should flexlink's default be changed? Wouldn't it be enough to add the option to utils/Makefile.{mingw,mingw6464}?
(0011639)
Camarade_Tux (reporter)
2014-06-03 17:44

As far as I'm concerned, I expect the dependency on this DLL and I don't see what the issue is. If someone wants static linking (especially for C libs), it should be done explicitely.

If I'm right, you're building from Cygwin and knowing who makes the packages, I doubt that DLL is missing. In other words, it sounds like an issue with the library lookup paths. If you're cross-compiling (which you are if you're using Cygwin), it's expected that the file is not in a standard place (and even more so for libgcc on Cygwin which, iirc, is _the_ reason it's not in the usual paths).

I definitely acknowledge the use of static linking and I do so to distribute my yypkg.exe from http://win-builds.org [^] but I don't think it should be the default, especially when you can do it fairly easily.

PS: I vote against rewriting: if there's a whole library and it's built as shared library, it's because there are more than a couple functions and you're likely to pull that library for other symbols and reasons too (especially if you interface with C and C++).
(0011640)
protz (manager)
2014-06-03 17:56

> If I'm right, you're building from Cygwin and knowing who makes the packages, I doubt that DLL is missing.

The DLL is definitely not missing. It's just that I'm unsure which one of switching to static linking or distributing the DLL along with the OCaml binaries is the best choice.
(0011641)
frisch (developer)
2014-06-03 18:04

> As far as I'm concerned, I expect the dependency on this DLL and I don't see what the issue is.

The native ports (mingw / msvc) are supposed to produce standalone programs, i.e. programs which can run on a bare Windows machine. This is not strictly the case for the msvc ports, because they create dependencies to e.g. msvcr90.dll, which are not parts of Windows, but is still widely available (e.g. it is installed with recent .Net). Visual Studio C++ itself has the same problem. Until recently, the mingw port only created a dependency to msvcrt.dll, which is shipped with Windows. It was a very nice property, and it would be bad to loose it.

It seems quite bad to require OCaml users to ship a copy of a Cygwin DLL, even if licensing is ok, together with their native executables.
(0011642)
Camarade_Tux (reporter)
2014-06-03 18:15

It's part of the C toolchain and it comes with the C toolchain and if it's not there, it's an issue with the C toolchain. It's simply in a different path.

For instance, in my cross-toolchains, it's in something like:
/opt/cross_toolchain_32/i686-w64-mingw32/lib/libgcc_s_sjlj-1.dll

And there's even an autotools variable for that place: toolexeclibdir which is used by nothing but GCC and for that shared library.

You can have a look at :
  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=40125 [^]
and
  http://cygwin.com/ml/cygwin/2010-07/msg00187.html [^]

It's really only an issue with the lookup path.

Btw, you CANNOT ship that library because it changes depending on the exception handling system and it's not possible to foresee that.

And if you don't want to change lookup paths, simply copy the file at the end of OCaml's installation on Windows. Crude but effective.
(0011643)
frisch (developer)
2014-06-03 18:18

I'm not sure to follow the discussion. Why should we require people who need to execute a native OCaml program to have a C toolchain? Or are we only talking about making a native OCaml distribution?
(0011644)
Camarade_Tux (reporter)
2014-06-03 18:29

This is a GCC file, definitely not a Cygwin one. The reason cygwin is involved is because the C toolchain is Cygwin's and is a cross-compiler which makes it write that .dll file in a specific location which is compatible with cross-compilation and multilib systems.

If you check the original error message it is for bytecode but it also mentions bigarray which has C stubs and I guess that's how libgcc gets linked in.

Apart from that, it should only matter to native-compiled executables except maybe that ocamlrun.exe itself might depend on it (it seems it currently doesn't or it would have failed earlier than when loading bigarray.cma stubs and if it did, there would have been no reason for the OS to lookup the library again [ and fail that ])

As for distributing additional files, I doubt most applications distribute 0 additional files and if they do, they can explicitely link statically. I'm doing just that in yypkg.
(0011645)
protz (manager)
2014-06-03 23:29

I think we're conflating two issues here.
- Should the OCaml binary distribution (i.e. the OCaml installer) depend on libgcc_s_sjlj-1.dll or not?
  * If we keep the current setup (NO -static-libgcc), we need to ship libgcc_s_sjlj-1.dll along with the installer.
  * If we apply Jun's patch, we do NOT need to ship this dll as it will be statically linked in bigarray.cma
- Should the programs generated by ocamlopt depend on this DLL or not?
  * If we keep the current behavior of flexlink, programs generated by ocamlopt with a recent enough mingw64 toolchain WILL depend on either libgcc_s_sjlj-1.dll or libgcc_s_dwarf2-whatever.dll depending on the exact toolchain used by the user, so we need to instruct users to ship this DLL with their ocamlopt-compiled programs.
  * If we do change the behavior of flexlink and make it use -static-libgcc then programs generated by ocamlopt will not depend on the external DLL. This will incidentally solve the first point.
(0011647)
frisch (developer)
2014-06-04 09:04

> - Should the programs generated by ocamlopt depend on this DLL or not?

I think this is the most important question. The decision about the first one (OCaml binary distribution) can be made by the guys creating the Windows binary distributions; it's more a packaging issue.

> If we do change the behavior of flexlink and make it use -static-libgcc then programs generated by ocamlopt will not depend on the external DLL.

I still don't understand why we need to change the default behavior of flexlink. It seems to me that Jun's patch would also work. The FLEXLINK variable in config/Makefile.{mingw,msvc,...} affects how ocamlopt invokes flexlink.
(0011872)
shinwell (developer)
2014-07-17 11:40

Documentation for -{static,shared}-libgcc:
https://gcc.gnu.org/onlinedocs/gcc-4.9.1/gcc/Link-Options.html#Link-Options [^]
-static-libgcc has been around at least since 4.1.2.
(0011873)
frisch (developer)
2014-07-17 11:40

Unless someone objects: let's add the -static-libgcc by default in config/Makefile.mingw
(0011878)
shinwell (developer)
2014-07-17 16:24

And let's reference the GCC documentation.
(0011879)
frisch (developer)
2014-07-17 16:28

Fixed on 4.02 only (for now).

- Issue History
Date Modified Username Field Change
2014-05-12 06:00 furuse New Issue
2014-05-12 06:02 furuse Note Added: 0011427
2014-05-13 09:20 doligez Tag Attached: patch
2014-05-13 15:14 protz Note Added: 0011437
2014-05-13 16:00 protz Note Added: 0011439
2014-05-14 12:23 frisch Note Added: 0011454
2014-05-14 13:53 furuse Note Added: 0011456
2014-05-14 13:55 furuse Note Edited: 0011456 View Revisions
2014-05-14 16:37 protz Note Added: 0011459
2014-05-16 17:49 doligez Note Added: 0011497
2014-05-16 22:28 Camarade_Tux Note Added: 0011506
2014-05-21 17:21 xleroy Note Added: 0011535
2014-05-21 17:36 doligez Target Version => 4.02.0+dev
2014-05-30 15:02 shinwell Note Added: 0011602
2014-05-30 15:02 shinwell Assigned To => frisch
2014-05-30 15:02 shinwell Status new => assigned
2014-05-31 18:34 frisch Note Added: 0011613
2014-06-03 14:26 protz Note Added: 0011636
2014-06-03 14:39 frisch Note Added: 0011638
2014-06-03 17:44 Camarade_Tux Note Added: 0011639
2014-06-03 17:56 protz Note Added: 0011640
2014-06-03 18:04 frisch Note Added: 0011641
2014-06-03 18:15 Camarade_Tux Note Added: 0011642
2014-06-03 18:18 frisch Note Added: 0011643
2014-06-03 18:29 Camarade_Tux Note Added: 0011644
2014-06-03 23:29 protz Note Added: 0011645
2014-06-04 09:04 frisch Note Added: 0011647
2014-07-17 10:58 frisch Status assigned => feedback
2014-07-17 11:40 shinwell Note Added: 0011872
2014-07-17 11:40 frisch Note Added: 0011873
2014-07-17 16:24 shinwell Note Added: 0011878
2014-07-17 16:28 frisch Note Added: 0011879
2014-07-17 16:28 frisch Status feedback => resolved
2014-07-17 16:28 frisch Fixed in Version => 4.02.0+dev


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker