Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Win32 port build with Mingw: -link -static-libgcc is required for flexlink call #6411

Closed
vicuna opened this issue May 12, 2014 · 25 comments
Closed
Assignees
Milestone

Comments

@vicuna
Copy link

vicuna commented May 12, 2014

Original bug ID: 6411
Reporter: furuse
Assigned to: @alainfrisch
Status: closed (set by @xavierleroy on 2015-12-11T18:28:14Z)
Resolution: open
Priority: normal
Severity: major
Version: 4.01.0
Target version: 4.02.0+dev
Fixed in version: 4.02.0+dev
Category: platform support (windows, cross-compilation, etc)
Tags: patch

Bug description

I built OCaml 4.01.0 with mingw64-i686-gcc 4.8.2-2. The build and installation were successful, but I got a linking problem of bigarray.cma:

$ ocaml bigarray.cma
Cannot load required shared library dllbigarray.
Reason: C:/ocamlmgw/lib/stublibs\dllbigarray.dll: The specified module could not be found.

@vicuna
Copy link
Author

vicuna commented May 12, 2014

Comment author: furuse

After investigating what is happening I found that the stublib dllbigarray.dll depends on libgcc_s_sjlj-1.dll but it is missing.

Googling about it I found a fix: we need -static-libgcc linker option at the creation of dllbigarray.dll. I do not know exactly which mingw64-i686-gcc versions require this but at least the following patch works fine with 4.8.2-2.

Longer and probably redundant explanation is available at: http://d.hatena.ne.jp/camlspotter/20140502/1399024687


*** ocaml-4.01.0.old/config/Makefile Fri May 2 17:25:36 2014
--- ocaml-4.01.0/config/Makefile Fri May 2 18:17:36 2014


*** 100,106 ****
CPP=$(BYTECC) -E

Flexlink

! FLEXLINK=flexlink -chain mingw -stack 16777216
FLEXDIR=$(shell $(FLEXLINK) -where)
IFLEXDIR=-I"$(FLEXDIR)"
MKDLL=$(FLEXLINK)
--- 100,106 ----
CPP=$(BYTECC) -E

Flexlink

! FLEXLINK=flexlink -chain mingw -stack 16777216 -link -static-libgcc
FLEXDIR=$(shell $(FLEXLINK) -where)
IFLEXDIR=-I"$(FLEXDIR)"
MKDLL=$(FLEXLINK)

@vicuna
Copy link
Author

vicuna commented May 13, 2014

Comment author: @protz

Right, but -static-libgcc is not the only way to fix that: you can just dynamically load libgcc_s_sjlj-1.dll by making sure the directory where the dll lives is in the PATH.

jonathan@farquaad:~/Code/ocaml (trunk) $ export PATH=/usr/i686-w64-mingw32/sys-root/mingw/bin/:$PATH
jonathan@farquaad:~/Code/ocaml (trunk) $ ocaml bigarray.cma
                             OCaml version 4.03.0+dev0-2014-05-12

# exit 1;;

I kind of think that the motto these days is that static linking should be avoided, but I don't have enough background to determine whether having Alain add -static-libgcc to the flexlink invocations or having users of OCaml on Windows tweak their path is the right thing to do. Alain, do you have an opinion on that?

@vicuna
Copy link
Author

vicuna commented May 13, 2014

Comment author: @protz

After some investigations, it turns out that mmap_win32.c contains an int64 division (line 89) and an int64 modulo (line 109). Both of these lines, when compiled by gcc (http://stackoverflow.com/questions/18448343/divdi3-division-used-for-long-long-by-gcc-on-x86), generate calls to external library functions (___divi3 and ___modi3), hence requiring libgcc_s_sjlj-1.dll. As far as Damien and I could tell, these are the only places which generate a dependency on the runtime library.

We could probably get away with replacing the division and the modulo by bit-shifting and masking respectively, since the divisor is a power of two in both cases.

@vicuna
Copy link
Author

vicuna commented May 14, 2014

Comment author: @alainfrisch

If we can manage to get rid of the dependency to the external library, this seems good to me.

I don't think we can tell users to add a non-default directory to their PATH. Cannot we instruct the linker to add the directory to the dynamic load path when building executables?

@vicuna
Copy link
Author

vicuna commented May 14, 2014

Comment author: furuse

If we can remove the dependency completely it is the best.

If we dynamically link it then users of binary executables by OCaml+MinGW may need to install the library, which is not good.

I am not sure about the license of the library but libgcc_s_sjlj-1.dll sounds pretty GNU. Not good for whom want to build closed source OCaml apps.

@vicuna
Copy link
Author

vicuna commented May 14, 2014

Comment author: @protz

I believe we all agree that the dependency ought to be removed. The only question is whether we should do it by rewriting the two operations on long long's by hand, or by static-linking libgcc. We would have to figure out the license implications of the latter solution, though. After a quick search, it seems to be fine (http://www.gnu.org/licenses/gcc-exception.html).

@vicuna
Copy link
Author

vicuna commented May 16, 2014

Comment author: @damiendoligez

I vote for rewriting: inelegant, but it's only a few lines.

@vicuna
Copy link
Author

vicuna commented May 16, 2014

Comment author: Camarade_Tux

libgcc_s_sjlj-1.dll but it is missing

Your toolchain sounds broken too.

@vicuna
Copy link
Author

vicuna commented May 21, 2014

Comment author: @xavierleroy

On line 89, "array_size" (the divisor) is definitely not a power of 2.

I probably miss the obvious, but aren't those helper functions __divd3 already used by the runtime system, and if so why aren't they available for DLLs thanks to FlexDLL magic?

@vicuna
Copy link
Author

vicuna commented May 30, 2014

Comment author: @mshinwell

Alain, are you able to help with this?

@vicuna
Copy link
Author

vicuna commented May 31, 2014

Comment author: @alainfrisch

I did not look at the issue concretely, and there might indeed be an issue with flexlink, but if the runtime system depends on those functions and they are provided by a dll which is not in the default dll load path, how is a standalone program (either ocamlrun or a bytecode program compiled with -custom) supposed to work out of the box on a machine without cygwin?

@vicuna
Copy link
Author

vicuna commented Jun 3, 2014

Comment author: @protz

I believe the two options are as follows.

  1. flexlink should now pass the -static-libgcc flag at link-time by default. Functions that used to be statically linked with the executable are now in a separate DLL; the -static-libgcc flag reverts to the former behavior. This means that generated programs linked with this flag will not depend on the external DLL.
  2. We should rewrite the two operations on long long's "by hand" (seems complicated) so as to not depend on the functions from the now-external DLL.

Alain, do you think option 1) would be ok for you?

@vicuna
Copy link
Author

vicuna commented Jun 3, 2014

Comment author: @alainfrisch

Why should flexlink's default be changed? Wouldn't it be enough to add the option to utils/Makefile.{mingw,mingw6464}?

@vicuna
Copy link
Author

vicuna commented Jun 3, 2014

Comment author: Camarade_Tux

As far as I'm concerned, I expect the dependency on this DLL and I don't see what the issue is. If someone wants static linking (especially for C libs), it should be done explicitely.

If I'm right, you're building from Cygwin and knowing who makes the packages, I doubt that DLL is missing. In other words, it sounds like an issue with the library lookup paths. If you're cross-compiling (which you are if you're using Cygwin), it's expected that the file is not in a standard place (and even more so for libgcc on Cygwin which, iirc, is the reason it's not in the usual paths).

I definitely acknowledge the use of static linking and I do so to distribute my yypkg.exe from http://win-builds.org but I don't think it should be the default, especially when you can do it fairly easily.

PS: I vote against rewriting: if there's a whole library and it's built as shared library, it's because there are more than a couple functions and you're likely to pull that library for other symbols and reasons too (especially if you interface with C and C++).

@vicuna
Copy link
Author

vicuna commented Jun 3, 2014

Comment author: @protz

If I'm right, you're building from Cygwin and knowing who makes the packages, I doubt that DLL is missing.

The DLL is definitely not missing. It's just that I'm unsure which one of switching to static linking or distributing the DLL along with the OCaml binaries is the best choice.

@vicuna
Copy link
Author

vicuna commented Jun 3, 2014

Comment author: @alainfrisch

As far as I'm concerned, I expect the dependency on this DLL and I don't see what the issue is.

The native ports (mingw / msvc) are supposed to produce standalone programs, i.e. programs which can run on a bare Windows machine. This is not strictly the case for the msvc ports, because they create dependencies to e.g. msvcr90.dll, which are not parts of Windows, but is still widely available (e.g. it is installed with recent .Net). Visual Studio C++ itself has the same problem. Until recently, the mingw port only created a dependency to msvcrt.dll, which is shipped with Windows. It was a very nice property, and it would be bad to loose it.

It seems quite bad to require OCaml users to ship a copy of a Cygwin DLL, even if licensing is ok, together with their native executables.

@vicuna
Copy link
Author

vicuna commented Jun 3, 2014

Comment author: Camarade_Tux

It's part of the C toolchain and it comes with the C toolchain and if it's not there, it's an issue with the C toolchain. It's simply in a different path.

For instance, in my cross-toolchains, it's in something like:
/opt/cross_toolchain_32/i686-w64-mingw32/lib/libgcc_s_sjlj-1.dll

And there's even an autotools variable for that place: toolexeclibdir which is used by nothing but GCC and for that shared library.

You can have a look at :
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=40125
and
http://cygwin.com/ml/cygwin/2010-07/msg00187.html

It's really only an issue with the lookup path.

Btw, you CANNOT ship that library because it changes depending on the exception handling system and it's not possible to foresee that.

And if you don't want to change lookup paths, simply copy the file at the end of OCaml's installation on Windows. Crude but effective.

@vicuna
Copy link
Author

vicuna commented Jun 3, 2014

Comment author: @alainfrisch

I'm not sure to follow the discussion. Why should we require people who need to execute a native OCaml program to have a C toolchain? Or are we only talking about making a native OCaml distribution?

@vicuna
Copy link
Author

vicuna commented Jun 3, 2014

Comment author: Camarade_Tux

This is a GCC file, definitely not a Cygwin one. The reason cygwin is involved is because the C toolchain is Cygwin's and is a cross-compiler which makes it write that .dll file in a specific location which is compatible with cross-compilation and multilib systems.

If you check the original error message it is for bytecode but it also mentions bigarray which has C stubs and I guess that's how libgcc gets linked in.

Apart from that, it should only matter to native-compiled executables except maybe that ocamlrun.exe itself might depend on it (it seems it currently doesn't or it would have failed earlier than when loading bigarray.cma stubs and if it did, there would have been no reason for the OS to lookup the library again [ and fail that ])

As for distributing additional files, I doubt most applications distribute 0 additional files and if they do, they can explicitely link statically. I'm doing just that in yypkg.

@vicuna
Copy link
Author

vicuna commented Jun 3, 2014

Comment author: @protz

I think we're conflating two issues here.

  • Should the OCaml binary distribution (i.e. the OCaml installer) depend on libgcc_s_sjlj-1.dll or not?
    • If we keep the current setup (NO -static-libgcc), we need to ship libgcc_s_sjlj-1.dll along with the installer.
    • If we apply Jun's patch, we do NOT need to ship this dll as it will be statically linked in bigarray.cma
  • Should the programs generated by ocamlopt depend on this DLL or not?
    • If we keep the current behavior of flexlink, programs generated by ocamlopt with a recent enough mingw64 toolchain WILL depend on either libgcc_s_sjlj-1.dll or libgcc_s_dwarf2-whatever.dll depending on the exact toolchain used by the user, so we need to instruct users to ship this DLL with their ocamlopt-compiled programs.
    • If we do change the behavior of flexlink and make it use -static-libgcc then programs generated by ocamlopt will not depend on the external DLL. This will incidentally solve the first point.

@vicuna
Copy link
Author

vicuna commented Jun 4, 2014

Comment author: @alainfrisch

  • Should the programs generated by ocamlopt depend on this DLL or not?

I think this is the most important question. The decision about the first one (OCaml binary distribution) can be made by the guys creating the Windows binary distributions; it's more a packaging issue.

If we do change the behavior of flexlink and make it use -static-libgcc then programs generated by ocamlopt will not depend on the external DLL.

I still don't understand why we need to change the default behavior of flexlink. It seems to me that Jun's patch would also work. The FLEXLINK variable in config/Makefile.{mingw,msvc,...} affects how ocamlopt invokes flexlink.

@vicuna
Copy link
Author

vicuna commented Jul 17, 2014

Comment author: @mshinwell

Documentation for -{static,shared}-libgcc:
https://gcc.gnu.org/onlinedocs/gcc-4.9.1/gcc/Link-Options.html#Link-Options
-static-libgcc has been around at least since 4.1.2.

@vicuna
Copy link
Author

vicuna commented Jul 17, 2014

Comment author: @alainfrisch

Unless someone objects: let's add the -static-libgcc by default in config/Makefile.mingw

@vicuna
Copy link
Author

vicuna commented Jul 17, 2014

Comment author: @mshinwell

And let's reference the GCC documentation.

@vicuna
Copy link
Author

vicuna commented Jul 17, 2014

Comment author: @alainfrisch

Fixed on 4.02 only (for now).

@vicuna vicuna closed this as completed Dec 11, 2015
@vicuna vicuna added this to the 4.02.0 milestone Mar 14, 2019
@vicuna vicuna added the bug label Mar 20, 2019
avsm pushed a commit to avsm/ocaml that referenced this issue Jun 21, 2020
avsm pushed a commit to avsm/ocaml that referenced this issue Jun 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants