New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Patch from PR6733 does not work on OS X #6927
Comments
Comment author: @whitequark Note that this leaves the output_shared tag used but not defined. This was an oversight at first, but then I realized that it would be convenient for myocamlbuild.ml authors dealing with some kind of weird platform, so I think it should stay. |
Comment author: @whitequark Anyone reading this and using 4.02.2 can use the following myocamlbuild rules to override the broken behavior: open Ocamlbuild_plugin dispatch (function |
Comment author: @whitequark The issue is actually more complex than this, because the OS X bundles are not equivalent to Linux shared libraries and Windows dll's: you cannot link them directly and have to use dlopen() and friends. The differences between bundles and dylibs are rather subtle. At 10.4 and before, you could not dlopen() a dylib--you could only link to it, in fact there was no dlopen()--and you could use the corresponding NS* API to load and unload a bundle. After 10.4, bundles and dylibs are almost entirely equivalent; you can dlopen() and dlclose() both. The only differences are that 1) you cannot directly link a dylib and 2) rpath is not handled in the same way. (I have grepped ld64, XNU and dyld from Apple's opensource repository to independently confirm this information.) Now, let's look at OCaml. OCaml creates dynamic libraries in three different ways:
Currently, all of these create bundles: the former two because plugins and stubs match the semantics of the bundles as originally created (i.e. at 10.4 and before), the latter because ocamlbuild reuses the configuration options. As a side effect, the .native.so can not be linked against, which is a severe problem for the people trying to link OCaml code into a larger binary portably. I propose to always build dylibs instead. At 10.5 and later, none of the behavior that OCaml depends on will change (the rpath handling change is irrelevant because neither OCaml-built plugins nor OCaml-built .{byte,native}.so files import any libraries except system ones). Thus, both plugins and .{byte,native}.so files could be both dynamically loaded and linked against. Admittedly, linking against plugins would not be very useful, though who knows. |
Comment author: @whitequark OK, there is actually one more problem on OS X. That is, without the -all_load flag (mostly equivalent to -Wl,--whole-archive) flag, the generated library does not include the symbols from libasmrun.a(startup.o), notably caml_startup. On the other hand, with the -all_load flag, not all _caml_curry functions are generated that are required for the stdlib. The reason this happens is that OCaml's behavior of lazily generating curry functions depends on the absence of -Wl,--whole-archive flag, because it determines which curry functions to include on the level of an imported .cmx, corresponding to a .o inside a .a. I'm actually not completely sure why the startup.o file is included on Linux. It might be some kind of platform-specific symbol dependency. Either way, one way to solve it would be to pass -force_load /path/to/libasmrun.a when linking a .so with -output-complete-obj on OS X, so that all the archive members are included. I'm not sure if there are other ways. |
Comment author: @whitequark Let me know what the maintainers think about this and I can produce patches. |
Comment author: @gasche I asked (I don't know anything about OSX myself, sorry). |
Comment author: @damiendoligez This startup.o problem may be related to #5693, see #5693#c7797 . |
Comment author: @whitequark Yes, this seems to be the same issue. |
Comment author: @garrigue Answering 14211: |
Comment author: @whitequark There are no problems with building the dylibs. I have tried building OCaml like that and it works. However, I anticipate problems with tools, since none of them know about the extension. Specifically:
So on one hand, using .so for .dylib is kind of user-hostile. A .so is not a .dylib and there are significant semantic differences between the two, most importantly with the XCode GUI tools, which would be the most likely way a .dylib will be processed by an end user on OS X. On another one, this change would probably break a substantial part of the OCaml ecosystem on OS X, which is already under-tested, until at least one major release. |
Comment author: @whitequark Oh, and this is compounded by the fact that the stdlib does not allow to determine whether you're running on Darwin or not. I'm actually not sure what would be a decent way to do this from OCaml, at all. |
Comment author: @damiendoligez Related to #988 |
Comment author: @damiendoligez ocamlbuild is now a separate project that lives on GitHub. |
Original bug ID: 6927
Reporter: @whitequark
Assigned to: @whitequark
Status: resolved (set by @damiendoligez on 2017-02-24T12:07:38Z)
Resolution: suspended
Priority: normal
Severity: minor
OS: OS X
Version: 4.02.2
Target version: later
Category: -for ocamlbuild use https://github.com/ocaml/ocamlbuild/issues
Tags: patch
Related to: #6733
Monitored by: @gasche
Bug description
Amusingly, this is a case of doing more work than necessary and failing at it, because the linker invocation already includes the -shared/-bundle flag, added by the compiler driver (ocamlc/ocamlopt). On Linux the second instance of -shared, added by the flag ["ocaml"; "link"; "output_shared"] built-in rule, is ignored, but on OS X it conflicts with -bundle.
A patch is attached. I verified that it works on Linux and OS X. It probably also works on Windows, since now output_so includes no platform details not used in the compiler elsewhere, but I'm not sure. Flexlink might or might not be a problem for the 3rd party code that wishes to load the resulting DLLs.
Additional information
The target is a bundle, but it is called x.{byte,native}.so. This is actually common practice, and there is no generally agreed upon file extension for bundles (although .bundle is often used), but ocamlbuild rules mention .dylib, which does not actually happen. A second patch is attached to alleviate that.
File attachments
The text was updated successfully, but these errors were encountered: