Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross-compilers cannot link bytecode executables using custom primitives #6779

Closed
vicuna opened this issue Feb 11, 2015 · 18 comments
Closed
Assignees
Labels
Milestone

Comments

@vicuna
Copy link

vicuna commented Feb 11, 2015

Original bug ID: 6779
Reporter: @whitequark
Assigned to: @damiendoligez
Status: closed (set by @xavierleroy on 2016-12-07T10:29:43Z)
Resolution: fixed
Priority: normal
Severity: minor
Target version: 4.02.2+dev / +rc1
Fixed in version: 4.02.2
Category: ~DO NOT USE (was: OCaml general)
Related to: #6845
Monitored by: @gasche

Bug description

The reason is that Symtable.num_of_prim uses Dll, but a cross-compiler cannot load shared libraries at compile-time. Thus a cross-compiler should always build executables with -custom, or at least do that if any cma's using custom primitives are linked in.

Alternatively, just drop the check entirely if cross-compiling. It does not seem to serve any purpose besides making really sure that the primitive actually exists.

@vicuna
Copy link
Author

vicuna commented Feb 11, 2015

Comment author: @gasche

This could be discussed with Adrien Nader which worked on the topic previously -- I suspect generally on cross-compilation you're duplicating a bit of the design/work he's done, so maybe his opinion could be a good double-check.

@vicuna
Copy link
Author

vicuna commented Feb 15, 2015

Comment author: adrien

I've already raised the issue but I don't remember where the discussion happened: caml-list or mantis (it's already been more than one year).

Basically, Xavier Leroy said he was quite eager to keep the current compile-time safety rather than delaying the checks at run-time. I don't think there's a simple and fast way to do such checks for cross-compilation.

I have to say it's quite low on my list of issues: full support for cross-compilation in upstream releases is quite far from being available and this is simple to patch away and I've settled on keeping a patch that disables the check.

@vicuna
Copy link
Author

vicuna commented Feb 15, 2015

Comment author: @whitequark

Actually 4.02.2 is likely to have patch-less support for cross-compilation (assuming you provide Makefile.config, m.h and s.h.)

I've built a lot of libraries for opam-android1. Pure OCaml cma, cmxa, cmxs build perfectly well as well as those with C components. I think this would be the main issue left actually.

@vicuna
Copy link
Author

vicuna commented Feb 15, 2015

Comment author: @whitequark

I actually think the check could be still made during cross-compiling in the same way as objinfo currently works: using a helper executable using libbfd.

I think it would be even better if this was done also when /not/ cross-compiling, so that the compiler will not invoke any side effects on loading a stubs library (like invoking the global C++ constructors). I believe this is actually a problem for stubs produced for statically linked LLVM.

@vicuna
Copy link
Author

vicuna commented Feb 15, 2015

Comment author: adrien

Actually 4.02.2 is likely to have patch-less support for cross-compilation (assuming you provide Makefile.config, m.h and s.h.)

I think that still leaves out Windows which is my criteria (along with using ./configure).

I actually think the check could be still made during cross-compiling in the same way as objinfo currently works: using a helper executable using libbfd.

Quite probably. The need to spawn an external executable is why I wrote "fast" too. :)

I think it would be even better if this was done also when /not/ cross-compiling, so that the compiler will not invoke any side effects on loading a stubs library (like invoking the global C++ constructors).

Agreed. I won't be touching this anytime soon however so at least it's unlikely that effort is duplicated.

@vicuna
Copy link
Author

vicuna commented Feb 15, 2015

Comment author: @whitequark

The need to spawn an external executable is why I wrote "fast" too. :)

Well, you could make a direct interface using C stubs, but that's considerably more annoying to handle conditionally. libbfd isn't present everywhere, too, although the detection code is already there for objinfo.

We could reuse it to get rid of the objinfo_helper, too...

@vicuna
Copy link
Author

vicuna commented Feb 15, 2015

Comment author: adrien

BFD is GPL.

@vicuna
Copy link
Author

vicuna commented Feb 15, 2015

Comment author: @whitequark

spawn it is, then.

@vicuna
Copy link
Author

vicuna commented Apr 28, 2015

Comment author: @damiendoligez

I don't think it's reasonable to expect a BFD-based solution before 4.02.2.

How easy is it to check for cross-compilation? Dropping the check for all compilations is not acceptable, but we could do it for cross-compilation as an interim solution.

@vicuna
Copy link
Author

vicuna commented Apr 28, 2015

Comment author: @whitequark

OCaml's buildsystem currently has no knowledge of whether it builds a cross-compiler or not. An explicit HOST/TARGET variables in Makefile.config might be the solution.

@vicuna
Copy link
Author

vicuna commented Apr 28, 2015

Comment author: adrien

HOST and TARGET variables are already there.

These variables can be found in config/Makefile and utils/config.mlp at least. They should be visible everywhere.

@vicuna
Copy link
Author

vicuna commented Apr 30, 2015

Comment author: @damiendoligez

I have committed a fix in 4.02 (rev 16057). Please test and tell me if it does the right thing in a cross-compilation setup.

@vicuna
Copy link
Author

vicuna commented May 2, 2015

Comment author: @whitequark

Nice short fix. I confirm that it works.

@vicuna
Copy link
Author

vicuna commented May 2, 2015

Comment author: @whitequark

I think the fix should be forward-ported to trunk.

@vicuna
Copy link
Author

vicuna commented Jul 25, 2015

Comment author: @gasche

This has now been merged in trunk by Damien's big merge commit.

@vicuna
Copy link
Author

vicuna commented Sep 20, 2015

Comment author: adrien

I only recently realized that such checks were wanted by more than OCaml. For instance, mageia uses ld's --no-undefined:

   --no-undefined
   -z defs
       Report unresolved symbol references from regular object files.
       This is done even if the linker is creating a non-symbolic shared
       library.  The switch --[no-]allow-shlib-undefined controls the
       behaviour for reporting unresolved references found in shared
       libraries being linked in.

There's also libtool for Windows which has -no-undefined (only one dash and not the same executable) which hits the same things but differently I think.

I know both work for cross-compilation and it should be interesting to see whether they can be used to get the check for ocaml's cross-compilation too (because of lack of time for that, I haven't /tried/ to think about much so I have absolutely no idea whether that will be useful or not but I thought it was important to share these if someone were to start working on this).

@vicuna
Copy link
Author

vicuna commented Sep 20, 2015

Comment author: @whitequark

Yes, it should be very much possible to use --no-undefined for this purpose. However, it sometimes breaks badly written code, so some care would be needed to roll it out.

@vicuna
Copy link
Author

vicuna commented Sep 21, 2015

Comment author: adrien

I'm fairly confident. There are around 120 (C) packages in win-builds.org and I haven't yet seen such issues or they were issues that would have happened at runtime anyway. Moreover, Mageia uses that for all of its packages and they have quite a lot of them.

@vicuna vicuna closed this as completed Dec 7, 2016
@vicuna vicuna added this to the 4.02.2 milestone Mar 14, 2019
@vicuna vicuna added the bug label Mar 20, 2019
avsm pushed a commit to avsm/ocaml that referenced this issue Jun 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants