Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005441OCamlOCaml generalpublic2011-12-23 11:542014-09-26 22:27
Reporterygrek 
Assigned To 
PriorityhighSeverityminorReproducibilityalways
StatusconfirmedResolutionopen 
PlatformOSOS Version
Product Version3.12.1 
Target Version4.02.2+devFixed in Version 
Summary0005441: order of -ccopt and -cclib wrt other arguments
DescriptionIn light of (not so) recent changes to binutils and ld defaults it becomes critically important to be able to pass linker options in predictable order.
ocamlbuild should guarantee that options from flags are inserted into command-line in the same order as they are encountered in the source (this is the case now afair), and ocamlopt should preserve the order of -cclib and -ccopt arguments wrt other entries on the command-line (this is NOT the case now).

Moreover the order in which ocamlopt inserts -ccopt had changed in 3.12 which caused build failures :

$ ocamlc.opt -custom -verbose -ccopt -lev lwtc.c lwtml.ml
+ gcc -o 'a.out' '-L/usr/lib/ocaml' '/tmp/camlprim50bdbc.c' 'lwtc.o' '-lcamlrun' -I'/usr/lib/ocaml' -lm -ldl -lcurses -lpthread -lev

$ /opt/ocaml-3.12.1/bin/ocamlc.opt -custom -verbose -ccopt -lev lwtc.c lwtml.ml
+ gcc -o 'a.out' '-L/opt/ocaml-3.12.1/lib/ocaml' -lev '/tmp/camlprimdac052.c' 'lwtc.o' '-lcamlrun' -I'/opt/ocaml-3.12.1/lib/ocaml' -lm -ldl -lcurses -lpthread
Additional InformationSome real-world examples :

the order of -l arguments - http://stackoverflow.com/questions/8301367/order-of-linked-libraries-in-ocamlbuild [^]

the order of linker options wrt linked libraries - https://savannah.nongnu.org/patch/?7579 [^]
Tagspatch
Attached Filespatch file icon 0001-Fix-PR-5441-Ensure-that-the-ccopt-arguments-will-com.patch [^] (1,243 bytes) 2012-02-15 20:23 [Show Content]

- Relationships

-  Notes
(0006577)
gerd (reporter)
2011-12-31 19:11
edited on: 2011-12-31 19:11

So far I know, shared libraries behave now similar to static libraries wrt. the order of linking. What makes it a bit difficult to solve is that the order for ld is the reverse of the order for the ocaml compilers, i.e. "ld -l1 -l2" means that lib1 can use lib2, whereas in ocaml we do it the other way round.

So what about this: When ocamlc processes arguments it translates each arg separately into a list of cc options, yielding a list of lists, and finally the outer list is reversed, and the "standard trailer" is appended (-lm -ldl -lcurses -lpthread). This would be quite predictable, and would also work with cc options pulled from cma/cmxa files.

(0006928)
meyer (developer)
2012-02-15 20:26

Attached patch prepared some while ago is the simplest and ad-hoc way to fix the problem. However it was discussed that might be worth to preserve the location of the -ccopt and -cclib options within the stream of arguments. That would be the most semantically correct behaviour of these options, however that needs some major rework in the code responsible for the argument processing.
(0007727)
doligez (administrator)
2012-07-12 13:59

It's too late for a major rework before 4.00.0. Unless somebody has a strong objection, I will soon postpone this PR to after 4.00.0.
(0007735)
xleroy (administrator)
2012-07-12 16:34

I think there is a misunderstanding here. As explained in the reference manual, -llib libraries should be given via the -cclib flag, not the -ccopt flag. -ccopt is for C compiler/linker options like -L/some/path or -g.

Then, ocamlc and ocamlopt call the C compiler with -ccopt flags first on the command line and -cclib libraries last. This does the job both with static libraries and new-style dynamic libraries (where order matters) and with old-style dynamic libraries (where order doesn't matter).

The change in 3.12 mentioned by ygrek fixes a mistake whereas -ccopt flags were not passed first, causing problems with -L/some/path options. This change is maybe not perfect (we need to check if the order of -ccopt options matter, and if so whether this order is preserved) but it goes in the right direction.

So, in summary: stick to the -cclib/-ccopt convention and things should work pretty well.
(0007736)
ygrek (reporter)
2012-07-12 17:02
edited on: 2012-07-12 17:02

The order of -ccopt wrt -cclib does matter too. Consider an example:

  -ccopt -Wl,-Bstatic -cclib -lsmth -ccopt -Wl,-Bdynamic

(0007741)
gerd (reporter)
2012-07-12 19:05

ygrek: This is an extreme example, but you are right, this may happen. FYI, I'm using a cc wrapper for controlling the cc arguments in this level of detail. (Which turns to be easier anyway, because you can be safe to catch all -lfoo args so that they can be statically linked, even those that are pulled in from a cma/cmxa.)

Btw, this example is also a good one, because the wish to link certain libraries statically is frequent, and there is no easy answer (-static links everything, but is broken on most OS). Maybe a separate feature wish?
(0007742)
xleroy (administrator)
2012-07-12 19:14

ygrek and gerd: I agree the current heuristic isn't perfect, and some of us Caml developers dream about better solutions. Still, the situation is nowhere as bad as when -ccopt is used for all -llib parameters. And in the 15 years that the -ccopt/-cclib heuristic has been in use, this is the first time that I hear about someone actually needing -Bstatic and -Bdynamic in real life. So, do tell us about actual needs, but don't overlook what already works.
(0007743)
gerd (reporter)
2012-07-12 19:34

Xavier, a couple of weeks ago I was actually asked to link in static C libraries for a commercial project. This is very useful when you want to distribute binaries, especially on platforms like OSX where you cannot just ask the users to install the dynamic libraries with a package manager. So the ability to do this is a real requirement for creating commercial binaries.

IMHO, this is only loosely connected with -cclib/-ccopt, hence I think this should be discussed in a different bug report. Having -cclib separately could even be a simplification here. There is currently the workaround to write a cc wrapper that you activate with -cc at link time, so a solution is in no way urgent.
(0007746)
ygrek (reporter)
2012-07-12 22:22

The described linking could be useful for mldonkey to provide static binaries that do still link dynamically to glibc. OTOH I've already achieved that the other way (cause mldonkey doesn't have external deps and all C libs are known and can be gathered in single variable in one Makefile). The cc wrapper solution also sounds plausible. That clarifies ccopt/cclib issue.
The only question left is whether there are no other scenarios when cclib order wrt other parameters is important.

- Issue History
Date Modified Username Field Change
2011-12-23 11:54 ygrek New Issue
2011-12-31 13:39 meyer Assigned To => meyer
2011-12-31 13:39 meyer Status new => assigned
2011-12-31 19:11 gerd Note Added: 0006577
2011-12-31 19:11 gerd Note Edited: 0006577 View Revisions
2012-02-15 20:23 meyer File Added: 0001-Fix-PR-5441-Ensure-that-the-ccopt-arguments-will-com.patch
2012-02-15 20:26 meyer Note Added: 0006928
2012-07-06 16:20 doligez Target Version => 4.00.0+dev
2012-07-12 13:59 doligez Note Added: 0007727
2012-07-12 13:59 doligez Status assigned => feedback
2012-07-12 16:34 xleroy Note Added: 0007735
2012-07-12 16:40 doligez Priority normal => urgent
2012-07-12 17:02 ygrek Note Added: 0007736
2012-07-12 17:02 ygrek Status feedback => assigned
2012-07-12 17:02 ygrek Note Edited: 0007736 View Revisions
2012-07-12 19:05 gerd Note Added: 0007741
2012-07-12 19:14 xleroy Note Added: 0007742
2012-07-12 19:15 xleroy Severity major => minor
2012-07-12 19:15 xleroy Target Version 4.00.0+dev => 4.01.0+dev
2012-07-12 19:34 gerd Note Added: 0007743
2012-07-12 22:22 ygrek Note Added: 0007746
2012-07-31 13:36 doligez Target Version 4.01.0+dev => 4.00.1+dev
2012-09-18 15:27 doligez Priority urgent => high
2012-09-20 18:05 doligez Target Version 4.00.1+dev => 4.01.0+dev
2013-08-19 14:27 doligez Target Version 4.01.0+dev => 4.01.1+dev
2013-11-29 13:00 doligez Tag Attached: patch
2014-01-21 11:54 doligez Assigned To meyer =>
2014-05-25 20:20 doligez Target Version 4.01.1+dev => 4.02.0+dev
2014-08-18 20:37 doligez Status assigned => confirmed
2014-08-18 20:37 doligez Target Version 4.02.0+dev => 4.02.1+dev
2014-09-04 00:25 doligez Target Version 4.02.1+dev => undecided
2014-09-26 22:27 doligez Target Version undecided => 4.02.2+dev


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker