Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0006797OCaml~DO NOT USE (was: OCaml general)public2015-02-27 15:352017-02-16 15:18
Reporterwhitequark 
Assigned Todoligez 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target Version4.02.2+dev / +rc1Fixed in Version4.02.2+dev / +rc1 
Summary0006797: -output-obj should support autolink
DescriptionAs I've just realized, in http://caml.inria.fr/mantis/view.php?id=6733 [^] I actually abused -output-obj: the compiler thought it was building an executable and thus invoked gcc and not ld -r ($PARTIALLD in Makefile.config). Thus it also linked in the runtime and most importantly the C stubs.

In fact if you try to use -output-obj as "intended", it doesn't link in the runtime, which is not too problematic, but also it doesn't link in the C stubs. It is unreasonable to expect the user to hunt down the C stubs manually and link them in explicitly, therefore I think -output-obj is completely broken.
Tagspatch
Attached Filespatch file icon output_complete_obj.patch [^] (9,149 bytes) 2015-03-25 19:02 [Show Content]

- Relationships
related to 0006733closedgasche Teach ocamlbuild to create shared libraries using -output-obj 
related to 0006796closedwhitequark -cclib should pass flags to the ld -r invocation used in -output-obj 
related to 0006625closedwhitequark ocamlbuild should pass -linkpkg together with -output-obj 
related to 0006929resolvedwhitequark incomplete fix of PR6797; ocamlbuild not taken into account 
related to 0006918resolved Using ocamlbuild to generate dynamic libraries (*.so/*.dll) with a C-API and an OCaml implementation? 

-  Notes
(0013355)
whitequark (developer)
2015-02-27 15:38

It would seem as the same problem as when you don't pass -custom to ocamlc, however there is of course no ocamlopt -custom.
(0013356)
whitequark (developer)
2015-02-27 15:44

It actually seems that ocamlc -output-obj suffers from the same problem.
(0013360)
whitequark (developer)
2015-02-27 20:31

@gasche, I've attached a patch that fixes this. Specifically:
  * Any defined behavior is unchanged.
  * When -output-obj is used together with -runtime-variant, a previously nonsensical combination, both runtime and autolink libraries are linked inside the output object.

This also fixes issue 0006796 in the case where -runtime-variant is passed.

The remove_Wl function is necessary now because some packages (ctypes specifically) rely on the fact that the linker options are processed by C compiler driver (the gcc or clang binary, usually), and try to pass arguments to actual linker by escaping them via -Wl. However, partial linking, on which -output-obj relies, invokes the linker (the ld binary, usually) directly.

The remove_Wl function performs unescaping in the same way as the compiler driver would. Furthermore, it would be nonsensical to pass any options not recognized either directly by the driver (like -L) or escaped for passing to the linker (i.e. -Wl), so this should cover all reasonable behavior. Even if not, which is unlikely, this would not break any existing code.

Normally the user would pass -runtime_variant '', or, equivalently, when 0006733 is merged, put runtime_variant() in _tags.

The following line:
    (if !Clflags.nopervasives || main_obj_runtime then "" else Config.native_c_libraries)
is needed because Config.native_c_libraries includes -ldl, which naturally has no static variant, and so linking that in would fail. It is reasonable (unlike with autolink) to require the host application to link this small, platform-specific, unchanging list of libraries.

Since the change is made to be noninvasive, I'd like to nominate it for 4.02.2.
(0013361)
doligez (administrator)
2015-02-27 22:52

You need to protect the calls to `String.sub` by testing the length of `cclib` first.

Also, I don't understand how passing `-runtime-variant ''` would make any difference, as the empty string is already the default value.
(0013362)
whitequark (developer)
2015-02-27 22:56

Oops. Okay, I need some other method to indicate "put the runtime and stubs into this object file". Suggestions? Introducing a separate flag just for this seems mildly fishy.
(0013523)
doligez (administrator)
2015-03-20 22:49

I don't think it's a good idea to use `-runtime-variant` for this purpose.

You should probably introduce a new flag. How about `-output-standalone-obj` or `-output-complete-obj`?
(0013524)
whitequark (developer)
2015-03-20 22:54

`-output-complete-obj` looks fine. Can we get this into 4.02.2?
(0013551)
whitequark (developer)
2015-03-25 19:02

Patch updated to use -output-complete-obj and also extended to ocamlc for completeness.
(0013742)
doligez (administrator)
2015-04-27 18:05

patch applied to 4.02 branch (rev 16054).
(0013781)
whitequark (developer)
2015-05-02 16:17

Reopening because of a slight bug in ocamlc handlign (ocamlopt is fine), demonstrated by:

$ ocamlfind -toolchain android ocamlc -verbose -package sqlite3 -linkpkg t.ml -o foo.o -passopt -output-complete-obj
Effective set of compiler predicates: pkg_sqlite3,autolink,byte
+ /home/whitequark/.opam/4.02.1+32bit/arm-linux-androideabi/bin/ocamlc -verbose -o foo.o -output-complete-obj -I /home/whitequark/.opam/4.02.1+32bit/arm-linux-androideabi/lib/sqlite3 /home/whitequark/.opam/4.02.1+32bit/arm-linux-androideabi/lib/sqlite3/sqlite3.cma t.ml
+ /home/whitequark/.opam/4.02.1+32bit/lib/android-ndk/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86/bin/arm-linux-androideabi-gcc --sysroot /home/whitequark/.opam/4.02.1+32bit/lib/android-ndk/platforms/android-17/arch-arm -I /home/whitequark/.opam/4.02.1+32bit/arm-linux-androideabi/include -L /home/whitequark/.opam/4.02.1+32bit/arm-linux-androideabi/lib -O2 -fno-defer-pop -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -fPIC -c '-I/home/whitequark/.opam/4.02.1+32bit/arm-linux-androideabi/lib/sqlite3' -I'/home/whitequark/.opam/4.02.1+32bit/arm-linux-androideabi/lib/ocaml' 'foo.c'
+ /home/whitequark/.opam/4.02.1+32bit/lib/android-ndk/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86/bin/arm-linux-androideabi-ld --sysroot /home/whitequark/.opam/4.02.1+32bit/lib/android-ndk/platforms/android-17/arch-arm -r -o'foo.o' '-L/home/whitequark/.opam/4.02.1+32bit/arm-linux-androideabi/lib/sqlite3' '-L/home/whitequark/.opam/4.02.1+32bit/arm-linux-androideabi/lib/ocaml' '/tmp/camlobjc51d50.o' '-lsqlite3_stubs' '-lcamlrun'
/home/whitequark/.opam/4.02.1+32bit/lib/android-ndk/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86/bin/arm-linux-androideabi-ld: error: /tmp/camlobjc51d50.o: file is empty
/home/whitequark/.opam/4.02.1+32bit/lib/android-ndk/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86/bin/arm-linux-androideabi-ld: internal error in target, at /s/ndk-toolchain/src/build/../binutils/binutils-2.24/gold/parameters.h:105
File "t.ml", line 1:
Error: Error while building custom runtime system
/home/whitequark/.opam/4.02.1+32bit/arm-linux-androideabi/bin/ocamlc returned with exit code 2

The fix is to change bytecomp/bytelink.c:576:

    let c_file = basename ^ ".c"

to:

    let c_file =
      if !Clflags.output_complete_object
      then Filename.temp_file "camlobj" ".c"
      else basename ^ ".c"
(0013782)
whitequark (developer)
2015-05-02 16:18

After the fix it should also be forward-ported to trunk, I think.
(0013853)
doligez (administrator)
2015-05-06 22:01
edited on: 2015-05-06 22:02

I applied your fix to 4.02 (rev 16095).

By default, this will be merged into trunk right after the release of 4.02.2. If you want it sooner, please say so.

(0015115)
frisch (developer)
2015-12-11 11:25

The -L options were not recognized by Microsoft linker, thus producing warnings when -pack'ing. I've changed them /libath:.

- Issue History
Date Modified Username Field Change
2015-02-27 15:35 whitequark New Issue
2015-02-27 15:35 whitequark Relationship added related to 0006733
2015-02-27 15:35 whitequark Relationship added related to 0006796
2015-02-27 15:38 whitequark Note Added: 0013355
2015-02-27 15:44 whitequark Summary Bare ocamlopt -output-obj is not useful => Bare -output-obj is not useful
2015-02-27 15:44 whitequark Description Updated View Revisions
2015-02-27 15:44 whitequark Note Added: 0013356
2015-02-27 15:58 whitequark Summary Bare -output-obj is not useful => -output-obj should support autolink
2015-02-27 20:21 whitequark Tag Attached: patch
2015-02-27 20:21 whitequark File Added: obj-runtime.patch
2015-02-27 20:31 whitequark Note Added: 0013360
2015-02-27 20:32 whitequark File Deleted: obj-runtime.patch
2015-02-27 20:32 whitequark File Added: obj-runtime.patch
2015-02-27 21:05 whitequark Relationship added related to 0006625
2015-02-27 22:52 doligez Note Added: 0013361
2015-02-27 22:56 whitequark Note Added: 0013362
2015-03-20 22:49 doligez Note Added: 0013523
2015-03-20 22:54 whitequark Note Added: 0013524
2015-03-25 19:02 whitequark File Deleted: obj-runtime.patch
2015-03-25 19:02 whitequark File Added: output_complete_obj.patch
2015-03-25 19:02 whitequark Note Added: 0013551
2015-04-27 18:05 doligez Note Added: 0013742
2015-04-27 18:05 doligez Status new => closed
2015-04-27 18:05 doligez Resolution open => fixed
2015-04-27 18:05 doligez Fixed in Version => 4.02.2+dev / +rc1
2015-05-02 16:17 whitequark Assigned To => doligez
2015-05-02 16:17 whitequark Note Added: 0013781
2015-05-02 16:17 whitequark Status closed => feedback
2015-05-02 16:17 whitequark Resolution fixed => reopened
2015-05-02 16:18 whitequark Note Added: 0013782
2015-05-02 16:18 whitequark Status feedback => assigned
2015-05-06 22:01 doligez Note Added: 0013853
2015-05-06 22:02 doligez Status assigned => resolved
2015-05-06 22:02 doligez Resolution reopened => fixed
2015-05-06 22:02 doligez Note Edited: 0013853 View Revisions
2015-07-12 20:56 whitequark Relationship added related to 0006929
2015-07-22 17:29 doligez Relationship added child of 0006918
2015-07-22 17:30 doligez Relationship deleted child of 0006918
2015-07-22 17:30 doligez Relationship added related to 0006918
2015-12-11 11:25 frisch Note Added: 0015115
2017-02-16 15:18 xleroy Status resolved => closed
2017-02-23 16:36 doligez Category OCaml general => -OCaml general
2017-03-03 17:55 doligez Category -OCaml general => -(deprecated) general
2017-03-03 18:01 doligez Category -(deprecated) general => ~deprecated (was: OCaml general)
2017-03-06 17:04 doligez Category ~deprecated (was: OCaml general) => ~DO NOT USE (was: OCaml general)


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker