MantisBT - OCaml
View Issue Details
0005693OCamlotherlibspublic2012-07-20 18:452017-03-15 11:23
thelema 
doligez 
lowminoralways
assignedopen 
x86_64OS X10.7.4
3.12.1 
 
0005693: Missing '_environ' symbol used in libunix.a
When building a mixed OCaml/C library that includes unix.cmxa, the osx linker gives an error about an undefined symbol "_environ".

IRC Log: http://tunes.org/~nef/logs/ocaml/12.07.20 [^] - starts at 07:02:49
Bug reporter cannot reproduce.
When compiling a library as follows:
ocamlopt -o libocaml_rl_pi_rnn.so -I /usr/local/lib/ocaml/
                   -I /usr/local/lib/ocaml/site-lib/extlib -I
                   /usr/local/lib/ocaml/site-lib/batteries unix.cmxa nums.cmxa
                   batteries.cmxa rnnRL.cmxa ocaml_rl_pi_bi.ml -ccopt
                   ocaml_interface.c -ccopt -I/usr/local/lib/ocaml/ -ccopt
                   -shared

The following error happens.

Undefined symbols for architecture x86_64:
"_environ", referenced from:
       _unix_execvpe in libunix.a(execvp.o)
       _unix_environment in libunix.a(envir.o)
10:17 < thelema> http://trac.macports.org/browser/trunk/dports/lang/hugs98/files/patch-packages-base-include-HsBase.h.diff?rev=81676 [^]
...
10:36 < [Joshuah]> its a bug in the unix code. i added the lines posted in the
                   link you gave me to unix.c and another .c file in
                   otherlibs/unix
10:36 < [Joshuah]> it now compiles and runs

The patch is:
-extern char** environ;
+#ifdef __APPLE__
+#include <crt_externs.h>
+#define environ (*_NSGetEnviron())
+#else
+ extern char **environ;
+#endif
No tags attached.
tgz dynlink-example.tgz (628) 2012-07-21 21:25
https://caml.inria.fr/mantis/file_download.php?file_id=707&type=bug
tgz dynlink-example2.tgz (756) 2012-07-23 16:29
https://caml.inria.fr/mantis/file_download.php?file_id=709&type=bug
gz dynlink-example3.tar.gz (1,210) 2012-07-27 13:18
https://caml.inria.fr/mantis/file_download.php?file_id=713&type=bug
Issue History
2012-07-20 18:45thelemaNew Issue
2012-07-20 18:58xleroyNote Added: 0007787
2012-07-20 18:58xleroyStatusnew => feedback
2012-07-20 20:10xleroyNote Added: 0007788
2012-07-20 20:10xleroyNote Edited: 0007788bug_revision_view_page.php?bugnote_id=7788#r410
2012-07-20 23:01thelemaNote Added: 0007789
2012-07-20 23:01thelemaStatusfeedback => new
2012-07-21 21:25doligezFile Added: dynlink-example.tgz
2012-07-21 21:26doligezNote Added: 0007792
2012-07-21 21:26doligezStatusnew => feedback
2012-07-23 16:20doligezNote Added: 0007797
2012-07-23 16:21doligezFile Added: dynlink-example-new.tgz
2012-07-23 16:29doligezFile Deleted: dynlink-example-new.tgz
2012-07-23 16:29doligezFile Added: dynlink-example2.tgz
2012-07-23 16:33doligezNote Added: 0007798
2012-07-23 16:33doligezStatusfeedback => confirmed
2012-07-23 16:33doligezTarget Version => 4.01.0+dev
2012-07-24 09:38doligezNote Edited: 0007798bug_revision_view_page.php?bugnote_id=7798#r412
2012-07-27 13:18doligezFile Added: dynlink-example3.tar.gz
2012-07-27 13:19doligezNote Added: 0007826
2012-07-31 13:36doligezTarget Version4.01.0+dev => 4.00.1+dev
2012-09-21 14:51doligezPrioritynormal => low
2012-09-21 14:51doligezTarget Version4.00.1+dev => 4.01.0+dev
2013-07-22 13:05frischTarget Version4.01.0+dev => 4.02.0+dev
2014-07-30 23:16doligezTarget Version4.02.0+dev => 4.02.1+dev
2014-09-04 00:25doligezTarget Version4.02.1+dev => undecided
2014-09-26 20:38doligezTarget Versionundecided => 4.02.2+dev / +rc1
2015-02-24 23:07doligezTarget Version4.02.2+dev / +rc1 => 4.03.0+dev / +beta1
2016-04-19 13:29doligezTarget Version4.03.0+dev / +beta1 => 4.03.1+dev
2017-02-16 14:01doligezTarget Version4.03.1+dev => undecided
2017-02-23 16:42doligezCategoryOCaml otherlibs => otherlibs
2017-03-15 11:23doligezAssigned To => doligez
2017-03-15 11:23doligezStatusconfirmed => assigned
2017-03-15 11:23doligezTarget Versionundecided =>

Notes
(0007787)
xleroy   
2012-07-20 18:58   
Which version of XCode are you using?

As I said on caml-list, the "environ" variable is standard according to POSIX and Single Unix Specification, so something went terribly wrong here.

Damien, Xavier C.: can you reproduce on your Macs?
(0007788)
xleroy   
2012-07-20 20:10   
More info posted by reporter on caml-list:

My setting is:

Mac OS X 10.7.4
gcc version: i686-apple-darwin11-llvm-gcc-4.2
ocaml 3.12.1
Xcode 4.3.3 (4E3002)

I don't develop in Xcode, but use makefiles instead. Homebrew was used to install ocaml.

The error only occurs when linking to a dynamic lib, not when I link against an executable (main loop in c).

(0007789)
thelema   
2012-07-20 23:01   
Per Benedikt Meurer on caml-list:


Of course... as stated in environ(7):

"Shared libraries and bundles don't have direct access to environ, which
 is only available to the loader ld(1) when a complete program is being
 linked. The environment routines can still be used, but if direct access
 to environ is needed, the _NSGetEnviron() routine, defined in
 <crt_externs.h>, can be used to retrieve the address of environ at run-
 time."

It has always been this way on OS X, so Keyan's patch is correct.
(0007792)
doligez   
2012-07-21 21:26   
That command line looks suspect to me:
1. "-o libocaml_rl_pi_rnn.so": this tells ocamlopt to name its output file libocaml_rl_pi_rnn.so, but there is nothing to tell it that you want to build a shared library, so ocamlopt is going to make an executable and name it libocaml_rl_pi_rnn.so. This is probably not what you want. Note that you never get "undefined symbol" errors when building shared libraries.
2. "-ccopt -shared":
   from gcc's man page:
       -shared
           Produce a shared object which can then be linked with other objects
           to form an executable. Not all systems support this option. For
           predictable results, you must also specify the same set of options
           that were used to generate code (-fpic, -fPIC, or model suboptions)
           when you specify this option.[1]

           This option is not supported on Mac OS X.

In any case, I cannot reproduce the bug as reported. When I do "ocamlopt -o foo.so unix.cmxa -ccopt -shared", I get errors, not only on _environ, but also on _caml_atom_table, _caml_code_area_start, and _caml_code_area_end.

I have uploaded an example that's almost working, but when I run a.out I get this error:
   dlopen(./foo.so, 2): Symbol not found: _caml_atom_table
     Referenced from: /Users/doligez/dynlink-example/foo.so
     Expected in: flat namespace
    in /Users/doligez/dynlink-example/foo.so

Xavier, can you tell us what's I'm missing?
(0007797)
doligez   
2012-07-23 16:20   
OK, what I was missing is the surprising semantics of COMMON data in C object files and its variations across C linkers...

We'll probably have a better fix in a future version of OCaml, but for now you need to reference a function from "startup.c", to prevent the Mac OS X linker from dropping the file when building your shared library.

See the new version of my example code (I've also fixed bar.c to avoid a segfault).
(0007798)
doligez   
2012-07-23 16:33   
(edited on: 2012-07-24 09:38)
Note for the future: the best solution we found so far is to add an initializer to every global variable of the runtime system, to force them into the data segment.
An other possibility: make the OCaml compiler output references to caml_atom_table as common rather than imported symbol.

(0007826)
doligez   
2012-07-27 13:19   
I have uploaded a more complete example that demonstrates passing arguments, returning values, and catching exceptions (dynlink-example3).