ocamlrun should check compatibility of stublibs in CAML_LD_LIBRARY_PATH #6532

vicuna · 2014-09-01T19:52:19Z

Original bug ID: 6532
Reporter: @edwintorok
Status: acknowledged (set by @damiendoligez on 2014-09-15T13:19:56Z)
Resolution: open
Priority: normal
Severity: feature
Platform: x86_64
OS: Linux
OS Version: Linux debian 3.1
Version: 4.02.0+beta1 / +rc1
Target version: later
Category: runtime system and C interface
Monitored by: @avsm

Bug description

My system OCaml is 4.01.0, and for some reason opam had this in variables.sh
(don't know why, running opam config setup -a again fixed it):
CAML_LD_LIBRARY_PATH=/home/edwin/.opam/4.02.0/lib/stublibs:/usr/lib/ocaml/stublibs

This caused utop and utop-full to hang/crash, because ocamlrun 4.02.0 loads dllunix.so (and other stublibs) from CAML_LD_LIBRARY_PATH even if that .so is meant for another OCaml version (4.01.0 in this case).
It would be nice to have a version check to detect mismatches, otherwise applications can crash if CAML_LD_LIBRARY_PATH is set wrongly.

Steps to reproduce

have a system-wide OCaml installation of version < 4.02 (for example 4.01.0 from Debian unstable)
opam switch 4.02.0
export CAML_LD_LIBRARY_PATH=$HOME/.opam/4.02.0/lib/stublibs:/usr/lib/ocaml/stublibs
utop-full

*** Error in `/home/edwin/.opam/4.02.0/bin/ocamlrun': malloc(): memory corruption (fast): 0x0000000000be35d0 ***
Aborted

Additional information

strace /home/edwin/.opam/4.02.0/bin/ocamlrun utop-full 2>&1|grep open|grep dllunix.so
open("/usr/lib/ocaml/stublibs/dllunix.so", O_RDONLY|O_CLOEXEC) = 4

valgrind which ocamlrun utop-full
==7716== Invalid write of size 1
==7716== at 0x4A0BB1B: memcpy@@GLIBC_2.14 (mc_replace_strmem.c:882)
==7716== by 0x5E6F3EE: lwt_unix_blit_string_bytes (in /home/edwin/.opam/4.02.0/lib/stublibs/dlllwt-unix_stubs.so)
==7716== by 0x41C020: caml_interprete (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41D842: caml_main (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41A9C0: main (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== Address 0x5448ab3 is 531 bytes inside a block of size 1,136 free'd
==7716== at 0x4A08730: free (vg_replace_malloc.c:468)
==7716== by 0x40B289: caml_stat_free (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x413952: caml_input_val (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x4139FE: caml_input_value (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41BE95: caml_interprete (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41D842: caml_main (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41A9C0: main (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== Thread 2:
==7716== Syscall param read(buf) points to unaddressable byte(s)
==7716== at 0x31E480EADD: ??? (syscall-template.S:81)
==7716== by 0x5E6AC93: worker_bytes_read (in /home/edwin/.opam/4.02.0/lib/stublibs/dlllwt-unix_stubs.so)
==7716== by 0x5E6FEF1: execute_job (in /home/edwin/.opam/4.02.0/lib/stublibs/dlllwt-unix_stubs.so)
==7716== by 0x5E703CA: worker_loop (in /home/edwin/.opam/4.02.0/lib/stublibs/dlllwt-unix_stubs.so)
==7716== by 0x31E48080A3: start_thread (pthread_create.c:309)
==7716== Address 0x692e1d0 is 0 bytes after a block of size 0 alloc'd
==7716== at 0x4A074C0: malloc (vg_replace_malloc.c:291)
==7716== by 0x5C5FF7B: caml_ba_alloc (in /usr/lib/ocaml/stublibs/dllbigarray.so)
==7716== by 0x5C6011D: caml_ba_create (in /usr/lib/ocaml/stublibs/dllbigarray.so)
==7716== by 0x41BF52: caml_interprete (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41D842: caml_main (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41A9C0: main (in /home/edwin/.opam/4.02.0/bin/ocamlrun)

The text was updated successfully, but these errors were encountered:

vicuna · 2015-10-28T17:44:07Z

Comment author: @gasche

We get the following error report¹ from p4bl0, by just running "utop" from a shell in a half-badly-configured OPAM state.

*** Error in `/home/p4bl0/.opam/4.02.3/bin/ocamlrun': double free or corruption (out): 0x0000000001314880 ***
Abandon

Edwin thinks is an instance of the present issue. It would be really nice if we could improve the user experience there.

¹: https://www.reddit.com/r/ocaml/comments/3qjs1q/utop_is_a_much_better_toplevel_than_ocaml_if_you/cwfv811

vicuna · 2015-11-20T17:28:40Z

Comment author: @xavierleroy

I agree that versioning of DLLs would be a nice touch, but I have no idea how to do it portably. Suggestions welcome.

In the meantime, I hope OPAM no longer mucks with CAML_LD_LIBRARY_PATH, as this is generally a recipe for disaster.

vicuna · 2015-11-28T10:16:46Z

Comment author: @xavierleroy

I see OPAM still dfeines CAML_LD_LIBRARY_PATH. It must have its reasons.

I'm pushing this to after 4.03, as I don't foresee a technical solution in time for 4.03.

vicuna · 2015-12-20T12:20:24Z

Comment author: @gasche

Just today another beginner reported the same issue on the #ocaml IRC channel.

vicuna · 2015-12-25T18:33:09Z

Comment author: @avsm

OPAM historically sets CAML_LD_LIBRARY_PATH to account for stub libraries installed by a user into the ~/.opam space but not added to the system compiler's ld.conf.

ocaml/opam#198

Is there a better way to add the ~/.opam stublibs path to the compiler search path other than setting CAML_LD_LIBRARY_PATH?

github-actions · 2020-05-11T04:20:35Z

This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc.

xavierleroy · 2020-05-11T10:05:04Z

@avsm:

OPAM historically sets CAML_LD_LIBRARY_PATH to account for stub libraries installed by a user into the ~/.opam space but not added to the system compiler's ld.conf.

We're not talking about the system compiler's ld.conf ! ocamlrun from an OPAM switch looks into $OPAM/$SWITCH/lib/ocaml/ld.conf, whose default contents are

$OPAM/$SWITCH/lib/stublibs
$OPAM/$SWITCH/lib/ocaml

Hence, setting the CAML_LD_LIBRARY_PATH to these two directories is just redundant when the ocamlrun from the switch is called, or incorrect when another ocamlrun is called.

Is there a better way to add the ~/.opam stublibs path to the compiler search path other than setting CAML_LD_LIBRARY_PATH?

Yes: do not set CAML_LD_LIBRARY_PATH at all!

dbuenzli · 2020-05-11T10:20:09Z

@xavierleroy I suspect what you mention doesn't work with system switches where the ld.conf you mention is not managed by opam but by the system packager and will likely point to something like /usr/local/lib/ocaml/stublibs rather than the opam directory.

System switches have been an endless source of problems (I think they should have been killed) but unfortunately it seems users do like them.

dra27 · 2020-05-11T10:20:15Z

The versioning issue here will be dealt with as part of the relocatable compiler (stub libraries get the same treatment as ocamlrun under that, so in this misconfigured state you would get a message that the appropriate dllunix cannot be found, rather than the world falling apart as here).

This report pre-dates opam 2.0 which could improve on this very easily by moving the CAML_LD_LIBRARY_PATH logic from the ocaml package specifically to ocaml-system - this would mean that compilers which are compiled in ~/.opam would not set or mess around with CAML_LD_LIBRARY_PATH at all.

However, that only reduces the problem - I still don't think there's a way to add a directory to the stublib search path when you can't write to ld.conf (which is the case for a system package-manager maintained compiler).

xavierleroy · 2020-05-11T10:27:45Z

Thanks for the feedback. I concede that CAML_LD_LIBRARY_PATH could need to be set for system switches. But for other switches (the ones I care about) I still contend it should not be set.

dra27 · 2021-07-21T06:14:01Z

Patch set which addresses this is in progress…

dra27 · 2022-07-27T07:17:22Z

PRs for this are actually due soon (there’s a talk on the whole relocatable compiler at this year’s OCaml Workshop)

vicuna added the stdlib label Mar 14, 2019

vicuna mentioned this issue Mar 14, 2019

make ocamlrun versioned, make ocaml bytecode executable (non -custom) find runtime on PATH #6976

Closed

vicuna added the feature-wish label Mar 20, 2019

github-actions bot added the Stale label May 11, 2020

xavierleroy mentioned this issue May 11, 2020

add new variable %{ld_conf}% ocaml/opam#198

Closed

dra27 mentioned this issue May 11, 2020

Only set CAML_LD_LIBRARY_PATH for system switches ocaml/opam-repository#16406

Open

github-actions bot removed the Stale label Jun 15, 2020

github-actions bot added the Stale label Jul 21, 2021

dra27 removed the Stale label Jul 21, 2021

ocaml deleted a comment from github-actions bot Jul 21, 2021

github-actions bot added the Stale label Jul 27, 2022

dra27 removed the Stale label Jul 27, 2022

ocaml deleted a comment from github-actions bot Jul 27, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ocamlrun should check compatibility of stublibs in CAML_LD_LIBRARY_PATH #6532

ocamlrun should check compatibility of stublibs in CAML_LD_LIBRARY_PATH #6532

vicuna commented Sep 1, 2014

vicuna commented Oct 28, 2015

vicuna commented Nov 20, 2015

vicuna commented Nov 28, 2015

vicuna commented Dec 20, 2015

vicuna commented Dec 25, 2015

github-actions bot commented May 11, 2020

xavierleroy commented May 11, 2020

dbuenzli commented May 11, 2020

dra27 commented May 11, 2020

xavierleroy commented May 11, 2020

dra27 commented Jul 21, 2021

dra27 commented Jul 27, 2022