Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ocamlrun should check compatibility of stublibs in CAML_LD_LIBRARY_PATH #6532

Open
vicuna opened this issue Sep 1, 2014 · 12 comments
Open

Comments

@vicuna
Copy link

vicuna commented Sep 1, 2014

Original bug ID: 6532
Reporter: @edwintorok
Status: acknowledged (set by @damiendoligez on 2014-09-15T13:19:56Z)
Resolution: open
Priority: normal
Severity: feature
Platform: x86_64
OS: Linux
OS Version: Linux debian 3.1
Version: 4.02.0+beta1 / +rc1
Target version: later
Category: runtime system and C interface
Monitored by: @avsm

Bug description

My system OCaml is 4.01.0, and for some reason opam had this in variables.sh
(don't know why, running opam config setup -a again fixed it):
CAML_LD_LIBRARY_PATH=/home/edwin/.opam/4.02.0/lib/stublibs:/usr/lib/ocaml/stublibs

This caused utop and utop-full to hang/crash, because ocamlrun 4.02.0 loads dllunix.so (and other stublibs) from CAML_LD_LIBRARY_PATH even if that .so is meant for another OCaml version (4.01.0 in this case).
It would be nice to have a version check to detect mismatches, otherwise applications can crash if CAML_LD_LIBRARY_PATH is set wrongly.

Steps to reproduce

  1. have a system-wide OCaml installation of version < 4.02 (for example 4.01.0 from Debian unstable)
  2. opam switch 4.02.0
  3. export CAML_LD_LIBRARY_PATH=$HOME/.opam/4.02.0/lib/stublibs:/usr/lib/ocaml/stublibs
  4. utop-full

*** Error in `/home/edwin/.opam/4.02.0/bin/ocamlrun': malloc(): memory corruption (fast): 0x0000000000be35d0 ***
Aborted

Additional information

strace /home/edwin/.opam/4.02.0/bin/ocamlrun utop-full 2>&1|grep open|grep dllunix.so
open("/usr/lib/ocaml/stublibs/dllunix.so", O_RDONLY|O_CLOEXEC) = 4

valgrind which ocamlrun utop-full
==7716== Invalid write of size 1
==7716== at 0x4A0BB1B: memcpy@@GLIBC_2.14 (mc_replace_strmem.c:882)
==7716== by 0x5E6F3EE: lwt_unix_blit_string_bytes (in /home/edwin/.opam/4.02.0/lib/stublibs/dlllwt-unix_stubs.so)
==7716== by 0x41C020: caml_interprete (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41D842: caml_main (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41A9C0: main (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== Address 0x5448ab3 is 531 bytes inside a block of size 1,136 free'd
==7716== at 0x4A08730: free (vg_replace_malloc.c:468)
==7716== by 0x40B289: caml_stat_free (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x413952: caml_input_val (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x4139FE: caml_input_value (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41BE95: caml_interprete (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41D842: caml_main (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41A9C0: main (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== Thread 2:
==7716== Syscall param read(buf) points to unaddressable byte(s)
==7716== at 0x31E480EADD: ??? (syscall-template.S:81)
==7716== by 0x5E6AC93: worker_bytes_read (in /home/edwin/.opam/4.02.0/lib/stublibs/dlllwt-unix_stubs.so)
==7716== by 0x5E6FEF1: execute_job (in /home/edwin/.opam/4.02.0/lib/stublibs/dlllwt-unix_stubs.so)
==7716== by 0x5E703CA: worker_loop (in /home/edwin/.opam/4.02.0/lib/stublibs/dlllwt-unix_stubs.so)
==7716== by 0x31E48080A3: start_thread (pthread_create.c:309)
==7716== Address 0x692e1d0 is 0 bytes after a block of size 0 alloc'd
==7716== at 0x4A074C0: malloc (vg_replace_malloc.c:291)
==7716== by 0x5C5FF7B: caml_ba_alloc (in /usr/lib/ocaml/stublibs/dllbigarray.so)
==7716== by 0x5C6011D: caml_ba_create (in /usr/lib/ocaml/stublibs/dllbigarray.so)
==7716== by 0x41BF52: caml_interprete (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41D842: caml_main (in /home/edwin/.opam/4.02.0/bin/ocamlrun)
==7716== by 0x41A9C0: main (in /home/edwin/.opam/4.02.0/bin/ocamlrun)

@vicuna
Copy link
Author

vicuna commented Oct 28, 2015

Comment author: @gasche

We get the following error report¹ from p4bl0, by just running "utop" from a shell in a half-badly-configured OPAM state.

*** Error in `/home/p4bl0/.opam/4.02.3/bin/ocamlrun': double free or corruption (out): 0x0000000001314880 ***
Abandon

Edwin thinks is an instance of the present issue. It would be really nice if we could improve the user experience there.

¹: https://www.reddit.com/r/ocaml/comments/3qjs1q/utop_is_a_much_better_toplevel_than_ocaml_if_you/cwfv811

@vicuna
Copy link
Author

vicuna commented Nov 20, 2015

Comment author: @xavierleroy

I agree that versioning of DLLs would be a nice touch, but I have no idea how to do it portably. Suggestions welcome.

In the meantime, I hope OPAM no longer mucks with CAML_LD_LIBRARY_PATH, as this is generally a recipe for disaster.

@vicuna
Copy link
Author

vicuna commented Nov 28, 2015

Comment author: @xavierleroy

I see OPAM still dfeines CAML_LD_LIBRARY_PATH. It must have its reasons.

I'm pushing this to after 4.03, as I don't foresee a technical solution in time for 4.03.

@vicuna
Copy link
Author

vicuna commented Dec 20, 2015

Comment author: @gasche

Just today another beginner reported the same issue on the #ocaml IRC channel.

@vicuna
Copy link
Author

vicuna commented Dec 25, 2015

Comment author: @avsm

OPAM historically sets CAML_LD_LIBRARY_PATH to account for stub libraries installed by a user into the ~/.opam space but not added to the system compiler's ld.conf.

ocaml/opam#198

Is there a better way to add the ~/.opam stublibs path to the compiler search path other than setting CAML_LD_LIBRARY_PATH?

@github-actions
Copy link

This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc.

@github-actions github-actions bot added the Stale label May 11, 2020
@xavierleroy
Copy link
Contributor

@avsm:

OPAM historically sets CAML_LD_LIBRARY_PATH to account for stub libraries installed by a user into the ~/.opam space but not added to the system compiler's ld.conf.

We're not talking about the system compiler's ld.conf ! ocamlrun from an OPAM switch looks into $OPAM/$SWITCH/lib/ocaml/ld.conf, whose default contents are

$OPAM/$SWITCH/lib/stublibs
$OPAM/$SWITCH/lib/ocaml

Hence, setting the CAML_LD_LIBRARY_PATH to these two directories is just redundant when the ocamlrun from the switch is called, or incorrect when another ocamlrun is called.

Is there a better way to add the ~/.opam stublibs path to the compiler search path other than setting CAML_LD_LIBRARY_PATH?

Yes: do not set CAML_LD_LIBRARY_PATH at all!

@dbuenzli
Copy link
Contributor

@xavierleroy I suspect what you mention doesn't work with system switches where the ld.conf you mention is not managed by opam but by the system packager and will likely point to something like /usr/local/lib/ocaml/stublibs rather than the opam directory.

System switches have been an endless source of problems (I think they should have been killed) but unfortunately it seems users do like them.

@dra27
Copy link
Member

dra27 commented May 11, 2020

The versioning issue here will be dealt with as part of the relocatable compiler (stub libraries get the same treatment as ocamlrun under that, so in this misconfigured state you would get a message that the appropriate dllunix cannot be found, rather than the world falling apart as here).

This report pre-dates opam 2.0 which could improve on this very easily by moving the CAML_LD_LIBRARY_PATH logic from the ocaml package specifically to ocaml-system - this would mean that compilers which are compiled in ~/.opam would not set or mess around with CAML_LD_LIBRARY_PATH at all.

However, that only reduces the problem - I still don't think there's a way to add a directory to the stublib search path when you can't write to ld.conf (which is the case for a system package-manager maintained compiler).

@xavierleroy
Copy link
Contributor

Thanks for the feedback. I concede that CAML_LD_LIBRARY_PATH could need to be set for system switches. But for other switches (the ones I care about) I still contend it should not be set.

@dra27
Copy link
Member

dra27 commented Jul 21, 2021

Patch set which addresses this is in progress…

@github-actions github-actions bot added the Stale label Jul 27, 2022
@dra27 dra27 removed the Stale label Jul 27, 2022
@ocaml ocaml deleted a comment from github-actions bot Jul 27, 2022
@dra27
Copy link
Member

dra27 commented Jul 27, 2022

PRs for this are actually due soon (there’s a talk on the whole relocatable compiler at this year’s OCaml Workshop)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants