Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005950OCaml~DO NOT USE (was: OCaml general)public2013-03-14 20:132015-07-02 15:35
Assigned To 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0005950: relocatable ocaml compiler
DescriptionDear OCaml Team,

I wonder whether you considered making the ocaml compiler relocatable? Basically, this avoids hardcoding the "prefix", or any other paths that depend on the installation location, into the binaries. In consequence, the ocaml install can be moved around after compilation and installation. That would greatly simplify binary distribution.

A common technique is to compute the prefix from the path to the executable, as e.g. returned by Sys.executable_name. I actually tried to modify utils/ accordingly, but it seems it's not enough to make the compiler relocatable.

What do you think?
Steps To Reproduce./configure --prefix=/opt
make world opt opt.opt
make install
mv /opt /opt2
<compiler does not work anymore ...>
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
frisch (developer)
2013-03-14 20:24

FWIW, we do something along the line you proposer for LexiFi's version of OCaml, which we redistribute together with our application to our clients. In native code, this works well (and we do it only for native code). For bytecode (except in -custom mode), the tricky part is that the "header" (either a #! directive under Unix or a small executable) needs a way to locate ocamlrun which has no reason to be in a fixed position relative to the bytecode executable.
kanig (reporter)
2013-03-14 20:40

Thanks for your feedback.

Your patch for the native code sounds interesting. Can you share it? We do not actually distribute OCaml ourselves, but even the setup for nightly tests would be simplified with that.

I do not understand your remark concerning bytecode, but that's probably because I don't know much about the OCaml compiler. Is the path to the ocamlrun executable actually hardcoded in the result of bytecode compilation? Indeed, that would make it very hard to obtain something relocatable.

That makes me think that I should retry my own attempt, because I only tried bytecode, not native ... For our purposes, native would be sufficient as well.
xleroy (administrator)
2013-03-24 12:09

You can achieve some degree of relocation by setting the OCAMLLIB and CAML_LD_LIBRARY_PATH environment variables. This is what the Windows binary distribution of OCaml does.

One problem, mentioned by Alain, is for bytecode executables generated by ocamlc, more precisely: which ocamlrun is used to execute them? Under Unix, ocamlc hard-wires #!/some/path/to/ocamlrun. Under Windows, ocamlrun is searched in the PATH when the bytecode executable is started. None of these solutions is good.

Another problem is that the OCaml compilers can't assume that Sys.argv.(0) is the correct full path to their own executable. It is guaranteed under Linux because we use /proc/self/exe, a nonstandard Linux extension to the Unix standards. On other systems, we use portable heuristics that may fail.

Finally, the general trend recently is to make it easier to install and compile OCaml (via OPAM), rather than binary distribution.

Unless you can make a really convincing use case for relocating a binary distribution of OCaml on a non-Windows OS, I move to close this PR.
kanig (reporter)
2013-03-24 16:28

Thanks for your input. I have a few questions, although I am not sure if it the best place to discuss these issues.

Why the difference in behavior to locate ocamlrun, on Windows and Linux? It seems that searching the PATH is slightly better, because it actually allows relocating ocamlrun. But of course there is the danger of confusing two installed compilers.

Yes, looking at Sys.argv.(0) is not enough, in the general case you need to search in the PATH variable to find your own executable. I know that GCC does such trickery, and I assumed that Sys.executable_name also did. Am I wrong?

Why do you want me to provide a use case for non-Windows OS only, do you feel that the problem is solved on Windows?

Use cases for binary distribution are: shipping the OCaml compiler with another product (see Alain's comment), having a common OCaml install copied to different build systems (instead of having to compile it on each of them). There is no difference here between Windows and Linux.

But as you said, it is not a blocking issue, there are (at least) two workarounds: use the environment variables you mentioned, or compile from source.
xleroy (administrator)
2013-03-24 17:00

A few answers and then I "resolve/suspend" this PR.

> Why the difference in behavior to locate ocamlrun, on Windows and Linux?

Unix has the #! mechanism, which we really want to use for bytecode executables as it is compact and efficient, but it requires an absolute path: #!/path/to/ocamlrun. In contrast, Windows lacks any equivalent of #!, so we make do with a small C executable for which it was easier to search ocamlrun in PATH.

> It seems that searching the PATH is slightly better

Not in general, because you risk selecting the wrong ocamlrun to run your bytecode executable, if you have several versions of OCaml installed in different places and select one by playing on PATH (a common scenario - I do that all the time).

> Yes, looking at Sys.argv.(0) is not enough, in the general case you need to search in the PATH variable to find your own executable.

The OCaml runtime does this, obviously. But it is still not reliable, as nothing prevents the shell or the program launching your OCaml compiler to pass the wrong argv[0], e.g.

execl("/dir/not/in/path/ocamlc", "ocamlc", /* other args */, NULL);

> Why do you want me to provide a use case for non-Windows OS only, do you feel that the problem is solved on Windows?

Not 100%, but Windows is the only platform I know where it is customary to 1- install precompiled binaries, and 2- install them in any directory of the user's choice. The Unix tradition, as embodied in modern package managers, is either to compile from sources before installing, or install precompiled binaries at known locations (/usr/bin, etc).

frisch (developer)
2013-03-25 09:45

FWIW, we need to ship a binary Linux version of the toolchain (bytecode + native code) to a customer, and they need to be able to install it where they want. The current situation is fine for us, though. The tools we distribute are only the .opt versions (ocamlc.opt, ocamlopt.opt, ocamldoc.opt, ocamldep.opt, etc), and for the bytecode programs compiled by the customer, we advise them to use to use "-custom" or to call "ocamlrun" explicitly.
nycbrit (reporter)
2013-08-13 21:55

The "unix tradition" of fixed install paths causes insurmountable problems for users who don't have full control of their file system & can't rebuild the binary for logistical reasons.
Can you get ocamlc to look in the same (canonical) directory for ocamlrun?
samoht (reporter)
2015-07-02 14:56

> Finally, the general trend recently is to make it easier to install and compile OCaml (via OPAM), rather than binary distribution.

even with OPAM, having a relocatable compiler would be very helpful to avoid recompiling the same compiler on different switches. Could this be a configure-time option (which will be disable on systems where `/proc/self/exe` is not available?
frisch (developer)
2015-07-02 15:35

For the compilers, it's indeed quite easy to support relocation (using a relative path starting from the compiler/toplevel to locate the library directory). The difficult part is for bytecode user programs (not compiled with -custom) and this is likely to be required to build many packages. These programs somehow need a way to locate some compatible ocamlrun. Either this is searched in the PATH (as for Windows today; and perhaps using "#!/usr/bin/env ocamlrun" under Unix), but this means that the PATH must be altered, which is not very nice. Or we continue hard-coding the path to the desired ocamlrun in the bytecode program (and I think we should be uniform between Windows and Unix, i.e. the "prefix program" under Windows should extract the path to ocamlrun from the bytecode program), but then the same program would not work if installed on a machine where the toolchain is at a different location.

All in all, I believe that the proper way to address that would be to deprecate the notion of calling the bytecode program (if not compiled with -custom) directly, and encourage users to do instead "ocamlrun myprog". This way, they can choose whether to rely on the PATH or on a specific toolchain, and we remove the expectation that the bytecode program can find ocamlrun on its own (which is incompatible with the idea of relocation).

- Issue History
Date Modified Username Field Change
2013-03-14 20:13 kanig New Issue
2013-03-14 20:24 frisch Note Added: 0008966
2013-03-14 20:40 kanig Note Added: 0008967
2013-03-24 12:09 xleroy Note Added: 0009005
2013-03-24 12:09 xleroy Status new => feedback
2013-03-24 16:28 kanig Note Added: 0009008
2013-03-24 16:28 kanig Status feedback => new
2013-03-24 17:00 xleroy Note Added: 0009010
2013-03-24 17:00 xleroy Status new => closed
2013-03-24 17:00 xleroy Resolution open => suspended
2013-03-24 17:01 xleroy Status closed => resolved
2013-03-25 09:45 frisch Note Added: 0009011
2013-08-13 21:55 nycbrit Note Added: 0010175
2015-07-02 14:56 samoht Note Added: 0014183
2015-07-02 15:35 frisch Note Added: 0014184
2017-02-23 16:36 doligez Category OCaml general => -OCaml general
2017-03-03 17:55 doligez Category -OCaml general => -(deprecated) general
2017-03-03 18:01 doligez Category -(deprecated) general => ~deprecated (was: OCaml general)
2017-03-06 17:04 doligez Category ~deprecated (was: OCaml general) => ~DO NOT USE (was: OCaml general)

Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker