Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make ocamlrun versioned, make ocaml bytecode executable (non -custom) find runtime on PATH #6976

Closed
vicuna opened this issue Sep 1, 2015 · 9 comments

Comments

@vicuna
Copy link

vicuna commented Sep 1, 2015

Original bug ID: 6976
Reporter: Str
Status: closed (set by @mshinwell on 2016-12-08T08:41:33Z)
Resolution: won't fix
Priority: normal
Severity: feature
Platform: Unix
OS: any
OS Version: any
Version: 4.02.3
Category: runtime system and C interface
Monitored by: @gasche @diml

Bug description

Percieved shortcomings:

ocamlc non-custom executables have as first line a hash-bang path pointing possibly into the user-private opam directory.

Another problem with administering bytecode executables is that ocamlc output has a compiler version marker in the last bytes, requiring the corresponding ocamlrun, but versions are not visible.

I am proposing adding a version number to ocamlrun like ocamlrun.nnn, and building the non-custom bytecode with "#!/usr/bin/env ocamlrun.nnn".

This will ease (or rather 'make possible') ocamlc binary administration on the server and desktop.

@vicuna
Copy link
Author

vicuna commented Sep 2, 2015

Comment author: Str

steps:

  1. decide if using OCAML_VERSION_STRING or last 3 bytes of EXEC_MAGIC,
    the latter is shorter but cryptic
  2. make ocamlrun report version mismatch "found xxx, required xxx"
  3. build ocamlrun with name ocamlrunxxx
  4. change ocamlc to store desired hash-bang path /usr/bin/env ocamlrunxxx

@vicuna
Copy link
Author

vicuna commented Sep 13, 2015

Comment author: gerd

Isn't this more a shortcoming of opam, or a bad default? The point is that you can completely avoid this problem by choosing per-version install directories.

The problem is not restricted to ocamlrun. At least the dll's are also version-specific, and packages might need private files to run code. So you need the per-version directories anyway. ocamlrun is only the entry point.

Personally I never liked ~/.opam as default, because users don't think then about the install directory, but just take it, even when it is likely that they run into problems after the next upgrade.

There is one point worth thinking about, namely adding a strict(er) check of the binary compatibility, so that ocamlrun accepts only bytecode files it can really execute. This has three parts at least: format compatibility, instruction set compatibility, and primitive compatibility. The first (and maybe second) point is already addressed via the magic string. There is no check on primitive compatibility yet (e.g. a simple mechanism would be that the dll provides a version string).

@vicuna
Copy link
Author

vicuna commented Nov 20, 2015

Comment author: @xavierleroy

I'd like to understand the issue better, in particular w.r.t. OPAM. A good feature of OPAM is that it can handle multiple versions of OCaml, putting them into separate directories, e.g. ~/.opam/4.02.3/bin/ocamlrun. This is, in effect, the "per-version install directory" approach that Gerd mentions and that, in my opinion, is the best way to handle the issue.

@vicuna
Copy link
Author

vicuna commented Nov 20, 2015

Comment author: Str

The multiple version support of opam is just great. The problems are with flexible handling
of non-custom bytecode binaries, possibly from different compiler versions:

  • I want to run my binaries from /etc/cron.daily (or the like)
  • they depend on opam which currently is not recommended as site wide install
    (sure I can change that, but they still depend on a fixed ocamlrun location)
  • I can't copy ocamlrun to a usual server location (/usr/local/bin or so) because it
    will lose the property of the versioned path. I would have to recompile all my
    binaries with this exact version if I do that.
  • and I can't copy ocamlrun because the binaries have the fixed hash bang path

So in summary my binaries are always bound to a fixed path with the version in it, lacking flexibility, no matter where you put opam.

I switched to "ocamlc -custom", this avoids these problems but multiplies ocamlrun.

@vicuna
Copy link
Author

vicuna commented Nov 21, 2015

Comment author: gerd

What you can do is to set the opam destination to something like /opt/opam-4.02.3, which should generally work great, and copy this tree to machines where you need it. I guess the recommendation not to do a site-wide install just means not to install as root. Just use a different user account, perhaps a specially created one.

Note that some libraries (but only a few to my knowledge) are not fully self-contained anyway and need helper files, so I'd recommend to use this method even when you create -custom binaries.

Another note: actually you can relocate binaries with #! path by just prepending another #! path (the binary is decoded from the end). If you move ocamlrun around, this doesn't change the paths baked into ocamlrun, though, such as where to find libararies. Also be careful because there are only limited compatibility checks.

As you see, any relocation method raises questions, and they are difficult to answer, way beyond how a binary finds it interpreter.

@vicuna
Copy link
Author

vicuna commented Nov 21, 2015

Comment author: Str

Yes, some packages come with a .so-library that is not linked into the bytecode. And my opam path is /opt/opam/4.02.3.

Prepending another hash bang path is a hack that I would only do if it is a life saver.

I am sure we find a good way for dealing with shared libraries not statically linked in by ocaml{c,opt}

But this is not an argument against the changes I am proposing, it just means there still will be work to do. I think "devops" is not only a buzzword, but a sysadmin view is a valuable contribution.

Are there any unwanted side effects by making ocamlrun versioned, have it report byte code version mismatch exactly, and store a different #! path in the binary? My answer is no.

@vicuna
Copy link
Author

vicuna commented Nov 27, 2015

Comment author: @damiendoligez

Prepending another hash bang path is a hack that I would only do if it is a life saver.

Why? You want us to change the #! path for everyone. Gerd's point is that you can do it yourself in a very simple way after compilation (for example in your Makefile). This is a good workaround to use until we implement your feature (assuming we will).

@vicuna
Copy link
Author

vicuna commented Nov 28, 2015

Comment author: @xavierleroy

We already have two good solutions:
1- compile with ocamlopt (what most users do these days) or ocamlc -custom
2- install different versions of OCaml in different directories, which is what OPAM does automatically.

Versioning ocamlrun is a half-solution only, because shared libraries containing OCaml/C stub code would need versioning as well (see #6532).

All in all, that would be a lot of work for a problem that already has two solutions. I move to suspend this PR.

@vicuna
Copy link
Author

vicuna commented Dec 8, 2016

Comment author: @mshinwell

This proposal would also require some careful thinking about what exactly "version number" would mean.

Given the alternative approaches available to solving this problem it doesn't seem to me that changing this functionality is worth the fair amount of effort that it is going to entail. Given Xavier's previous response and the fact that there has been no further activity on this PR for the last year, I'm going to close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant