Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loading rules from ocamlfind package #6093

Closed
vicuna opened this issue Jul 27, 2013 · 18 comments
Closed

Loading rules from ocamlfind package #6093

vicuna opened this issue Jul 27, 2013 · 18 comments

Comments

@vicuna
Copy link

vicuna commented Jul 27, 2013

Original bug ID: 6093
Reporter: meyer
Assigned to: @gasche
Status: closed (set by @xavierleroy on 2016-12-07T10:47:08Z)
Resolution: fixed
Priority: high
Severity: feature
Version: 4.02.0+dev
Target version: undecided
Category: -for ocamlbuild use https://github.com/ocaml/ocamlbuild/issues
Related to: #5680
Monitored by: @gasche @hcarty gerd

Bug description

This was proposed independently by gasche and me. We should support a command line flag, that will load selected ocamlfind library, with a side effect of updating the rules.

Additional information

I set the target this release, as it would be really nice to have, especially that we already broken the backward compability by enabling ocamlfind by default.

@vicuna
Copy link
Author

vicuna commented Jul 27, 2013

Comment author: @dbuenzli

This really needs to be carefully designed as it could be a cornerstone for ocamlbuild, namely reusable rulesets.

My first reaction is why a command line flag ? That solution entails that we cannot blindly invoke ocamlbuild in a directory. We need external knowledge and hence a driving shell script.

Why not a tag or something I can write in a _tags file ?

e.g. rules(ocamlfind_pkg) and rules(path_to_file.ml).

@vicuna
Copy link
Author

vicuna commented Jul 27, 2013

Comment author: meyer

Not oposing other solutions than flags, flag was just an example of the interface. I also prefer something that is stored on the file system.

And I also recognised this as an huge advantage over what is now. Finaly we could share the rules as libraries and not on wiki, and that's just sounds fantastic!

@vicuna
Copy link
Author

vicuna commented Jul 27, 2013

Comment author: @gasche

Do take in account that it's early in the morning so I'm naturally a bit grumpy, but I think it's not reasonable to set this as a target for this release. This (and -use-ocamlfind by default etc.) are important design changes that should be done right on the first try, which means we will need some time to think about it more, possibly backtrack and change our mind, etc. Doing that in the hurry of a release is not reasonable. Besides, we have enough non-breaking but important issues (such as getting better error messages, or improving the documentation) to work on before the release!

I should you should set future such reports to after-next, with the understanding that if we get something we feel is ready before the timeline, it might get delivered before.

@vicuna
Copy link
Author

vicuna commented Jul 27, 2013

Comment author: @dbuenzli

I don't know when the release is but I also prefer a good design later rather than a flawed one now. However since it was hinted the dev team wants ocamlbuild out of the distribution anyway (which I can understand). Why not fork ocamlbuild out of the distribution and distribute it as a pinnable opam package. This would allow ocamlbuild to quickly ask for feedback to its users and/or allow a release cycle that is independent of the compiler.

@vicuna
Copy link
Author

vicuna commented Jul 27, 2013

Comment author: meyer

We also consider that, it might be an improvement to what we have now, but keeping the ocamlbuild in the distro has it's own merits too. OCamlbuild is tight to the release of OCaml but that's not that problematic as it appears, the user can still use the trunk compiler which is relatively stable.

From the other advantages, is that ocamlbuild would remain a reference build system, with possibility to implement new language features that build system depends on.

We also think that ocamlbuild is very easy to use, so for example, in teaching, hides all the details of the build system, and leaves the users with an easy interface to build theire software, so having that it there out of the box has a real benefit.

There are also technical reasons that's not so trivial to detach ocamlbuild, at the moment camlp4 depends on it for instance. We might have a clone hosted on the github and keep it in sync with the INRIA svn, that however risks forking the project and has also obvious licensing implications.

That's or other way, we are hearing voices saying that, so it would be good to think now about future proof strategy.

@vicuna
Copy link
Author

vicuna commented Jul 27, 2013

Comment author: meyer

So there are two ways of loading findlib package:

  • when compiling the plugin myocamlbuild.ml which does not happen always
  • dynlinking the cma or cmxs

Im my perception we should go for first option, forcing to compile myocamlbuild.

@vicuna
Copy link
Author

vicuna commented Jul 27, 2013

Comment author: meyer

Apart from that, it would be of course useful to make ocamlbuild recognise some of the rulset targets, and install them using findlib.

@vicuna
Copy link
Author

vicuna commented Jul 28, 2013

Comment author: @dbuenzli

Bear with me I don't know how these work.

But why not simply distribute rules as .ml files. Gather all the ml files corresponding to the rulesets mentioned in the _tags file, cat them and build the myocamlbuild plugin in the _build dir. If myocamlbuild.ml already exists proceed as it was always done (or try to add them in some way to that file).

Dynlinking seems more brittle to me, it's not supported on all platforms.

@vicuna
Copy link
Author

vicuna commented Jul 28, 2013

Comment author: meyer

So, the idea here is to use ocamlfind library management to distribute rules. Simple .ml files preclude it, although you might query the directory. In practice there is not much difference linking cmxa of cma than using ml file.

OCamlbuild plugins make it relatively easily (both solutions), because they just compile myocamlbuild.ml as a standalone executable which runs the compilation. So it's matter of adding "-package foo_rules" or as you suggested "<foo_rules_directory>/foo_rules.ml"

Compilation of rules might be a little bit slower than linking though, but I'd not think it would be a problem.

@vicuna
Copy link
Author

vicuna commented Jul 28, 2013

Comment author: meyer

Another advantage of using ml files versus libraries that they are source based, therefore the user might look at the source code at any time when the library is installed.

@vicuna
Copy link
Author

vicuna commented Jul 28, 2013

Comment author: meyer

We might go in completely flexible route, and allow the user to specify any options that will be catenated to the compilation command of myocamlbuild.ml.

Let's say we want to load rules from foo.build ocamlfind package, then we could say:

ocamlbuild -popts "-package foo.build" bar.native

or we want to add rules from a file in the source file in project directory:

ocamlbuild -popts build/foo.ml bar.native

Of course we want to avoid specyfing it in the command line so we can say in _tags:

popts(build/foo.ml)

or

popts(-package foo.build)

Note that this will work change semantics of _tags file, if the predicate is skipped then we have global level tags.

@vicuna
Copy link
Author

vicuna commented Jul 28, 2013

Comment author: meyer

Another idea: allow user to add new commandline options in the plugin.

For instance user wants to options and tags for the js_of_ocaml compiler.
Then in his myocamlbuild.ml he will say:

let jopts := ref ""
dispatch begin function
| Before_construct_options ->
  add_option ("-jopts", String (fun x ->  jopts := !jopts ^ " " ^ x, " js_of_ocaml options")
| _ -> () end

(EDIT: This become now: #6097 )

@vicuna
Copy link
Author

vicuna commented Jul 28, 2013

Comment author: @gasche

I'm not fond of the idea of concatenating source files into a final myocamlbuild.ml. One of the design decisions of ocamlbuild was to use OCaml as a scripting language. It has downsides, but it should also encourage us to think of the myocamlbuild as a regular OCaml program (or plugin, at least for now). Concatenating source code is not a very ocamlish solution. If we can make it convenient to use already-compiled OCaml modules as part of a project ocamlbuild description (for example through findlib), I feel this is a solution to be preferred.

(It is of course interesting to think about which aspects would be lacking with such a solution, and try to improve of them. Wojciech mentioned that direct source has the advantage of making it easy for the user to understand what's the semantic of the final myocamlbuild.ml. OCamlbuild has self-documentation features (ocamlbuild -documentation), is it possible to use them (after possible improvements) to get a similar level of understanding?)

@vicuna
Copy link
Author

vicuna commented Jul 28, 2013

Comment author: meyer

We don't have to catenate them. In short ocamlbuild will use something like:

ocamlopt -I +ocamlbuild ocamlbuildlib.a build/foo.ml myocamlbuild.ml

instead of:

ocamlopt -I +ocamlbuild ocamlbuildlib.a myocamlbuild.ml

or

ocamlfind ocamlopt -package foo.build -package ocamlbuild.lib -linkpkg myocamlbuild.ml

The approach is to allow user to add any needed options when compiling the plugin.

@vicuna
Copy link
Author

vicuna commented Jul 28, 2013

Comment author: meyer

Maybe it's not so obvious that actually the plugins are compiled and become standalone build systems on it's own during the building. So there is always a binary in the _build directory called myocamlbuild, which is invoked by ocamlbuild to build the sources. This makes the approach above possible without concatenating sources or using dynlinking.

@vicuna
Copy link
Author

vicuna commented Jul 28, 2013

Comment author: @dbuenzli

Sure concatening is very ugly. I didn't remember plugins were actually compiled to an executable it's better that way (and makes me wonder why all these years we were only allowed to have a single myocamlbuild.ml).

@vicuna
Copy link
Author

vicuna commented Aug 5, 2013

Comment author: @gasche

Daniel and Wojciech, I made a lengthy comment about composability of ocamlbuild plugins over at #5680, which I think more accurately describes the topic of discussion ( #5680 ). I would welcome your comments.

@vicuna
Copy link
Author

vicuna commented Jan 18, 2015

Comment author: @gasche

This was mostly solved in 4.01 with the -plugin-tags option that allows to compile the myocamlbuild.ml tags with arbitrary flags (in particular package(foo)). I suspect there are still tweak to do to the whole mechanism, so one should feel free to create new issues for use-cases that are not well-supported currently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants