Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0006093OCamlOCamlbuild (the tool)public2013-07-27 02:572014-05-25 20:24
Reportermeyer 
Assigned To 
PriorityhighSeverityfeatureReproducibilityN/A
StatusassignedResolutionopen 
PlatformOSOS Version
Product Version4.02.0+dev 
Target Version4.02.0+devFixed in Version 
Summary0006093: Loading rules from ocamlfind package
DescriptionThis was proposed independently by gasche and me. We should support a command line flag, that will load selected ocamlfind library, with a side effect of updating the rules.
Additional InformationI set the target this release, as it would be really nice to have, especially that we already broken the backward compability by enabling ocamlfind by default.
TagsNo tags attached.
Attached Files

- Relationships
related to 0005680confirmed myocamlbuild.ml should be splittable into several files/plugins 

-  Notes
(0009884)
dbuenzli (reporter)
2013-07-27 03:24

This really needs to be carefully designed as it could be a cornerstone for ocamlbuild, namely reusable rulesets.

My first reaction is why a command line flag ? That solution entails that we cannot blindly invoke ocamlbuild in a directory. We need external knowledge and hence a driving shell script.

Why not a tag or something I can write in a _tags file ?

e.g. rules(ocamlfind_pkg) and rules(path_to_file.ml).
(0009885)
meyer (developer)
2013-07-27 03:46

Not oposing other solutions than flags, flag was just an example of the interface. I also prefer something that is stored on the file system.

And I also recognised this as an huge advantage over what is now. Finaly we could share the rules as libraries and not on wiki, and that's just sounds fantastic!
(0009888)
gasche (developer)
2013-07-27 05:49

Do take in account that it's early in the morning so I'm naturally a bit grumpy, but I think it's not reasonable to set this as a target for this release. This (and -use-ocamlfind by default etc.) are important design changes that should be done right on the first try, which means we will need some time to think about it more, possibly backtrack and change our mind, etc. Doing that in the hurry of a release is not reasonable. Besides, we have enough non-breaking but important issues (such as getting better error messages, or improving the documentation) to work on before the release!

I should you should set future such reports to after-next, with the understanding that if we get something we feel is ready before the timeline, it might get delivered before.
(0009894)
dbuenzli (reporter)
2013-07-27 12:41

I don't know when the release is but I also prefer a good design later rather than a flawed one now. However since it was hinted the dev team wants ocamlbuild out of the distribution anyway (which I can understand). Why not fork ocamlbuild out of the distribution and distribute it as a pinnable opam package. This would allow ocamlbuild to quickly ask for feedback to its users and/or allow a release cycle that is independent of the compiler.
(0009907)
meyer (developer)
2013-07-28 00:33
edited on: 2013-07-28 00:33

We also consider that, it might be an improvement to what we have now, but keeping the ocamlbuild in the distro has it's own merits too. OCamlbuild is tight to the release of OCaml but that's not that problematic as it appears, the user can still use the trunk compiler which is relatively stable.

From the other advantages, is that ocamlbuild would remain a reference build system, with possibility to implement new language features that build system depends on.

We also think that ocamlbuild is very easy to use, so for example, in teaching, hides all the details of the build system, and leaves the users with an easy interface to build theire software, so having that it there out of the box has a real benefit.

There are also technical reasons that's not so trivial to detach ocamlbuild, at the moment camlp4 depends on it for instance. We might have a clone hosted on the github and keep it in sync with the INRIA svn, that however risks forking the project and has also obvious licensing implications.

That's or other way, we are hearing voices saying that, so it would be good to think now about future proof strategy.

(0009908)
meyer (developer)
2013-07-28 01:34

So there are two ways of loading findlib package:

* when compiling the plugin myocamlbuild.ml which does not happen always
* dynlinking the cma or cmxs

Im my perception we should go for first option, forcing to compile myocamlbuild.
(0009910)
meyer (developer)
2013-07-28 01:40

Apart from that, it would be of course useful to make ocamlbuild recognise some of the rulset targets, and install them using findlib.
(0009911)
dbuenzli (reporter)
2013-07-28 02:39

Bear with me I don't know how these work.

But why not simply distribute rules as .ml files. Gather all the ml files corresponding to the rulesets mentioned in the _tags file, cat them and build the myocamlbuild plugin in the _build dir. If myocamlbuild.ml already exists proceed as it was always done (or try to add them in some way to that file).

Dynlinking seems more brittle to me, it's not supported on all platforms.
(0009917)
meyer (developer)
2013-07-28 03:54

So, the idea here is to use ocamlfind library management to distribute rules. Simple .ml files preclude it, although you might query the directory. In practice there is not much difference linking cmxa of cma than using ml file.

OCamlbuild plugins make it relatively easily (both solutions), because they just compile myocamlbuild.ml as a standalone executable which runs the compilation. So it's matter of adding "-package foo_rules" or as you suggested "<foo_rules_directory>/foo_rules.ml"

Compilation of rules might be a little bit slower than linking though, but I'd not think it would be a problem.
(0009918)
meyer (developer)
2013-07-28 03:55

Another advantage of using ml files versus libraries that they are source based, therefore the user might look at the source code at any time when the library is installed.
(0009919)
meyer (developer)
2013-07-28 07:41
edited on: 2013-07-28 07:43

We might go in completely flexible route, and allow the user to specify any options that will be catenated to the compilation command of myocamlbuild.ml.

Let's say we want to load rules from foo.build ocamlfind package, then we could say:

ocamlbuild -popts "-package foo.build" bar.native

or we want to add rules from a file in the source file in project directory:

ocamlbuild -popts build/foo.ml bar.native

Of course we want to avoid specyfing it in the command line so we can say in _tags:

popts(build/foo.ml)

or

popts(-package foo.build)

Note that this will work change semantics of _tags file, if the predicate is skipped then we have global level tags.

(0009920)
meyer (developer)
2013-07-28 07:51
edited on: 2013-07-28 13:34

Another idea: allow user to add new commandline options in the plugin.

For instance user wants to options and tags for the js_of_ocaml compiler.
Then in his myocamlbuild.ml he will say:

let jopts := ref ""
dispatch begin function
| Before_construct_options ->
  add_option ("-jopts", String (fun x ->  jopts := !jopts ^ " " ^ x, " js_of_ocaml 
options")
| _ -> () end


(EDIT: This become now: http://caml.inria.fr/mantis/view.php?id=6097 [^] )

(0009921)
gasche (developer)
2013-07-28 08:51

I'm not fond of the idea of concatenating source files into a final `myocamlbuild.ml`. One of the design decisions of ocamlbuild was to use OCaml as a scripting language. It has downsides, but it should also encourage us to think of the myocamlbuild as a regular OCaml program (or plugin, at least for now). Concatenating source code is not a very ocamlish solution. If we can make it convenient to use already-compiled OCaml modules as part of a project ocamlbuild description (for example through findlib), I feel this is a solution to be preferred.

(It is of course interesting to think about which aspects would be lacking with such a solution, and try to improve of them. Wojciech mentioned that direct source has the advantage of making it easy for the user to understand what's the semantic of the final myocamlbuild.ml. OCamlbuild has self-documentation features (ocamlbuild -documentation), is it possible to use them (after possible improvements) to get a similar level of understanding?)
(0009925)
meyer (developer)
2013-07-28 12:59

We don't have to catenate them. In short ocamlbuild will use something like:

ocamlopt -I +ocamlbuild ocamlbuildlib.a build/foo.ml myocamlbuild.ml

instead of:

ocamlopt -I +ocamlbuild ocamlbuildlib.a myocamlbuild.ml

or

ocamlfind ocamlopt -package foo.build -package ocamlbuild.lib -linkpkg myocamlbuild.ml


The approach is to allow user to add any needed options when compiling the plugin.
(0009927)
meyer (developer)
2013-07-28 13:16

Maybe it's not so obvious that actually the plugins are compiled and become standalone build systems on it's own during the building. So there is always a binary in the _build directory called myocamlbuild, which is invoked by ocamlbuild to build the sources. This makes the approach above possible without concatenating sources or using dynlinking.
(0009935)
dbuenzli (reporter)
2013-07-28 14:29

Sure concatening is very ugly. I didn't remember plugins were actually compiled to an executable it's better that way (and makes me wonder why all these years we were only allowed to have a single myocamlbuild.ml).
(0010116)
gasche (developer)
2013-08-05 18:32
edited on: 2013-08-05 18:47

Daniel and Wojciech, I made a lengthy comment about composability of ocamlbuild plugins over at PR#5680, which I think more accurately describes the topic of discussion ( http://caml.inria.fr/mantis/view.php?id=5680 [^] ). I would welcome your comments.


- Issue History
Date Modified Username Field Change
2013-07-27 02:57 meyer New Issue
2013-07-27 02:59 meyer Severity major => feature
2013-07-27 03:24 dbuenzli Note Added: 0009884
2013-07-27 03:46 meyer Note Added: 0009885
2013-07-27 05:49 gasche Note Added: 0009888
2013-07-27 09:18 meyer Target Version 4.01.0+dev => 4.01.1+dev
2013-07-27 12:41 dbuenzli Note Added: 0009894
2013-07-28 00:33 meyer Note Added: 0009907
2013-07-28 00:33 meyer Note Edited: 0009907 View Revisions
2013-07-28 00:33 meyer Note Edited: 0009907 View Revisions
2013-07-28 00:38 meyer Assigned To => meyer
2013-07-28 00:38 meyer Status new => assigned
2013-07-28 01:34 meyer Note Added: 0009908
2013-07-28 01:40 meyer Note Added: 0009910
2013-07-28 02:39 dbuenzli Note Added: 0009911
2013-07-28 03:54 meyer Note Added: 0009917
2013-07-28 03:55 meyer Note Added: 0009918
2013-07-28 07:41 meyer Note Added: 0009919
2013-07-28 07:43 meyer Note Edited: 0009919 View Revisions
2013-07-28 07:51 meyer Note Added: 0009920
2013-07-28 07:52 meyer Note Edited: 0009920 View Revisions
2013-07-28 07:52 meyer Note Edited: 0009920 View Revisions
2013-07-28 08:51 gasche Note Added: 0009921
2013-07-28 12:59 meyer Note Added: 0009925
2013-07-28 13:16 meyer Note Added: 0009927
2013-07-28 13:34 meyer Note Edited: 0009920 View Revisions
2013-07-28 14:29 dbuenzli Note Added: 0009935
2013-08-05 17:11 gasche Relationship added related to 0005680
2013-08-05 18:32 gasche Note Added: 0010116
2013-08-05 18:47 gasche Note Edited: 0010116 View Revisions
2014-01-21 11:54 doligez Assigned To meyer =>
2014-05-25 20:24 doligez Target Version 4.01.1+dev => 4.02.0+dev


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker