Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005547OCamlOCamlbuild (the tool)public2012-03-20 10:472014-07-16 12:14
Reportergasche 
Assigned To 
PrioritynormalSeverityfeatureReproducibilityN/A
StatusacknowledgedResolutionreopened 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version4.02.0+dev 
Summary0005547: Enable the "-use-ocamlfind" option by default
DescriptionOcamlfind is everywhere these days and I frequently forget to add the "-use-ocamlfind" knob to enable the nifty "package(foo)" syntax. I'm not sure why it was enabled by default (maybe there was a conflict in semantics somewhere) but I would like it if it became the standard (with why not an opt-out option "-no-ocamlfind").

It would be useful in particular when used with Oasis, which currently has the limitation that we can't pass command-line parameters to the ocamlbuild invocation directly. That's a bug that will eventually be fixed, but there may be similar situations present in other tools.
TagsNo tags attached.
Attached Files

- Relationships
related to 0005406resolvedgasche tag does not expect a parameter, but is used with parameter ... 
related to 0006118assignedgasche ocamlbuild non backward compatible change found 

-  Notes
(0007110)
gasche (developer)
2012-03-20 10:50

I have discussed this with xclerc and he agrees on principle -- but we should try to get feedback from the original implementors, npouillard and Romain Bardoux, before making changes. I'm directly marking the issue "acknowledged" to avoid burdening the bug triagers.

(Of course, feedback is still welcome.)
(0007116)
dbuenzli (reporter)
2012-03-20 11:59

Just two things.

1) What happens if you don't have ocamlfind installed ? The tool should still work IMHO.
2) For me I still have to invoke it sometimes without the option because of PR 0005531
(0007246)
william (reporter)
2012-03-29 17:50

For your information, I support this feature.

- having the option activated by default would be very nice, and one could deactivate it with "-no-ocamlfind".

- or activate it automatically when tag "package" is used

This is related to bug 0005406
(0007568)
meyer (developer)
2012-06-18 22:28

I agree with ubiquity of ocamlfind we should support it.

So based on your observations here are the remarks:

- ocamlbuild should still work if ocamlfind is not present and emit
  warning if any of the ocamlfind tags are used but findlib is not
  installed - this requires of course a check

- ocamlbuild should not fallback to ocamlfind (sounds a bit complicated)
  when none of the features ocamlfind are used. I have reservation to
  this point.

- it should show a nice error message if anybody tries to use any of
  these ocamlbuild tags, but again I am reserved about this - as one can
  imagine wants to override tags in the plugin, so extensibilty is
  maybe more important.

- there should be switch - -no-ocamlfind, where we shouldn't look at all
  if ocamlfind is installed

- of course it needs proper documentation and -use-ocamlfind switch
  marked as deprecated
(0007576)
william (reporter)
2012-06-19 09:20
edited on: 2012-06-19 09:23

Hello,
meyer, I would amend the behaviour that you imagined. The main principle would be not to have too many "automatic" features that often become a nightmare for both the developer and the user. What I would suggest :
- "-no-ocamlfind" is used : ocamlbuild does not try to use ocamlfind, and warns if ocamlfind tags are used
- "-no-ocamlfind" is not used : ocamlbuild tries to use ocamlfind, and makes an error if ocamlfind is not found, telling that ocamlfind is not found, and either to install findlib or to use "-no-ocamlfind"

based on your remarks, two would be modified :

- ocamlbuild should WORK WHITHOUT OCAMLFIND IF "-no-ocamlfind" IS USED and emit
  warning if any of the ocamlfind tags are used but findlib is not
  installed - this requires of course a check

- ocamlbuild should not fallback to ocamlfind (sounds a bit complicated)
  when none of the features ocamlfind are used. I have reservation to
  this point.
=> not applicable


And finally, the other possibility is do the exactly the opposite :
- if no tag is given, ocamlbuild does not try to use ocamlfind, and warns if ocamlfind tags are used
- if "use-ocamlfind" is given, use ocamlfind, and fails if ocamlfind is not found.
In this case, there is mainly only warnings to add to the current ocamlbuild version. This is what I implemented locally (did not have time yet to provide patch). But I think this solution is not as good as the one with "-no-ocamlfind".

Regards

(0007995)
meyer (developer)
2012-09-01 15:19

Agreed.

We'll mark the -use-ocamlfind switch as deprecated in this case.
(0008024)
william (reporter)
2012-09-04 22:59

Related to this,
It seems that ocamlbuild tries to simulate ocamlfind behaviour itself (in ocamlfind.ml), which does not feel sane...

With this, even when ocamlfind is not activated (in 3.12.1, with "use-ocamlfind" not used), ocamlbuild manages to compile some packages calling ocamlfind! (though it fails in the end).
(0008025)
meyer (developer)
2012-09-04 23:45

Indeed, this support is a bit outdated, I believe intentions were good, but it's confusing. I'll look into this area and consider depreceation or removal, with a reservation that some parts might be still useful.
(0009522)
Camarade_Tux (reporter)
2013-06-16 22:35

I was in favor of such a change at first but I see several issues now.

How to build the compiler (when ocamlfind isn't available)? Automatically fallback or explicitely disable ocamlfind would both work but that's still something to do

If ocamlbuild which is part of the standard distribution starts relying on ocamlfind, wouldn't it make sense to have ocamlfind be part of the standard distribution too? (As far as I'm concerned, I'd prefer to have ocaml{doc,debug,build,...} _outside_ of the core distribution and using ocamlfind; mostly because that ease updates and building of a cross-compiler).

This would conflict with oasis (or at least be made useless by oasis).

The support has been lagging so far with issues with threads iirc. I don't believe this would be better in the future: if there's something new in ocamlfind, it would need an update to the compiler and that's simply too much (did I mention I was in favor of splitting components?).

That said, this support is used, including by me, especially for short-lived codes, but that's when I don't want to take time to setup an actual build environment.



The way I see it:
- if ocamlfind lives outside of the core distribution, the support in ocamlbuild will most probably lag and it will be painful
- unless ocamlbuild gets a way to mix several "myocamlbuild.ml" files
- or the various tools in the ocaml distribution are split into dedicated packages that can be updated independently

All of these require work to be done before making ocamlbuild try to default to using ocamlfind. So as of today, I believe -use-ocamlfind shouldn't be the default and should be deprecated and removed in a future release.
(0009524)
gasche (developer)
2013-06-16 23:58

> How to build the compiler (when ocamlfind isn't available)?
> Automatically fallback or explicitely disable ocamlfind would both
> work but that's still something to do.

We would have an option -no-use-ocamlfind. Integrating that into the
distribution build system is an implementation detail; it has to be
done for the whole proposal to work, but I'm ready to take care or
that.

> If ocamlbuild which is part of the standard distribution starts
> relying on ocamlfind, wouldn't it make sense to have ocamlfind be
> part of the standard distribution too? (As far as I'm concerned, I'd
> prefer to have ocaml{doc,debug,build,...} _outside_ of the core
> distribution and using ocamlfind; mostly because that ease updates
> and building of a cross-compiler).

I am not aware of any plan to integrate more things in the standard
distribution. Moving ocaml{doc,build} and camlp4 outside the
distribution has been discussed. In any case, I don't think that's an
issue in practice given that ocamlfind has served us well for years
and has a remarkably stable interface and behaviour.

> This would conflict with oasis (or at least be made useless by oasis).

I don't understand this point; could you elaborate? Oasis has
a sophisticated plugin for ocamlfind support, but I don't think it is
designed as a build system auxiliary -- it's more than it was forced
to assume that role to work properly given the current limitations of
ocamlbuild. Improving ocamlbuild should make Oasis simpler; ideally
I think we would like Oasis to be rather agnostic of the build system
used, instead of reimplementing large part of its functionalities as
is currently the case. Given that there are numerous candidates for
build systems in OCaml (make, omake, ocamlbuild, ocp-build, jenga,
obuild to name a few), I think we want Oasis' evolution to be *less*
tied to ocamlbuild, rather than more.

> The support has been lagging so far with issues with threads
> iirc.

I don't know what you are referring to. Could you elaborate? Are you
talking about support for new ocamlfind features?

> That said, this support is used, including by me, especially for
> short-lived codes, but that's when I don't want to take time to
> setup an actual build environment.

What do you use otherwise?

> So as of today, I believe -use-ocamlfind shouldn't be the default
> and should be deprecated and removed in a future release.

What do you recommend using instead? The ocamlfind plugin that is
distributed inside Oasis?
(0009878)
meyer (developer)
2013-07-27 02:05

Fix on trunk, revision 13938.
(0009879)
meyer (developer)
2013-07-27 02:08

This will break OASIS and platform builds, so we have to think to up-stream the fixes before beta, but maybe at most after RC.
(0009880)
dbuenzli (reporter)
2013-07-27 02:30

Note that other players in the build area (obuild and ocp-build) provide built-in support for reading META files. That could be an alternative, less breaking, path.
(0009881)
hcarty (reporter)
2013-07-27 02:30

Will this make it into 4.01?
(0009882)
meyer (developer)
2013-07-27 02:44

dbuenzli: might be worth to try. I will experiment with that over the weekend.

hcarty: I think it will, if we are confident that rest of the infrastructure will be fixed.
(0009883)
meyer (developer)
2013-07-27 03:04

dbuenzli, I raised 6094, specifically for this work (parsing META files), as I found it a good feature.
(0009887)
gasche (developer)
2013-07-27 05:42

With the fix only in trunk and not in the version/4.01 branch, this will not (by default) get into 4.01.

I think that's the better choice, because this change may have far-raising consequences and the timeframe to fix things outside the distribution is too short. Really, at this point in the release cycle we shouldn't break outside stuff.
(0009938)
meyer (developer)
2013-07-28 16:25

I'm fine with shifting it to the next release, however I'd like people to test it with the platform now. If somebody can see if trunk still works with OPAM it would be great (I can do it myself, but currently my desktop machine is broken)
(0010030)
avsm (reporter)
2013-07-31 00:35

> If somebody can see if trunk still works with OPAM

I've just recompiled trunk and installed a few packages, and all seems normal enough. I'll kick off a bulk build later when some machines free up from 4.01.0dev tests.
(0010031)
meyer (developer)
2013-07-31 00:47

Thank you! Let me know how it goes, did you need to use -no-ocamlfind flag?
(I took great care to inform the user about deprecation of -use-ocamlfind)
(0010032)
avsm (reporter)
2013-07-31 00:52

I did not add any extra flags to any packages. I'll update this ticket if any regressions do show up.
(0010033)
meyer (developer)
2013-07-31 01:05

Interesting, I was expecting problems. Note that ocamlfind command will be always used for invoking the compiler, but maybe oasis does that by default too.
(0011470)
jpdeplaix (reporter)
2014-05-15 00:03

Related to: https://github.com/ocaml/ocaml/pull/20 [^]

Do we really want that ?
(0011546)
gasche (developer)
2014-05-22 12:03

We have to make a choice as to whether we keep the statu quo, which is to enable -use-ocamlfind by default, or revert the change and keep the default behavior (not using ocamlfind) of 4.01.

I'm "re-opening" this ticket as a central place for this discussion.

The argument in favor of changing the default is the following: all "small users" of ocamlbuild also use ocamlfind and always enable -use-ocamlfind, so enabling it by default is much more convenient. It also allows to document and discuss usage of ocamlbuild with ocamlfind integration directly, in a more user-friendly way. The -no-ocamlfind option would be reserved to expert/"large" users that have reasons to avoid ocamlfind (eg. Mirage).

The argument against are of various nature:
- changing a default behavior is always a bad thing
- this breaks a part of the packaging ecosystem as packages start depending on ocamlfind that didn't before; fixing this requires large-scale metadata information update
- many users are expected to use ocamlbuild indirectly through OASIS, so they're less concerned by the convenience aspect

A last remark is that this makes the default parameter for ocamlbuild (which is included in the OCaml distribution, at least for 4.02) depend on an external OCaml tool, which is "odd". I don't think this is problematic: arguably Stream depends on a camlp4 syntax extension for most sane uses, and generally it makes sense to make in-distribution tools interact better with the outside ecosystem when possible (of course a "bare" mode should be available, which it is).
Except for this last one, I consider all arguments above (in favor and against) valid.


External factors to consider are:
- the well-being of OPAM packagers and caretakers whose life is currently made difficult by the change ( discussion in https://github.com/ocaml/opam-repository/issues/2113 [^] )
- the serious proposal to split ocamlbuild outside the distribution for 4.03, which may inform decisions on the present issue ( http://caml.inria.fr/mantis/view.php?id=6402 [^] )


A possibility (suggested by Thomas) is to have ocamlbuild detect whether ocamlfind is available, and enable -use-ocamlfind by default only if it is -- curently it fails if ocamlfind is absent and -no-ocamlfind is not given. While nice on paper, I'm worried that this may create further problems down the line (harder to detect because of a moving default behavior).
(0011547)
avsm (reporter)
2014-05-22 12:14

I believe that changing the default behaviour while ocamlbuild is distributed as part of OCaml would cause more problems than it solves. It would be far simpler to proceed with the discussion in 0006402 to split ocamlbuild out in 4.03, and leave the resulting package free to depend on ocamlfind once it is external to the core distribution.

Also: how are packages supposed to use this across OCaml versions? Since `-no-ocamlfind` was only added in 4.02.0, it now requires a package that doesn't want ocamlfind to detect the version of OCaml it's compiling against, rather than just add a flag. Painful.
(0011549)
gasche (developer)
2014-05-22 14:29

My worry with the "let the splitters deal with the problem" argument is that it only delays the problem further. If the split-out package uses -use-ocamlfind by default, you will have the exact same backward-compatibility problem with -no-ocamlfind not being supporter on older versions. (Delaying the question from 4.01 to 4.02 didn't help, why would the move from 4.02 to 4.03 bring any clarity?)

I think we should make a decision now:
(1) either we decide to change a default behavior (bad!) and pay the associated costs (packaging changes and, yes, painful configure handling for some part of the ecosystem (which I hope is relatively dynamic))
(2) or we decide that it is too costly to change a default behavior and that it shall *never* change (... even in a split-off package)
(0011550)
gasche (developer)
2014-05-22 14:30

(Not sure why the status changes so much, it's not intentional; sorry for the noise)
(0011552)
avsm (reporter)
2014-05-22 16:48

Just add the -no-ocamlfind flag for this 4.02 release, and then the 4.03 upgrade path (spit or not) will be much easier.
(0011553)
ybarnoy (reporter)
2014-05-22 19:34

Maybe it's worth backporting this change to previous ocaml versions?
(0011581)
gasche (developer)
2014-05-29 10:22

Dear all,

Given the lack of apparent interest in the issue, my current plan is to heed Anil's advice and revert the change for 4.02. I'm not really satisfied with it, but time is running short and at some point you need to let go.

I plan to implement that shortly (before the week-end) so that it can be part of the next beta release, whenever that happens.
(0011582)
jpdeplaix (reporter)
2014-05-29 13:26

When you say « implement », do you suggest that there is more to do than just https://github.com/ocaml/ocaml/pull/60 [^] ?
(0011660)
gasche (developer)
2014-06-04 22:02

-no-ocamlfind is now the default again in 4.02 and trunk.

- Issue History
Date Modified Username Field Change
2012-03-20 10:47 gasche New Issue
2012-03-20 10:50 gasche Note Added: 0007110
2012-03-20 10:50 gasche Status new => acknowledged
2012-03-20 10:51 gasche Severity minor => tweak
2012-03-20 11:59 dbuenzli Note Added: 0007116
2012-03-29 17:50 william Note Added: 0007246
2012-03-29 19:14 gasche Relationship added related to 0005406
2012-06-11 04:21 meyer Assigned To => meyer
2012-06-11 04:21 meyer Status acknowledged => assigned
2012-06-18 22:28 meyer Note Added: 0007568
2012-06-19 09:20 william Note Added: 0007576
2012-06-19 09:22 william Note Edited: 0007576 View Revisions
2012-06-19 09:23 william Note Edited: 0007576 View Revisions
2012-09-01 15:19 meyer Note Added: 0007995
2012-09-04 22:59 william Note Added: 0008024
2012-09-04 23:45 meyer Note Added: 0008025
2012-09-06 16:43 doligez Target Version => 4.00.1+dev
2012-09-21 13:36 doligez Severity tweak => feature
2012-09-21 13:36 doligez Target Version 4.00.1+dev =>
2013-06-16 22:35 Camarade_Tux Note Added: 0009522
2013-06-16 23:58 gasche Note Added: 0009524
2013-07-27 02:05 meyer Note Added: 0009878
2013-07-27 02:08 meyer Note Added: 0009879
2013-07-27 02:30 dbuenzli Note Added: 0009880
2013-07-27 02:30 hcarty Note Added: 0009881
2013-07-27 02:44 meyer Note Added: 0009882
2013-07-27 03:04 meyer Note Added: 0009883
2013-07-27 05:42 gasche Note Added: 0009887
2013-07-28 16:25 meyer Note Added: 0009938
2013-07-28 16:26 meyer Status assigned => resolved
2013-07-28 16:26 meyer Fixed in Version => 4.01.1+dev
2013-07-28 16:26 meyer Resolution open => fixed
2013-07-31 00:35 avsm Note Added: 0010030
2013-07-31 00:47 meyer Note Added: 0010031
2013-07-31 00:52 avsm Note Added: 0010032
2013-07-31 01:05 meyer Note Added: 0010033
2013-08-21 11:12 doligez Fixed in Version 4.01.1+dev => 4.02.0+dev
2013-08-21 11:15 doligez Relationship added related to 0006118
2014-05-15 00:03 jpdeplaix Note Added: 0011470
2014-05-22 11:48 gasche Assigned To meyer =>
2014-05-22 11:48 gasche Status resolved => feedback
2014-05-22 11:48 gasche Resolution fixed => reopened
2014-05-22 12:03 gasche Note Added: 0011546
2014-05-22 12:03 gasche Status feedback => new
2014-05-22 12:06 gasche Status new => feedback
2014-05-22 12:14 avsm Note Added: 0011547
2014-05-22 14:29 gasche Note Added: 0011549
2014-05-22 14:29 gasche Status feedback => new
2014-05-22 14:30 gasche Note Added: 0011550
2014-05-22 14:30 gasche Status new => feedback
2014-05-22 16:48 avsm Note Added: 0011552
2014-05-22 19:34 ybarnoy Note Added: 0011553
2014-05-29 10:22 gasche Note Added: 0011581
2014-05-29 10:22 gasche Status feedback => new
2014-05-29 10:22 gasche Status new => feedback
2014-05-29 13:26 jpdeplaix Note Added: 0011582
2014-06-04 22:02 gasche Note Added: 0011660
2014-06-04 22:02 gasche Status feedback => new
2014-07-16 12:14 shinwell Status new => acknowledged


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker