Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005396OCamlOCaml generalpublic2011-11-11 09:462013-08-31 12:46
Reporterdastapov 
Assigned Tolefessan 
PrioritynormalSeverityfeatureReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version3.12.1 
Target VersionFixed in Version3.13.0+dev 
Summary0005396: ocamldep should have mode that does not make assumptions about build system
DescriptionRight now, ocamldep (without "-modules") makes too many assumptions about build system that it would be used with. In particular"

   1. It assumes that build system would not be able to distinguish between "ml only" and "ml+mli" cases on itself, so it uses the "cmo/cmx is a proxy for cmi" trick and outputs dependencies like "bar.cmi: foo.cmx", when in reality bar.cmi should depend on foo.cmi.

   2. It assumes that build system only checks modification time, so it does not output dependecies on .o files at all. In reality it is possible to change a module in such a way that .o would change, but .cmx would not (its md5sum would not change), and build system that would check modification time AND md5sum of files would not do a rebuild.

   3. It does not output dependencies on the source files

All this makes ocamldep broken and pretty much useless for general dependency processing (read: "if you are not using make, you are in a world of pain")

Let me make an example of how this could be improved. Consider the following toy project:

foo.ml:

let foo = 42

bar.ml:

let bar = Foo.foo + 1

bar.mli:

val bar : int

main.ml:

let () =
  Print.printf "%d\n" Bar.bar

Here the output of "ocamldep -native *.ml *.mli":

bar.cmo: foo.cmx bar.cmi
bar.cmx: foo.cmx bar.cmi
foo.cmo:
foo.cmx:
main.cmo: bar.cmi
main.cmx: bar.cmx
bar.cmi:

With imaginary "-no-assumptions" mode the output should look like this:

foo.cmo: foo.ml
foo.cmx: foo.ml
foo.cmi: foo.ml
bar.cmi: bar.mli
bar.cmo: bar.ml foo.cmi bar.cmi foo.cmo
bar.cmx: bar.ml foo.cmi bar.cmi foo.cmx foo.o
main.cmi: main.ml
main.cmo: main.ml bar.cmi bar.cmo
main.cmx: main.ml bar.cmi bar.cmx bar.o

TagsNo tags attached.
Attached Filespatch file icon ocamldep.ml.patch [^] (8,113 bytes) 2011-11-15 10:35 [Show Content]

- Relationships

-  Notes
(0006202)
dastapov (reporter)
2011-11-15 10:36

Patch against svn-trunk attached, see the output of "ocamldep -full"
(0006788)
lefessan (developer)
2012-01-23 22:16

Thanks for the patch, dastapov. Have you been using this modified ocamldep in some context ? Which one ?

In your patch, you put dependencies towards .o object files. Is there a reason for this ?

In the example given in the bug report:

bar.cmo: bar.ml foo.cmi bar.cmi foo.cmo
bar.cmx: bar.ml foo.cmi bar.cmi foo.cmx foo.o

there should be no dependencies towards .cmo and .o files (since they are not used by ocamlc/ocamlopt), i.e. the correct output should be:

bar.cmo: bar.ml foo.cmi bar.cmi
bar.cmx bar.o: bar.ml foo.cmi bar.cmi foo.cmx

Or is there another reason why you chose to put them explicitely ?
(0006790)
dastapov (reporter)
2012-01-25 13:07

I've been using this ocamldep at JaneStreet, but not extensively. It performed fine.

Now let me address your questions:
1)Dependencies on .o are there because it is possible to change .ml so that after recompilation .cmx will remain the same (md5sum-wise), but .o would change:

$ echo 'let baz = "baz"' > baz.ml
$ ocamlopt -c baz.ml
$ md5sum baz.c* baz.o
ca6ab45ef85dd930e7f8e29ee4eddf8f baz.cmi
617e472bd3854192a30c443e313b2757 baz.cmx
894e61c407ac77f5b5f813008aec38e3 baz.o

$ echo 'let baz = "baz baz baz"' > baz.ml
$ ocamlopt -c baz.ml
$ md5sum baz.c* baz.o
ca6ab45ef85dd930e7f8e29ee4eddf8f baz.cmi
617e472bd3854192a30c443e313b2757 baz.cmx
d05d63f7d7fa0f638982ef3d8a0eacf0 baz.o

Therefore, lots of modern build systems (that check md5sum/sha1 of the targets) would behave incorrectly if the dependency is just on the .cmx

2)You are completely right about .cmo and .o, that is clearly my error.
(0006791)
lefessan (developer)
2012-01-25 14:42

I understand the problem you show about the .cmx not changing (I think it would be worth another bug report, because many tools assume .cmx dependency is enough), but I still don't understand the need to put the .o in the .cmx dependencies. The .o is not used to build new pairs of .cmx/.o, but to build the final library or executable, so it should appear as a dependency of the .cmxa or the executable, together with the .cmx, no ?
(0006801)
frisch (developer)
2012-01-25 20:11

> so it should appear as a dependency of the .cmxa or the executable

Nitpicking: formally, the .cmxa does not depend on the .o files either. It is the associated .a file which depend on them. Of course, the .cmxa and the .a are built by the same rule, which should list the .o as dependencies.

> I think it would be worth another bug report, because many tools assume .cmx dependency is enough

Which tools? As far as I remember, default OCaml build rules shipped with omake are fine in this respect (and if this is not the case, this is a bug for omake, not ocaml). I'd be surprised if ocamlbuild was broken.
(0006817)
dastapov (reporter)
2012-01-27 09:26

Now I remember why I put the .o dependency there. If you are using build system like "tup", you have an ability so say "give me all the .o that are transitive dependencies of this thing here". Then you use it to build .a. If your dependencies state that .cmx depends on .cmx and .o, then you can do it. If they don't - you would have much harder time figuring out the names of .o file (because not every source file would produce one - consider the case of lone .mli).
(0006819)
dastapov (reporter)
2012-01-27 09:34

> Which tools? As far as I remember, default OCaml build rules shipped with omake are fine in this respect (and if this is not the case, this is a bug for omake, not ocaml). I'd be surprised if ocamlbuild was broken.

Well, ocamlbuild is not broken specifically because it contains a workaround that adds dependecy on .o when there is dependecy on .cmx (you could check it out in the sources or try it out on my toy example that I've shown above). OMake is indeed broken because it does not have that workaround. But I don't feel like it is omake's fault.

The way I see it, this turns building ocaml into an obscure art, if you will, where you either use the One Proper Tool (ocamlbuild) or you are on your own and you basically have to read through the sources of ocamlbuild, ocamldep and parts of compiler before you could use ocaml in existing project with different build system. And even then you basically have to build from scratch since there is no tool that would give you the full dependency tree of the module or binary or library or, for example, list sources from the single directory, topologically sorted.
(0006827)
lefessan (developer)
2012-01-27 15:00

I committed your patch partially in the SVN as r12089 and r12092. It will not print the dependencies towards .o files. "-full" was replaced by "-all". I also fixed a small mistake where dependencies towards .cmo file would be printed instead of towards the .cmi file (i.e. using a proxy again).

I also added an option "-sort", that prints the files in the order of their dependencies (like 'ocamlsort' would do).
(0006828)
dastapov (reporter)
2012-01-27 15:03

Awesome (especially the "sort")!
(0006863)
thelema (reporter)
2012-01-31 21:41

Another "Awesome!" on this improvement - I've been wishing for -sort for a long time, and it's great to now have it available as part of the standard distribution. Thank you, Fabrice!

- Issue History
Date Modified Username Field Change
2011-11-11 09:46 dastapov New Issue
2011-11-15 10:35 dastapov File Added: ocamldep.ml.patch
2011-11-15 10:36 dastapov Note Added: 0006202
2012-01-23 22:16 lefessan Note Added: 0006788
2012-01-25 13:07 dastapov Note Added: 0006790
2012-01-25 14:42 lefessan Note Added: 0006791
2012-01-25 20:11 frisch Note Added: 0006801
2012-01-27 09:26 dastapov Note Added: 0006817
2012-01-27 09:34 dastapov Note Added: 0006819
2012-01-27 15:00 lefessan Note Added: 0006827
2012-01-27 15:03 dastapov Note Added: 0006828
2012-01-30 16:49 lefessan Status new => resolved
2012-01-30 16:49 lefessan Fixed in Version => 3.13.0+dev
2012-01-30 16:49 lefessan Resolution open => fixed
2012-01-30 16:49 lefessan Assigned To => lefessan
2012-01-31 21:41 thelema Note Added: 0006863
2013-08-31 12:46 xleroy Status resolved => closed


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker