Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ocamldep should have mode that does not make assumptions about build system #5396

Closed
vicuna opened this issue Nov 11, 2011 · 10 comments
Closed
Assignees

Comments

@vicuna
Copy link

vicuna commented Nov 11, 2011

Original bug ID: 5396
Reporter: dastapov
Assigned to: @lefessan
Status: closed (set by @xavierleroy on 2013-08-31T10:46:19Z)
Resolution: fixed
Priority: normal
Severity: feature
Version: 3.12.1
Fixed in version: 3.13.0+dev
Category: ~DO NOT USE (was: OCaml general)
Monitored by: @gasche dastapov @lefessan mehdi @ygrek

Bug description

Right now, ocamldep (without "-modules") makes too many assumptions about build system that it would be used with. In particular"

  1. It assumes that build system would not be able to distinguish between "ml only" and "ml+mli" cases on itself, so it uses the "cmo/cmx is a proxy for cmi" trick and outputs dependencies like "bar.cmi: foo.cmx", when in reality bar.cmi should depend on foo.cmi.

  2. It assumes that build system only checks modification time, so it does not output dependecies on .o files at all. In reality it is possible to change a module in such a way that .o would change, but .cmx would not (its md5sum would not change), and build system that would check modification time AND md5sum of files would not do a rebuild.

  3. It does not output dependencies on the source files

All this makes ocamldep broken and pretty much useless for general dependency processing (read: "if you are not using make, you are in a world of pain")

Let me make an example of how this could be improved. Consider the following toy project:

foo.ml:

let foo = 42

bar.ml:

let bar = Foo.foo + 1

bar.mli:

val bar : int

main.ml:

let () =
Print.printf "%d\n" Bar.bar

Here the output of "ocamldep -native *.ml *.mli":

bar.cmo: foo.cmx bar.cmi
bar.cmx: foo.cmx bar.cmi
foo.cmo:
foo.cmx:
main.cmo: bar.cmi
main.cmx: bar.cmx
bar.cmi:

With imaginary "-no-assumptions" mode the output should look like this:

foo.cmo: foo.ml
foo.cmx: foo.ml
foo.cmi: foo.ml
bar.cmi: bar.mli
bar.cmo: bar.ml foo.cmi bar.cmi foo.cmo
bar.cmx: bar.ml foo.cmi bar.cmi foo.cmx foo.o
main.cmi: main.ml
main.cmo: main.ml bar.cmi bar.cmo
main.cmx: main.ml bar.cmi bar.cmx bar.o

File attachments

@vicuna
Copy link
Author

vicuna commented Nov 15, 2011

Comment author: dastapov

Patch against svn-trunk attached, see the output of "ocamldep -full"

@vicuna
Copy link
Author

vicuna commented Jan 23, 2012

Comment author: @lefessan

Thanks for the patch, dastapov. Have you been using this modified ocamldep in some context ? Which one ?

In your patch, you put dependencies towards .o object files. Is there a reason for this ?

In the example given in the bug report:

bar.cmo: bar.ml foo.cmi bar.cmi foo.cmo
bar.cmx: bar.ml foo.cmi bar.cmi foo.cmx foo.o

there should be no dependencies towards .cmo and .o files (since they are not used by ocamlc/ocamlopt), i.e. the correct output should be:

bar.cmo: bar.ml foo.cmi bar.cmi
bar.cmx bar.o: bar.ml foo.cmi bar.cmi foo.cmx

Or is there another reason why you chose to put them explicitely ?

@vicuna
Copy link
Author

vicuna commented Jan 25, 2012

Comment author: dastapov

I've been using this ocamldep at JaneStreet, but not extensively. It performed fine.

Now let me address your questions:
1)Dependencies on .o are there because it is possible to change .ml so that after recompilation .cmx will remain the same (md5sum-wise), but .o would change:

$ echo 'let baz = "baz"' > baz.ml
$ ocamlopt -c baz.ml
$ md5sum baz.c* baz.o
ca6ab45ef85dd930e7f8e29ee4eddf8f baz.cmi
617e472bd3854192a30c443e313b2757 baz.cmx
894e61c407ac77f5b5f813008aec38e3 baz.o

$ echo 'let baz = "baz baz baz"' > baz.ml
$ ocamlopt -c baz.ml
$ md5sum baz.c* baz.o
ca6ab45ef85dd930e7f8e29ee4eddf8f baz.cmi
617e472bd3854192a30c443e313b2757 baz.cmx
d05d63f7d7fa0f638982ef3d8a0eacf0 baz.o

Therefore, lots of modern build systems (that check md5sum/sha1 of the targets) would behave incorrectly if the dependency is just on the .cmx

2)You are completely right about .cmo and .o, that is clearly my error.

@vicuna
Copy link
Author

vicuna commented Jan 25, 2012

Comment author: @lefessan

I understand the problem you show about the .cmx not changing (I think it would be worth another bug report, because many tools assume .cmx dependency is enough), but I still don't understand the need to put the .o in the .cmx dependencies. The .o is not used to build new pairs of .cmx/.o, but to build the final library or executable, so it should appear as a dependency of the .cmxa or the executable, together with the .cmx, no ?

@vicuna
Copy link
Author

vicuna commented Jan 25, 2012

Comment author: @alainfrisch

so it should appear as a dependency of the .cmxa or the executable

Nitpicking: formally, the .cmxa does not depend on the .o files either. It is the associated .a file which depend on them. Of course, the .cmxa and the .a are built by the same rule, which should list the .o as dependencies.

I think it would be worth another bug report, because many tools assume .cmx dependency is enough

Which tools? As far as I remember, default OCaml build rules shipped with omake are fine in this respect (and if this is not the case, this is a bug for omake, not ocaml). I'd be surprised if ocamlbuild was broken.

@vicuna
Copy link
Author

vicuna commented Jan 27, 2012

Comment author: dastapov

Now I remember why I put the .o dependency there. If you are using build system like "tup", you have an ability so say "give me all the .o that are transitive dependencies of this thing here". Then you use it to build .a. If your dependencies state that .cmx depends on .cmx and .o, then you can do it. If they don't - you would have much harder time figuring out the names of .o file (because not every source file would produce one - consider the case of lone .mli).

@vicuna
Copy link
Author

vicuna commented Jan 27, 2012

Comment author: dastapov

Which tools? As far as I remember, default OCaml build rules shipped with omake are fine in this respect (and if this is not the case, this is a bug for omake, not ocaml). I'd be surprised if ocamlbuild was broken.

Well, ocamlbuild is not broken specifically because it contains a workaround that adds dependecy on .o when there is dependecy on .cmx (you could check it out in the sources or try it out on my toy example that I've shown above). OMake is indeed broken because it does not have that workaround. But I don't feel like it is omake's fault.

The way I see it, this turns building ocaml into an obscure art, if you will, where you either use the One Proper Tool (ocamlbuild) or you are on your own and you basically have to read through the sources of ocamlbuild, ocamldep and parts of compiler before you could use ocaml in existing project with different build system. And even then you basically have to build from scratch since there is no tool that would give you the full dependency tree of the module or binary or library or, for example, list sources from the single directory, topologically sorted.

@vicuna
Copy link
Author

vicuna commented Jan 27, 2012

Comment author: @lefessan

I committed your patch partially in the SVN as r12089 and r12092. It will not print the dependencies towards .o files. "-full" was replaced by "-all". I also fixed a small mistake where dependencies towards .cmo file would be printed instead of towards the .cmi file (i.e. using a proxy again).

I also added an option "-sort", that prints the files in the order of their dependencies (like 'ocamlsort' would do).

@vicuna
Copy link
Author

vicuna commented Jan 27, 2012

Comment author: dastapov

Awesome (especially the "sort")!

@vicuna
Copy link
Author

vicuna commented Jan 31, 2012

Comment author: thelema

Another "Awesome!" on this improvement - I've been wishing for -sort for a long time, and it's great to now have it available as part of the standard distribution. Thank you, Fabrice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants