0005624: Some dependencies cannot be detected by ocamldep

Notes
(0007464) gasche (developer) 2012-05-25 17:58	ocamldep is an heuristic tool based on some assumptions about how OCaml users typically organize their development. I'm not sure it is reasonable to try to evolve it into an omniscient tool capable of dealing with your sophisticated file manipulations. That's what "by hand" compilation scripts (or special-case tools designed hand-in-hand with your quite specific build system) are for: absolute flexibility. I'm not convinced in particular that doing deep modification to the semantics of .cmi files and/or the compilation process to work around limitations of ocamldep is such a good idea. I would rather welcome richer ways to inform the compiler of the mapping from Ocaml world compilation unit names to filesystem object files than the current rules allow; and/or potentially a type-checker design facilitating discovery of dependencies and interaction with library providers and build systems (implement what currently is ocamldep as a layer between parsing and type-checking, that resolves such mapping, is able to access existing .cmi files for semantically accurate naming information, and to make asynchronous queries to a build system for the missing .cmi).

(0007465) frisch (developer) 2012-05-25 18:10	I don't think it is reasonable to expect large code base to be based on a manual analysis of dependencies across the project: this needs to be automated. At least omake (and maybe ocamlbuild, I don't know) allows using customized rules (compiling individual files with different flags and -I dirs, copying rules, code generation like ocamlyacc and even when the generator tool itself is part of the project, etc) and still supports automatic analysis of dependencies. It really works well, except for the specific issue I raise here. Only my first proposal changes the "semantics" of the system. The second and third ones should not introduce any bad behavior. That said, I agree with you. More advanced build tools that communicate directly with the compilers would open new doors.

(0007612) doligez (administrator) 2012-06-26 18:12	I'm setting this PR to "feedback" because a fix is not obvious and it still needs discussion. Everyone's input is welcome.

(0009464) frisch (developer) 2013-06-12 17:16	Pushing to target version = "later". This problem has been around for a long time and is not critical.

(0009467) glondu (reporter) 2013-06-12 17:53	In the given example, I wouldn't blame ocamldep, but rather a missing dependency from b.cmi to a.cmi. Why is the missing dependency from c.cmi to a.cmi considered a bug?

(0009469) frisch (developer) 2013-06-12 18:21	ocamldep reports correctly that foo/b.mli depends on module A, which is then interpreted as a file dependency "foo/b.cmi: foo/a.cmi". Where do you see a missing dependency from b.cmi to a.cmi? > Why is the missing dependency from c.cmi to a.cmi considered a bug? If foo/a.cmi changes, c.cmo needs to be rebuild. It is generally assumed that it is fine if ocamldep returns correct dependencies "modulo a transitive closure", because the build system will perform this transitive closure. But the example shows that this is not the case. When foo/a.mli changes, and we ask make to ensure that c.cmo is up to date, it will refresh b.cmi, forcing foo/b.cmi to be refreshed, forcing foo/a.cmi to be refreshed. But when c.ml is finally recompiled, nothing forced a.cmi to be refreshed by copying foo/a.cmi over. This is because c.cmo does not depend on a.cmi, which is precisely our problem here.

(0009471) glondu (reporter) 2013-06-12 20:15	> Where do you see a missing dependency from b.cmi to a.cmi? I meant without the leading "foo/". If you add "b.cmi: a.cmi" (without leading "foo/") to your Makefile, then everything works as expected: # First time $ make c.cmo ocamlc -c -I foo foo/a.mli ocamlc -c -I foo foo/b.mli cp foo/a.cmi a.cmi cp foo/b.cmi b.cmi ocamlc -c -I . c.ml # Second time $ touch foo/a.mli $ make c.cmo ocamlc -c -I foo foo/a.mli ocamlc -c -I foo foo/b.mli cp foo/a.cmi a.cmi cp foo/b.cmi b.cmi ocamlc -c -I . c.ml # Just to be sure... $ make c.cmo make: « c.cmo » est à jour.

(0009472) frisch (developer) 2013-06-12 21:08	> If you add "b.cmi: a.cmi" Where would this information come from? Maybe the example with a Makefile is not very good, because the Makefile are usually not very compositional. Imagine an entire project controlled by, say, omake. In the foo/ directory, generic rules apply and deal with the dependency from b.cmi to a.cmi. Then, we somehow inject copy rules from foo/ to another directory (say, the root of the project, in the example), with proper dependencies (say, from b.cmi to foo/b.cmi). I don't know how to tell omake, in a modular way, that it should also copy the dependency from foo/b.cmi to foo/a.cmi in order to produce a dependency from b.cmi to a.cmi. And anyway, this would not be conceptually very clean: b.cmi does not depend on a.cmi, since b.cmi is obtained directly from foo/b.cmi; concretely, if I remove a.cmi at the root, there is no need to copy it back in order to bring b.cmi up to date. So forcing a dependency from b.cmi to a.cmi at the root is really a hack to work-around the problem. For me, the notion of dependency is rather simple: if the compilation command required to produce c.cmo needs to read a.cmi, then c.cmo depends on a.cmi. The fact is that ocamldep has no way to "predict" that.

Issue History
Date Modified	Username	Field	Change
2012-05-25 15:12	frisch	New Issue
2012-05-25 17:58	gasche	Note Added: 0007464
2012-05-25 18:10	frisch	Note Added: 0007465
2012-06-26 18:12	doligez	Note Added: 0007612
2012-06-26 18:12	doligez	Status	new => feedback
2012-07-09 14:54	doligez	Target Version	=> 4.01.0+dev
2012-07-31 13:36	doligez	Target Version	4.01.0+dev => 4.00.1+dev
2012-09-21 14:26	doligez	Target Version	4.00.1+dev => 4.01.0+dev
2013-06-12 17:16	frisch	Note Added: 0009464
2013-06-12 17:16	frisch	Status	feedback => new
2013-06-12 17:16	frisch	Status	new => feedback
2013-06-12 17:16	frisch	Target Version	4.01.0+dev => later
2013-06-12 17:53	glondu	Note Added: 0009467
2013-06-12 18:21	frisch	Note Added: 0009469
2013-06-12 18:21	frisch	Status	feedback => new
2013-06-12 20:15	glondu	Note Added: 0009471
2013-06-12 21:08	frisch	Note Added: 0009472
2013-06-28 17:27	doligez	Status	new => acknowledged
2015-12-09 13:48	frisch	Relationship added	related to 0007080
2017-01-28 18:33	xleroy	Relationship added	related to 0007470
2017-02-23 16:45	doligez	Category	OCaml typing => typing
2017-03-10 09:25	shinwell	Category	typing => tools (ocaml{lex,yacc,dep,debug,...})