New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Some dependencies cannot be detected by ocamldep #5624
Comments
Comment author: @gasche ocamldep is an heuristic tool based on some assumptions about how OCaml users typically organize their development. I'm not sure it is reasonable to try to evolve it into an omniscient tool capable of dealing with your sophisticated file manipulations. That's what "by hand" compilation scripts (or special-case tools designed hand-in-hand with your quite specific build system) are for: absolute flexibility. I'm not convinced in particular that doing deep modification to the semantics of .cmi files and/or the compilation process to work around limitations of ocamldep is such a good idea. I would rather welcome richer ways to inform the compiler of the mapping from Ocaml world compilation unit names to filesystem object files than the current rules allow; and/or potentially a type-checker design facilitating discovery of dependencies and interaction with library providers and build systems (implement what currently is ocamldep as a layer between parsing and type-checking, that resolves such mapping, is able to access existing .cmi files for semantically accurate naming information, and to make asynchronous queries to a build system for the missing .cmi). |
Comment author: @alainfrisch I don't think it is reasonable to expect large code base to be based on a manual analysis of dependencies across the project: this needs to be automated. At least omake (and maybe ocamlbuild, I don't know) allows using customized rules (compiling individual files with different flags and -I dirs, copying rules, code generation like ocamlyacc and even when the generator tool itself is part of the project, etc) and still supports automatic analysis of dependencies. It really works well, except for the specific issue I raise here. Only my first proposal changes the "semantics" of the system. The second and third ones should not introduce any bad behavior. That said, I agree with you. More advanced build tools that communicate directly with the compilers would open new doors. |
Comment author: @damiendoligez I'm setting this PR to "feedback" because a fix is not obvious and it still needs discussion. |
Comment author: @alainfrisch Pushing to target version = "later". This problem has been around for a long time and is not critical. |
Comment author: @glondu In the given example, I wouldn't blame ocamldep, but rather a missing dependency from b.cmi to a.cmi. Why is the missing dependency from c.cmi to a.cmi considered a bug? |
Comment author: @alainfrisch ocamldep reports correctly that foo/b.mli depends on module A, which is then interpreted as a file dependency "foo/b.cmi: foo/a.cmi". Where do you see a missing dependency from b.cmi to a.cmi?
If foo/a.cmi changes, c.cmo needs to be rebuild. It is generally assumed that it is fine if ocamldep returns correct dependencies "modulo a transitive closure", because the build system will perform this transitive closure. But the example shows that this is not the case. When foo/a.mli changes, and we ask make to ensure that c.cmo is up to date, it will refresh b.cmi, forcing foo/b.cmi to be refreshed, forcing foo/a.cmi to be refreshed. But when c.ml is finally recompiled, nothing forced a.cmi to be refreshed by copying foo/a.cmi over. This is because c.cmo does not depend on a.cmi, which is precisely our problem here. |
Comment author: @glondu
I meant without the leading "foo/". If you add "b.cmi: a.cmi" (without leading "foo/") to your Makefile, then everything works as expected: First time$ make c.cmo Second time$ touch foo/a.mli Just to be sure...$ make c.cmo |
Comment author: @alainfrisch
Where would this information come from? Maybe the example with a Makefile is not very good, because the Makefile are usually not very compositional. Imagine an entire project controlled by, say, omake. In the foo/ directory, generic rules apply and deal with the dependency from b.cmi to a.cmi. Then, we somehow inject copy rules from foo/ to another directory (say, the root of the project, in the example), with proper dependencies (say, from b.cmi to foo/b.cmi). I don't know how to tell omake, in a modular way, that it should also copy the dependency from foo/b.cmi to foo/a.cmi in order to produce a dependency from b.cmi to a.cmi. And anyway, this would not be conceptually very clean: b.cmi does not depend on a.cmi, since b.cmi is obtained directly from foo/b.cmi; concretely, if I remove a.cmi at the root, there is no need to copy it back in order to bring b.cmi up to date. So forcing a dependency from b.cmi to a.cmi at the root is really a hack to work-around the problem. For me, the notion of dependency is rather simple: if the compilation command required to produce c.cmo needs to read a.cmi, then c.cmo depends on a.cmi. The fact is that ocamldep has no way to "predict" that. |
This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc. |
@nojb given that dune exists, is ocamldep supported any more? It's not mentioned as part of OCaml Platform. |
I guess the reason it is not mentioned by the platform docs is that it is supposed to be used behind the scenes by other tools (eg |
This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc. |
Concerning codept, it works on current OCaml version, and it is able to unroll dependencies generated by type abbreviations. Thus this issue is theoretically solved; even if in practice, there is no (few?) users of codept. |
Original bug ID: 5624
Reporter: @alainfrisch
Status: acknowledged (set by @damiendoligez on 2013-06-28T15:27:08Z)
Resolution: open
Priority: normal
Severity: minor
Target version: later
Category: tools (ocaml{lex,yacc,dep,debug,...})
Related to: #7080 #7470
Monitored by: @nojb @ygrek @glondu "Julien Signoles"
Bug description
ocamldep, because it operates purely syntactically on the source files, misses some dependencies need to unroll type abbreviations. Example: c.ml refers to a type B.t, which is defined in b.mli by "type t = A.t"; if c.ml does not otherwise refer to module A, there is no possibility for ocamldep to detect a dependency from c.cmo to a.cmi. Nevertheless, the compiler will indeed need a.cmi when compiling c.cmo.
Usually, this is not problematic, but it can be with non-trivial build rules which copy files around.
Example:
and the following Makefile
Dependencies have been obtained with
ocamldep -modules
, and then translated to file dependencies according to the corresponding-I
flags used for compilation (this is how ocamldep, and -- I assume -- ocamlbuild work). The inferred dependencies are:Then
make c.cmo
fails because of a missing dependency fromc.cmo
toa.cmi
.Several approaches are possible to "fix" this:
Prevent the compiler from looking for a file
a.cmi
unlessocamldep -modules
finds a dependency to moduleA
(i.e. run the dependency analysis before type-checking). This is not very nice, since some type abbreviations are interpreted as abstract types, and this can break type-checking in ways which are not straightforward to understand (the fix is to add a dummy reference toA
if the abbreviation is needed). This is the approach we follow at LexiFi.Include dependent
.cmi
. In the example above,b.cmi
would contain a copy ofa.cmi
. Of course, this can lead to very large.cmi
files.An optimization is to inline only the required external type abbreviations in the
.cmi
files (in the example above,b.cmi
would be extended to include the fact thatA.t
=int
). Note that this solution can also improve compilation time, because the compiler doesn't need to search and open.cmi
files only to expand abbreviations.The text was updated successfully, but these errors were encountered: