New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set compiler output directory #3757
Comments
Comment author: hw I see there's not been any activity on this ticket for 5 years. This feature would be useful to me for the same reasons. Having the source, intermediate and output files all mixed up in the same directory gets pretty unwieldy once you have a large set of input files, especially in my setup where version control and editing happens on one system and compilation on another. For my file browser to be a useful way of examining the structure of my project I first have to do a rm *.{annot,cmi,cmx,cmo,o,a,cmxa} just to see what source files I'm working with. |
Comment author: @mshinwell This can be done using the "-o" option. |
I am reopening this issue becuase, no, Also, the issue is not at all Windows-specific and is still present in the It seems there are two possible approaches to let users specify
With the second option one would need to With the It seems @dra27 prefers |
Personally am not sure of the relevance of this particular request (which in 15 years attracted a grand total of 1 comment, over 10 years ago). I am not against adding a feature in this direction, but the stated motivations ("source control reasons") can all be fixed by using modern tools (build systems, editors); designing and agreeing on the semantics of a new flag just for this seems overkill. |
Let's also ping @dbuenzli |
Indeed, the original problem here is solved with |
Could you elaborate what you mean by or relative to it? As far as I understand, this is the current behavior of the compiler: source and target filenames are interpreted relatively to the current directory. On the contrary, I should admit I understand the point of view that the build logic should be handled as much as possible by the build system, keeping the compiler behavior simple. The only benefit I see for handling out-of-source build directly in the compiler is to get correct paths in |
Nicolás Ojeda Bär (2020/08/20 07:00 -0700):
Personally am not sure of the relevance of this particular request
(which in 15 years attracted a grand total of 1 comment, over 10 years
ago).
I am not against adding a feature in this direction, but the stated
motivations ("source control reasons") can all be fixed by using
modern tools (build systems, editors); designing and agreeing on the
semantics of a new flag just for this seems overkill.
I am copying here what I just wrote on the caml-devel pailing list,
perhaps the discussion shoudl go on here so that everybody can participate:
Several build systems do out-of-source-tree builds,
which I guess is enough to show that the feature is desired. Then, in my
opinion, the fact that each build system has to re-implement the feature
shows that build systems are not the right place to address it. Or, to be
more precise, that one could avoid a certain number of hacks and
simplify the implementation if the compiler itself would have a better
support for that.
In addition to the out-of-source builds, such a flag would be helpful to
distinguish build artifacts produced by different compilers, which is a
need that became obvious recently when we wanted to distinguish the
files produced by the bootstrap compiler from those produced by the
compiled compiler. In the same vein, when @Octachron and I supervised
an Outreachy internship about evaluating test coverage by the
testsuite, we noticed it was hard to re-compile an instrumented compiler
precisely because it was not possible to distinguish the instrumented
object files from their non-instrumented counterparts.
… |
David Allsopp (2020/08/20 07:07 -0700):
Indeed, the original problem here is solved with `-o` and ensuring
that you link only compiled object files (`ocamlc -o foo/bar foo.ml`
will put `foo.cmo` and `foo.cmi` in the current directory; all other
uses of `-o` with `-c` should put all artefacts in the directory
specified with `-o`)
Am I correct that you are finally suggesting to _not_ add any new flag,
@dra27, and to just be disciplined in how one uses the compiler?
Are you saying this would be enough?
|
thierry-martinez (2020/08/20 07:16 -0700):
> 2\. A `-ocwd` option which would put build artifacts in the current
> directory (or relative to it).
Could you elaborate what you mean by _or relative to it_? As far as I
understand, this is the current behavior of the compiler: source and
target filenames are interpreted relatively to the current directory.
For those mentionned on the command-line, yes. But for the names of
generated files, I think the compiler does not handle them in a coherent
way between compiling and linking.
On the contrary, `-ocwd` would make sense if build artifacts were
systematically put in the current directory, regardless of the
relative path of the source.
Yes, this makes sense to me.
I should admit I understand the point of view that the build logic
should be handled as much as possible by the build system, keeping the
compiler behavior simple.
Even if that means duplicating problem-resolution between several build
systems? Isn't it worth finding out what is needed in common and
integrating that to the compiler?
How about people who prefer not to rely on a particular build system?
The only benefit I see for handling out-of-source build directly in
the compiler is to get correct paths in `.cmt` files when compiling
with `absname=1`. Note that having paths on the copies would not be
problematic if `dune` made use of symbolic links instead of
duplicating `.ml` files in the `_build/` directory.
Have you considered reporting this as a Dune issue?
|
Apologies for being stubborn, but I still don't understand what it is that these build systems reimplement. For example, as far as I know, Perhaps it would be clearer if you could explain how build systems are supposed to use the new flag. Thanks! |
As far as I know, |
Yes, but this is not directly related to doing out-of-tree builds, it could use the files directly from the source tree, as far as I understand. |
Nicolás Ojeda Bär (2020/08/20 07:45 -0700):
> Several build systems do out-of-source-tree builds, which I guess is enough to show that the feature is desired. Then, in my opinion, the fact that each build system has to re-implement the feature shows that build systems are not the right place to address it. Or, to be more precise, that one could avoid a certain number of hacks and simplify the implementation if the compiler itself would have a better support for that.
Apologies for being stubborn, but I still don't understand what *it*
is that these build systems reimplement.
It may very well be that I am wrong here.
As far as I can remember, ocamlbuild and perhaps Dune had to either
symlink or copy all the files from the source tree to the build tree so
that they could then work exclusively in the build tree and leave the
source tree untouched.
Am I wrong here? Is -o now enough so that it is no longer necessary to
copy/symlik files from the source to the build tree?
For example, as far as I know, `dune` does out-of-tree builds just by
using `-o` to put the compilation artifacts where it wants them to be.
That's where I may be wrong then.
Incidentally, `dune` also needs to use `-o` in order to implement
"prefixed" libraries and executables, so it is not clear to me it
would be able to take advantage of the hypothetical `-odir/-ocwd`
flag.
Well it could be that it does not work for everything, but let's see
whether it would improve the situation at least a bit. :)
Perhaps it would be clearer if you could explain how build systems are
supposed to use the new flag. Thanks!
My current understanding is that if one uses the compiler "carefully",
i.e. if one compiles first and then links, then yes, -o can be enough to
control where build artifacts go and make sure none of them is added to
the source tree. If, however, one compiles and links at the same time,
then some artifacts will go to the source tree. The idea of that flag
would be to guarantee that nothing is ever added to the source tree,
whehter the compiler is used to compile, link or both.
|
Nicolás Ojeda Bär (2020/08/20 14:51 +0000):
> As far as I know, `dune` first copies all the source files in `_build` before running the compiler on these copies.
Yes, but this is not directly related to doing out-of-tree builds, it
could use the files directly from the source tree (as far as I
understand).
Are we sure?
Why does it copy them, then?
|
@nojb, you are right: |
One key reason for copying files around is to implement sandboxing: to make sure that only the dependencies stated in the Note that even while |
On the top of my head I think
|
I'm saying that it is possible to solve the problem here with the current implementation of The limitation which could be overcome with a new flag is that when you invoke the compiler, you must be in a consistent directory relative to your source file: mkdir source build
echo 'let answer = 42' > source/bar.ml
echo 'let answer = Bar.answer' > source/foo.ml
cd source
ocamlc -c -o ../build/bar.cmo bar.ml
ocamlc -c -o ../build/foo.cmo foo.ml the last command fails unless you add cd ../build
ocamlc -c -o bar.cmo ../source/bar.ml
ocamlc -c -o foo.cmo ../source/foo.ml except that |
thierry-martinez (2020/08/20 07:59 -0700):
> Yes, but this is not directly related to doing out-of-tree builds, it could use the files directly from the source tree, as far as I understand.
You are right: `ocamlc -c -o $BUILDDIR/x.cmo $SRCDIR/x.ml` appears to
produce all build artifacts in `$BUILDDIR`, I did not know that!
@shindere , would `-odir $BUILDDIR` make something else than that?
I guess it would in addition work also when compiling and linking are
performed simultaneously.
|
Many thanks for the dune-related information, @nojb. I was not at all
aware and find having this confusion cleared out very helpful.
|
Daniel Bünzli (2020/08/20 08:10 -0700):
On the top of my head I think `-o` is mostly fine nowadays to do out of source tree builds and be able to mix your source arguments from wherever they live without having to copy them. There are however two remaining outstanding issues as far as I'm concerned:
1. You can't use `-o` for compiling C files. You need to cwd to the
output directory. This is #7677.
Thanks. Will work on it.
2. You can't compile an OCaml implementation against an OCaml
interface that is not in the same directory, because of an absurd
(because semantically meaningless) file check. This is #9717.
Will work on that, too. Many thanks for the so detailed report, it's
very very helpful.
|
David Allsopp (2020/08/20 08:29 -0700):
I'm saying that it is _possible_ to solve the problem here with the
current implementation of `-o` combined with the discipline of not
compiling and linking at the same time.
Modulo the problems reported by @dbuenzli, I guess.
I don't think this was possible when the original issue was opened.
OK.
The limitation which _could_ be overcome with a new flag is that
when you invoke the compiler, you must be in a consistent directory
relative to your source file: `ocamlc -c foo/bar/baz.mli` and
`cd foo ; ocamlc -o bar/baz.cmi bar/baz.mli` generate a different
interface.
How do the interfaces differ, actually?
The benefit to being able to compile your object files
in the current directory is that you don't need to be including
the output directory with `-I` in order to pick up any _other_
modules already compiled. Consider:
```bash
mkdir source build
echo 'let answer = 42' > source/bar.ml
echo 'let answer = Bar.answer' > source/foo.ml
cd source
ocamlc -c -o ../build/bar.cmo bar.ml
ocamlc -c -o ../build/foo.cmo foo.ml
```
the last command fails unless you add `-I ../build`. This on the other hand works without needing any playing around with `-I`:
```bash
cd ../build
ocamlc -c -o bar.cmo ../source/bar.ml
ocamlc -c -o foo.cmo ../source/foo.ml
```
except that `Bar` and `Foo` have different interface checksums. I am
mildly interested in a flag which says "ignore the dirname of the
source files" and which would then make `ocamlopt -c` behave similarly
to `gcc -c`.
I think in the context of an out-of-source build one would be in the
second scenario rather than in the first one, i.e. the current directory
would be somewhere in the build rather than source tree.
Couldn't we have interface files that are similer, no matter where they
are compiled from?
|
David Allsopp (2020/08/20 08:29 -0700):
The limitation which _could_ be overcome with a new flag is that when
you invoke the compiler, you must be in a consistent directory
relative to your source file: `ocamlc -c foo/bar/baz.mli` and `cd foo
; ocamlc -o bar/baz.cmi bar/baz.mli` generate a different interface.
Well, the only difference seems to be the locations that are stored in
the `.cmi` files.
I just tried to compile one interface file from two different
directories. By default the resulting `.cmi` files are different, but
with the `-no-keep-locs` command-line option they become identical.
(thanks a lot to @trefis for having suggested this.)
I am assuming that if it is only a matter of locations it's an
acceptable difference and thus not such a big issue, right?
The benefit to being able to compile your object files in the current directory is that you don't need to be including the output directory with `-I` in order to pick up any _other_ modules already compiled. Consider:
```bash
mkdir source build
echo 'let answer = 42' > source/bar.ml
echo 'let answer = Bar.answer' > source/foo.ml
cd source
ocamlc -c -o ../build/bar.cmo bar.ml
ocamlc -c -o ../build/foo.cmo foo.ml
```
the last command fails unless you add `-I ../build`. This on the other hand works without needing any playing around with `-I`:
```bash
cd ../build
ocamlc -c -o bar.cmo ../source/bar.ml
ocamlc -c -o foo.cmo ../source/foo.ml
```
except that `Bar` and `Foo` have different interface checksums.
But again, just because of the locations. With -no-keep-locs the
checksums are the same.
I am mildly interested in a flag which says "ignore the dirname of the
source files" and which would then make `ocamlopt -c` behave similarly
to `gcc -c`.
That flag exists but I wonder whether it would be okay to use it to
enable out-of-source builds of, say, the standard library. We could of
course _not_ use the flag, but then, assuming the current working
directory is in the build tree, the locations will likely not be
reusable later. After all, isn't it a flag that specifies which dirname
to use for the locations that would be useful, actually?
|
This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc. |
Still needs work.
|
Would it be possible to document this better ? I read through documentation for |
Prateek Khandelwal (2022/01/05 19:31 -0800):
Would it be possible to document this better ? I read through
documentation for `ocamlc` multiple times skipping over the `-o` part
just because it says `-o exec-file`, and I just assumed that this
option is valid when creating executables only. I do see that there's
a line about it's use with `-c` flag at the end, however it's easy to
miss it.
See proposal in #11015.
|
Original bug ID: 3757
Reporter: administrator
Assigned to: @damiendoligez
Status: closed (set by @mshinwell on 2016-12-07T15:55:25Z)
Resolution: fixed
Priority: normal
Severity: feature
Category: ~DO NOT USE (was: OCaml general)
Related to: #3904 #6191
Monitored by: hw @hcarty
Bug description
Full_Name: Rares Ispas
Version: 3.0.6
OS: Windows 2003
Submission from: 193.226.167.125 (193.226.167.125)
Please provide a way (ie command line option) to set the compiler output
directory fo r compiled files. We like to keep our source and the generated
binaries separate, for source control reasons
The text was updated successfully, but these errors were encountered: