Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set compiler output directory #3757

Closed
vicuna opened this issue Aug 15, 2005 · 28 comments
Closed

Set compiler output directory #3757

vicuna opened this issue Aug 15, 2005 · 28 comments
Assignees

Comments

@vicuna
Copy link

vicuna commented Aug 15, 2005

Original bug ID: 3757
Reporter: administrator
Assigned to: @damiendoligez
Status: closed (set by @mshinwell on 2016-12-07T15:55:25Z)
Resolution: fixed
Priority: normal
Severity: feature
Category: ~DO NOT USE (was: OCaml general)
Related to: #3904 #6191
Monitored by: hw @hcarty

Bug description

Full_Name: Rares Ispas
Version: 3.0.6
OS: Windows 2003
Submission from: 193.226.167.125 (193.226.167.125)

Please provide a way (ie command line option) to set the compiler output
directory fo r compiled files. We like to keep our source and the generated
binaries separate, for source control reasons

@vicuna
Copy link
Author

vicuna commented May 31, 2010

Comment author: hw

I see there's not been any activity on this ticket for 5 years. This feature would be useful to me for the same reasons. Having the source, intermediate and output files all mixed up in the same directory gets pretty unwieldy once you have a large set of input files, especially in my setup where version control and editing happens on one system and compilation on another. For my file browser to be a useful way of examining the structure of my project I first have to do a rm *.{annot,cmi,cmx,cmo,o,a,cmxa} just to see what source files I'm working with.

@vicuna
Copy link
Author

vicuna commented Dec 7, 2016

Comment author: @mshinwell

This can be done using the "-o" option.

@shindere
Copy link
Contributor

I am reopening this issue becuase, no, -o is not enough to solve the
problem.

Also, the issue is not at all Windows-specific and is still present in the
latest OCaml versions.

It seems there are two possible approaches to let users specify
an output directory for their build artifacts.

  1. An -odir DIR option as suggested in other issues, which would
    basically put build artifacts in directory DIR (or at least in a place
    relative to DIR).

  2. A -ocwd option which would put build artifacts in the current
    directory (or relative to it).

With the second option one would need to cd to the desired
directory before invoking the compiler. This may look cumbersome but
would have the benefit of playing well with -I because the current
directory is taken into account in the search path.

With the -odir DIR approach, one would have to either have DIR also
added to the search path for modules, or one would need to explicitly
write -odir DIR -I DIR. The first option here may look simpler
but the second gives more fine-grained control.

It seems @dra27 prefers -ocwd. What are other people's preferences?

Cc @Drup @thierry-martinez @rgrinberg

@nojb
Copy link
Contributor

nojb commented Aug 20, 2020

Personally am not sure of the relevance of this particular request (which in 15 years attracted a grand total of 1 comment, over 10 years ago).

I am not against adding a feature in this direction, but the stated motivations ("source control reasons") can all be fixed by using modern tools (build systems, editors); designing and agreeing on the semantics of a new flag just for this seems overkill.

@trefis
Copy link
Contributor

trefis commented Aug 20, 2020

Let's also ping @dbuenzli

@dra27
Copy link
Member

dra27 commented Aug 20, 2020

Indeed, the original problem here is solved with -o and ensuring that you link only compiled object files (ocamlc -o foo/bar foo.ml will put foo.cmo and foo.cmi in the current directory; all other uses of -o with -c should put all artefacts in the directory specified with -o)

@thierry-martinez
Copy link
Contributor

2\. A `-ocwd` option which would put build artifacts in the current
    directory (or relative to it).

Could you elaborate what you mean by or relative to it? As far as I understand, this is the current behavior of the compiler: source and target filenames are interpreted relatively to the current directory. On the contrary, -ocwd would make sense if build artifacts were systematically put in the current directory, regardless of the relative path of the source.

I should admit I understand the point of view that the build logic should be handled as much as possible by the build system, keeping the compiler behavior simple. The only benefit I see for handling out-of-source build directly in the compiler is to get correct paths in .cmt files when compiling with absname=1. Note that having paths on the copies would not be problematic if dune made use of symbolic links instead of duplicating .ml files in the _build/ directory.

@shindere
Copy link
Contributor

shindere commented Aug 20, 2020 via email

@shindere
Copy link
Contributor

shindere commented Aug 20, 2020 via email

@shindere
Copy link
Contributor

shindere commented Aug 20, 2020 via email

@nojb
Copy link
Contributor

nojb commented Aug 20, 2020

Several build systems do out-of-source-tree builds, which I guess is enough to show that the feature is desired. Then, in my opinion, the fact that each build system has to re-implement the feature shows that build systems are not the right place to address it. Or, to be more precise, that one could avoid a certain number of hacks and simplify the implementation if the compiler itself would have a better support for that.

Apologies for being stubborn, but I still don't understand what it is that these build systems reimplement.

For example, as far as I know, dune does out-of-tree builds just by using -o to put the compilation artifacts where it wants them to be. Incidentally, dune also needs to use -o in order to implement "prefixed" libraries and executables, so it is not clear to me it would be able to take advantage of the hypothetical -odir/-ocwd flag.

Perhaps it would be clearer if you could explain how build systems are supposed to use the new flag. Thanks!

@thierry-martinez
Copy link
Contributor

For example, as far as I know, dune does out-of-tree builds just by using -o to put the compilation artifacts where it wants them to be.

As far as I know, dune first copies all the source files in _build before running the compiler on these copies.

@nojb
Copy link
Contributor

nojb commented Aug 20, 2020

As far as I know, dune first copies all the source files in _build before running the compiler on these copies.

Yes, but this is not directly related to doing out-of-tree builds, it could use the files directly from the source tree, as far as I understand.

@shindere
Copy link
Contributor

shindere commented Aug 20, 2020 via email

@shindere
Copy link
Contributor

shindere commented Aug 20, 2020 via email

@thierry-martinez
Copy link
Contributor

thierry-martinez commented Aug 20, 2020

Yes, but this is not directly related to doing out-of-tree builds, it could use the files directly from the source tree, as far as I understand.

@nojb, you are right: ocamlc -c -o $BUILDDIR/x.cmo $SRCDIR/x.ml appears to produce all build artifacts in $BUILDDIR, I did not know that! @shindere , would -odir $BUILDDIR make something else than that?

@nojb
Copy link
Contributor

nojb commented Aug 20, 2020

As far as I know, dune first copies all the source files in _build before running the compiler on these copies.

Yes, but this is not directly related to doing out-of-tree builds, it could use the files directly from the source tree (as far as I understand).

Are we sure? Why does it copy them, then?

One key reason for copying files around is to implement sandboxing: to make sure that only the dependencies stated in the dune files are made visible to the compiler.

Note that even while dune copies files from the source tree to the build tree, it continues to use -o to put artifacts in separate directories in order to implement sandboxing: for example, if you have the source of two libraries in the same directory, dune still needs to put the .cmis in separate directories in order to make it possible to make one library or the other visible to the compiler, according to what is written in the dune files.

@dbuenzli
Copy link
Contributor

On the top of my head I think -o is mostly fine nowadays to do out of source tree builds and be able to mix your source arguments from wherever they live without having to copy them. There are however two remaining outstanding issues as far as I'm concerned:

  1. You can't use -o for compiling C files. You need to cwd to the output directory. This is Clarify how -c and -o are supposed to work. #7677.
  2. You can't compile an OCaml implementation against an OCaml interface that is not in the same directory, because of an absurd (because semantically meaningless) file check. This is Allow to compile an implementation against an interface that does not reside in the same directory #9717.

@dra27
Copy link
Member

dra27 commented Aug 20, 2020

Am I correct that you are finally suggesting to not add any new flag,
@dra27, and to just be disciplined in how one uses the compiler?

Are you saying this would be enough?

I'm saying that it is possible to solve the problem here with the current implementation of -o combined with the discipline of not compiling and linking at the same time. I don't think this was possible when the original issue was opened.

The limitation which could be overcome with a new flag is that when you invoke the compiler, you must be in a consistent directory relative to your source file: ocamlc -c foo/bar/baz.mli and cd foo ; ocamlc -o bar/baz.cmi bar/baz.mli generate a different interface. The benefit to being able to compile your object files in the current directory is that you don't need to be including the output directory with -I in order to pick up any other modules already compiled. Consider:

mkdir source build
echo 'let answer = 42' > source/bar.ml
echo 'let answer = Bar.answer' > source/foo.ml

cd source
ocamlc -c -o ../build/bar.cmo bar.ml
ocamlc -c -o ../build/foo.cmo foo.ml

the last command fails unless you add -I ../build. This on the other hand works without needing any playing around with -I:

cd ../build
ocamlc -c -o bar.cmo ../source/bar.ml
ocamlc -c -o foo.cmo ../source/foo.ml

except that Bar and Foo have different interface checksums. I am mildly interested in a flag which says "ignore the dirname of the source files" and which would then make ocamlopt -c behave similarly to gcc -c.

@shindere
Copy link
Contributor

shindere commented Aug 21, 2020 via email

@shindere
Copy link
Contributor

shindere commented Aug 21, 2020 via email

@shindere
Copy link
Contributor

shindere commented Aug 25, 2020 via email

@shindere
Copy link
Contributor

shindere commented Aug 25, 2020 via email

@shindere
Copy link
Contributor

shindere commented Aug 26, 2020 via email

@github-actions
Copy link

This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc.

@github-actions github-actions bot added the Stale label Aug 30, 2021
@shindere
Copy link
Contributor

shindere commented Aug 30, 2021 via email

@github-actions github-actions bot removed the Stale label Sep 1, 2021
@prateek-khandelwal
Copy link

Indeed, the original problem here is solved with -o and ensuring that you link only compiled object files (ocamlc -o foo/bar foo.ml will put foo.cmo and foo.cmi in the current directory; all other uses of -o with -c should put all artefacts in the directory specified with -o)

Would it be possible to document this better ? I read through documentation for ocamlc multiple times skipping over the -o part just because it says -o exec-file, and I just assumed that this option is valid when creating executables only. I do see that there's a line about it's use with -c flag at the end, however it's easy to miss it.

@shindere
Copy link
Contributor

shindere commented Feb 15, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

9 participants