Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

patch to pack functors #5283

Closed
vicuna opened this issue Jun 6, 2011 · 12 comments
Closed

patch to pack functors #5283

vicuna opened this issue Jun 6, 2011 · 12 comments

Comments

@vicuna
Copy link

vicuna commented Jun 6, 2011

Original bug ID: 5283
Reporter: @lefessan
Assigned to: @xavierleroy
Status: resolved (set by @xavierleroy on 2016-12-07T11:04:09Z)
Resolution: won't fix
Priority: normal
Severity: feature
Platform: all
OS: all
OS Version: all
Version: 3.12.0
Category: ~DO NOT USE (was: OCaml general)
Tags: patch
Monitored by: abdallah @protz mehdi @glondu @chambart "Julien Signoles" @hcarty @alainfrisch

Bug description

This patch aims at building multi-units functors, i.e. a functor whose body is defined in several compilation units.

An example is provided in test-run.tar.gz.

Each compilation unit in the functor should be compiled with "-functor x.mli", where x.mli (or x.cmi) defines the interface of the functor argument (several -functor arguments can be provided).
To generate a functor, the option "-pack-functor FunctorName" should be provided. It will generate a module (as defined by "-o NameOfModule.cmo/.cmx"), containing the functor FunctorName, with one argument (corresponding to the -functor used for all its arguments), and all the provided modules as body.

As an independant feature, it allows to pack interfaces:
ocamlc -pack -o abc.cmi a.cmi b.cmi c.cmi
an option that was not implemented before, while still useful.

The patch is applied against 3.12.0, but should work with minimal changes against next versions.

Steps to reproduce

Example of usage:

    $(OCAMLC) -c arguments/x.mli
    $(OCAMLC) -c -functor arguments/x.mli arguments/y.mli
    $(OCAMLC) -c -functor arguments/y.mli -functor arguments/x.mli a.mli
    $(OCAMLC) -c -functor arguments/y.mli -for-pack Lib.Make_a -functor arguments/x.mli a.ml
    $(OCAMLC) -I arguments -functor arguments/x.mli -for-pack Lib -pack-functor Make -o make_a.$(CMO) a.$(CMO)
    $(OCAMLC) -c -functor arguments/x.mli -for-pack Lib b.ml
    $(OCAMLC) -I arguments -pack-functor Make -o lib.$(CMO) make_a.$(CMO) b.$(CMO)

File attachments

@vicuna
Copy link
Author

vicuna commented Jun 6, 2011

Comment author: @lefessan

I should add it was a request of Jane Street, and it builds on Alain's idea to extend the pack mechanism for functors, from this caml-list thread:

http://groups.google.com/group/fa.caml/browse_thread/thread/86d82441e7ce7e9a/246f18fe94ee8b06?hl=en&ie=UTF-8&q=yminsky+functorize+multiple+files&pli=1

@vicuna
Copy link
Author

vicuna commented Jun 6, 2011

Comment author: @zoggy

I'm afraid that programs will become harder to understandif part of the code moves to the Makefile.
And it will become a nightmare to develop tools (I'm thinking about ocamldoc here, for example).

@vicuna
Copy link
Author

vicuna commented Jun 13, 2011

Comment author: yminsky

Fabrice mentioned that Xavier was interested in seeing some examples. We certainly have them. Sadly, the description is a little abstract, but here goes:

One of our core applications is a big framework organized around a central functor where the business logic is contained. In that core application, we have a somewhat awkward mix of polymorphism and functor application, to avoid the need to functorize the entire library.

That core application worked out OK because the polymorphism that needed to be extended out was fairly small. But then we built a user-interface system for connecting to instances of this core application. The polymorphism in the UI was out of control, so that we ended up with types that had 8 free type variables. It was horribly confusing, and the whole thing would have gone away if we could have functorized the whole thing around a single module.

In the end, we used first-class packaged modules to dynamically choose the module in question based on an environment variable. That got rid of the confusing type parameters. But it's an ugly solution, and it also required us to restructure our code considerably to make this approach work.

Basically, we had weeks of rewriting of an application to deal with the lack of this feature, and we ended up with an uglier solution than we would have liked.

Sadly, I can point to no examples of this in our publically released code. That said, this thread from 9 years ago on the caml list came up in my work on the SKS keyserver, and hits on very much the same issue:

http://caml.inria.fr/pub/ml-archives/caml-list/2002/03/67cc0f6f75a9bd7ced9f03f724a92bdf.en.html

Here's the source for that:

http://code.google.com/p/sks-keyserver/

@vicuna
Copy link
Author

vicuna commented Jan 5, 2012

Comment author: @chambart

This can also be usefull for ocsigen.

In ocsigen a site is registered by calling the service creation functions in the right context. This is achieved by dynlinking the cma file of the site at the right time during the server initialisation.

The problem is that it is difficult to do static linking of the whole server with the site: There is no right order of evaluation of the different modules, the initialisation must occur in the middle of a function.

It is possible to circumvent that by doing the whole site initialisation inside a single function, but that adds the restriction that every services must be declared in the same file.

With this functor pack option, we could transform every site into a functor, and registration whould be applying the functor to its context.

@vicuna
Copy link
Author

vicuna commented Jan 23, 2012

Comment author: @xavierleroy

This proposal was discussed among the dev team, so let me record the pros and cons.

  • On the positive side: this is a natural extension of the "-pack" mechanism that can indeed be useful to functorize code a posteriori without the burden of functorizing by hand each compilation unit.

  • On the negative side: several important users of OCaml (e.g. Jane Street), as well as many of the dev team members, are unhappy about "-pack" (inefficient linking, lack of tool support, etc) and have been pushing for other approaches to namespace management that would subsume "-pack". Several namespace designs were floated on the caml-devel and caml-consortium mailing lists. My big concern here is that if we add packed functors (this patch) on top of packed modules (the -pack option), we get something that is very very hard to subsume via a namespace mechanism. In turn, this would, in my opinion, delay or even prohibit the integration of a different namespace mechanism.

So, here is my question to the supporters of this proposal: if you have to choose between packed functors (this proposal) or a future namespace mechanism, which one do you choose, knowing that you won't get both? Which one is more important to you?

To finish, Alain Frisch also remarked that for small to medium-sized collections of compilation units, one can always use "cat" and a bit of shell scripting to produce the packed functor as an OCaml source file, then compile it as usual.

@vicuna
Copy link
Author

vicuna commented Jan 23, 2012

Comment author: @alainfrisch

We don't have a real need either for the current -pack or for a new namespace mechanism. Being able to split the implementation of a functor's body over several files would be useful to us.

Another approach to this problem could be an ad hoc (independent of -pack) solution where the compiler is instructed to compile one functor by taking the source code in several files; this would be cleaner than the "cat" solution.

@vicuna
Copy link
Author

vicuna commented Jan 23, 2012

Comment author: @hcarty

I would much rather have a namespace mechanism.

@vicuna
Copy link
Author

vicuna commented Jan 23, 2012

Comment author: @glondu

Generating big files with a mechanism external to the compiler looks fine to me. A namespace mechanism looks more useful.

@vicuna
Copy link
Author

vicuna commented Jan 24, 2012

Comment author: yminsky

A few thoughts:

A cat based solution seems decidedly second class. Among things that won't work well:

a) resolution of error messages in the tools to the proper files. Maybe there's some way to make this work nicely, but cat isn't it.
b) It also plays poorly with other parts of the building process. What if you want to run different syntax extensions on different parts of the code thus combined?
c) We have some non-trivial codebases that we'd want to be able to use this with. Since this kills separate compilation, such codebases would become quite inconvenient to build.

All of which is to say: the "cat" implementation seems pretty painful. Maybe there's some intermediate solution that doesn't use -pack, as Alain suggests.

I agree that namespaces are more important, though, since some workaround for library-wide functors that addresses at least point (a) above is probably possible.

@vicuna
Copy link
Author

vicuna commented Jan 25, 2012

Comment author: @alainfrisch

Addressing a) is easy with a shell script (or within omake directly); it's just about adding a #line-directive before each included file.

For b), I've no real solution. Do you really use incompatible syntax extensions for various modules intended to be part of the same functor's body?

For c), are you concerned with the complexity of the build system or with slower builds?

@vicuna
Copy link
Author

vicuna commented Nov 30, 2012

Comment author: @bobzhang

For b) it can be solved in the future, each file can describe which syntax it used, self-explainable, and we could provide a include which respects this

@vicuna
Copy link
Author

vicuna commented Dec 7, 2016

Comment author: @xavierleroy

This PR has been sleeping for too long. I am sympathetic to the need expressed in this feature wish. If @lefessan still feels like supporting it, I suggest turning this into a Github pull request so that we can have it discussed and reviewed better than what's been done here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants