0005283: patch to pack functors

Notes
(0005984) lefessan (developer) 2011-06-06 18:24	I should add it was a request of Jane Street, and it builds on Alain's idea to extend the pack mechanism for functors, from this caml-list thread: http://groups.google.com/group/fa.caml/browse_thread/thread/86d82441e7ce7e9a/246f18fe94ee8b06?hl=en&ie=UTF-8&q=yminsky+functorize+multiple+files&pli=1 [^]

(0005985) guesdon (manager) 2011-06-06 18:31	I'm afraid that programs will become harder to understandif part of the code moves to the Makefile. And it will become a nightmare to develop tools (I'm thinking about ocamldoc here, for example).

(0006007) yminsky (reporter) 2011-06-14 00:09 edited on: 2011-06-14 00:12	Fabrice mentioned that Xavier was interested in seeing some examples. We certainly have them. Sadly, the description is a little abstract, but here goes: One of our core applications is a big framework organized around a central functor where the business logic is contained. In that core application, we have a somewhat awkward mix of polymorphism and functor application, to avoid the need to functorize the entire library. That core application worked out OK because the polymorphism that needed to be extended out was fairly small. But then we built a user-interface system for connecting to instances of this core application. The polymorphism in the UI was out of control, so that we ended up with types that had 8 free type variables. It was horribly confusing, and the whole thing would have gone away if we could have functorized the whole thing around a single module. In the end, we used first-class packaged modules to dynamically choose the module in question based on an environment variable. That got rid of the confusing type parameters. But it's an ugly solution, and it also required us to restructure our code considerably to make this approach work. Basically, we had weeks of rewriting of an application to deal with the lack of this feature, and we ended up with an uglier solution than we would have liked. Sadly, I can point to no examples of this in our publically released code. That said, this thread from 9 years ago on the caml list came up in my work on the SKS keyserver, and hits on very much the same issue: http://caml.inria.fr/pub/ml-archives/caml-list/2002/03/67cc0f6f75a9bd7ced9f03f724a92bdf.en.html [^] Here's the source for that: http://code.google.com/p/sks-keyserver/ [^]

(0006595) chambart (reporter) 2012-01-05 11:20	This can also be usefull for ocsigen. In ocsigen a site is registered by calling the service creation functions in the right context. This is achieved by dynlinking the cma file of the site at the right time during the server initialisation. The problem is that it is difficult to do static linking of the whole server with the site: There is no right order of evaluation of the different modules, the initialisation must occur in the middle of a function. It is possible to circumvent that by doing the whole site initialisation inside a single function, but that adds the restriction that every services must be declared in the same file. With this functor pack option, we could transform every site into a functor, and registration whould be applying the functor to its context.

(0006771) xleroy (administrator) 2012-01-23 12:01	This proposal was discussed among the dev team, so let me record the pros and cons. - On the positive side: this is a natural extension of the "-pack" mechanism that can indeed be useful to functorize code a posteriori without the burden of functorizing by hand each compilation unit. - On the negative side: several important users of OCaml (e.g. Jane Street), as well as many of the dev team members, are unhappy about "-pack" (inefficient linking, lack of tool support, etc) and have been pushing for other approaches to namespace management that would subsume "-pack". Several namespace designs were floated on the caml-devel and caml-consortium mailing lists. My big concern here is that if we add packed functors (this patch) on top of packed modules (the -pack option), we get something that is very very hard to subsume via a namespace mechanism. In turn, this would, in my opinion, delay or even prohibit the integration of a different namespace mechanism. So, here is my question to the supporters of this proposal: if you have to choose between packed functors (this proposal) or a future namespace mechanism, which one do you choose, knowing that you won't get both? Which one is more important to you? To finish, Alain Frisch also remarked that for small to medium-sized collections of compilation units, one can always use "cat" and a bit of shell scripting to produce the packed functor as an OCaml source file, then compile it as usual.

(0006772) frisch (developer) 2012-01-23 12:26	We don't have a real need either for the current -pack or for a new namespace mechanism. Being able to split the implementation of a functor's body over several files would be useful to us. Another approach to this problem could be an ad hoc (independent of -pack) solution where the compiler is instructed to compile one functor by taking the source code in several files; this would be cleaner than the "cat" solution.

(0006777) hcarty (reporter) 2012-01-23 14:57	I would much rather have a namespace mechanism.

(0006784) glondu (reporter) 2012-01-23 16:35	Generating big files with a mechanism external to the compiler looks fine to me. A namespace mechanism looks more useful.

(0006789) yminsky (reporter) 2012-01-24 02:45	A few thoughts: A cat based solution seems decidedly second class. Among things that won't work well: a) resolution of error messages in the tools to the proper files. Maybe there's some way to make this work nicely, but cat isn't it. b) It also plays poorly with other parts of the building process. What if you want to run different syntax extensions on different parts of the code thus combined? c) We have some non-trivial codebases that we'd want to be able to use this with. Since this kills separate compilation, such codebases would become quite inconvenient to build. All of which is to say: the "cat" implementation seems pretty painful. Maybe there's some intermediate solution that doesn't use -pack, as Alain suggests. I agree that namespaces are more important, though, since some workaround for library-wide functors that addresses at least point (a) above is probably possible.

(0006802) frisch (developer) 2012-01-25 20:20	Addressing a) is easy with a shell script (or within omake directly); it's just about adding a #line-directive before each included file. For b), I've no real solution. Do you really use incompatible syntax extensions for various modules intended to be part of the same functor's body? For c), are you concerned with the complexity of the build system or with slower builds?

(0008548) hongboz (developer) 2012-11-30 18:04	For b) it can be solved in the future, each file can describe which syntax it used, self-explainable, and we could provide a include which respects this

Issue History
Date Modified	Username	Field	Change
2011-06-06 18:19	lefessan	New Issue
2011-06-06 18:19	lefessan	Status	new => assigned
2011-06-06 18:19	lefessan	Assigned To	=> xleroy
2011-06-06 18:19	lefessan	File Added: lib-as-functor-3.12.0.2011-06-06.patch
2011-06-06 18:19	lefessan	File Added: test-run.tar.gz
2011-06-06 18:24	lefessan	Note Added: 0005984
2011-06-06 18:31	guesdon	Note Added: 0005985
2011-06-14 00:09	yminsky	Note Added: 0006007
2011-06-14 00:12	yminsky	Note Edited: 0006007
2012-01-05 11:20	chambart	Note Added: 0006595
2012-01-23 12:01	xleroy	Note Added: 0006771
2012-01-23 12:01	xleroy	Status	assigned => feedback
2012-01-23 12:26	frisch	Note Added: 0006772
2012-01-23 14:57	hcarty	Note Added: 0006777
2012-01-23 16:35	glondu	Note Added: 0006784
2012-01-24 02:45	yminsky	Note Added: 0006789
2012-01-25 20:20	frisch	Note Added: 0006802
2012-11-30 18:04	hongboz	Note Added: 0008548

Relationships