Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005904OCaml~DO NOT USE (was: OCaml general)public2013-01-23 23:152016-12-07 11:36
Assigned Tofrisch 
PrioritylowSeverityfeatureReproducibilityhave not tried
PlatformOSOS Version
Product Version 
Target VersionFixed in Version4.02.1+dev 
Summary0005904: -ppx could be made working in toplevel
DescriptionI'd like to test my -ppx code in toplevel. Currently -ppx works just offline, using the file system.

This is a feature wish, and probably requiring some effort, nevertheless I think it's worthwhile.
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
frisch (developer)
2013-01-25 10:15
edited on: 2013-01-25 10:22

Commit 13278 (trunk) adds support for -ppx to the toplevel. Each phrase (not directives) is passed to external rewriters. Not that when the toplevel is used to execute a script (passed on its command-line, or with #use), then each phrase is rewritten independently. It might be worth sending the whole script to the rewriter instead. It would also allow the rewriter to rewrite directives.

Actually, it would make sense to allow the rewriters to maintain their state across phrases. This could be done by letting the rewriters send (and read back) some extra custom data in addition to the ASTs; this requires the rewriter to maintain a clean notion of state, which can be easily serialized. Otherwise, we could keep the rewriters alive as external processes and talk with them for each phrase. But this requires more complex process control logic (from module Unix).

meyer (developer)
2013-01-27 23:24
edited on: 2013-01-27 23:24

That sounds awesome, thanks.

gasche (administrator)
2013-07-28 23:36

Is this issue resolved?
frisch (developer)
2013-07-29 10:36

Support for -ppx has been added to the toplevel, but it can be improved (see my note : "Actually, it would make sense to allow the rewriters to maintain their state across phrases...").
frisch (developer)
2014-05-13 16:46

A #ppx directive has been added, to add dynamically extra ppx rewriters during a toplevel session.
frisch (developer)
2014-09-23 16:57

Commit 15314 adds a notion of "persistent cookies", which can be set by ppx processors and retrieved on further invocations when called from the toplevel.
Cookies are identified by an arbitrary string and hold an arbitrary Parsetree expression.

It is the responsibility of each ppx processor to store a cookie that represent its state and then restore it (using Ast_mapper.get_cookie/set_cookie). Note that cookies are available when the function registered through Ast_mapper.register is executed, so it is possible to set some global variables there.
gasche (administrator)
2014-10-12 18:42

Sorry for reacting a bit late, but the current API for ppx
"environment" exposed in ast_mapper seems rather strange to me.

  (** {2 Helper functions to call external mappers} *)
  val add_ppx_context_str: tool_name:string -> Parsetree.structure -> Parsetree.structure
  (** Extract information from the current environment and encode it
      into an attribute which is prepended to the list of structure
      items in order to pass the information to an external
      processor. *)
  val add_ppx_context_sig: tool_name:string -> Parsetree.signature -> Parsetree.signature
  (** Same as [ad_ppx_context_str] for signatures. *)
  val drop_ppx_context_str: restore:bool -> Parsetree.structure -> Parsetree.structure
  (** Drop the ocaml.ppx.context attribute from a structure. If
      [restore] is true, also restore the associated data in the current
      process. *)
  val drop_ppx_context_sig: restore:bool -> Parsetree.signature -> Parsetree.signature
  (** Same as [drop_ppx_context_str] for signatures. *)
  (** {2 Cookies} *)
  (** Cookies are used to pass information from a ppx processor to
      a further invocation of itself, when called from the OCaml
      toplevel (or other tools that support cookies). *)
  val set_cookie: string -> Parsetree.expression -> unit
  val get_cookie: string -> Parsetree.expression option

I find this API quite indirect.

Here is what I understand it does:
1) I can query and update some "information from the current environment"
2) I can store and change the way this information is serialized as part of an AST

Aspects (1) and (2) are rather orthogonal, but the API only provide
direct actions for (2). If I want to change the current environment to
match a serialized one, I should do that by setting a an
optional ?drop parameter to true.

I would rather suggest an interface along the following lines:

- an explicit type for "ppx contexts" (this is what is implemented
  under the hood) along a description of what it actually stores

- functions to get and update that part of the currently-running
  compiler state

- functions to serialize and deserialize that state as an attribute
  (in a standard way)

- optionally, convenience functions to read and update this attribute
  from structure and signature items, as exposed here

Similarly, I would be interested in an immutable type for cookies with
pure get/set operations; the current {get,set}_cookie would still be
present as acting on a compiler-internal global reference.

As a concrete example of use for such immutable data structures,
Merlin has reimplemented the compiler Clflags module on top of
a pervasive record of booleans (src/ocaml_aux/clflags.mli) -- this is
used to handle having multiple Emacs/vim buffers with distinct
configurations talking to the same merlin daemon. The compiler exposes
overly imperative for historical reasons, but we shouldn't repeat that
mistake in new APIs.
frisch (developer)
2014-10-13 09:24

Hi Gabriel,

The API for cookies (set/get_cookie) is intended to be used by ppx rewriters.

The *_context_* function are "less public", and currently mostly intended to be used by the compiler itself or closely related tools. But don't hesitate to send a patch with a better API!

- Issue History
Date Modified Username Field Change
2013-01-23 23:15 meyer New Issue
2013-01-23 23:15 meyer Status new => assigned
2013-01-23 23:15 meyer Assigned To => frisch
2013-01-25 10:15 frisch Note Added: 0008797
2013-01-25 10:16 frisch Note Edited: 0008797 View Revisions
2013-01-25 10:22 frisch Note Edited: 0008797 View Revisions
2013-01-27 23:24 meyer Note Added: 0008802
2013-01-27 23:24 meyer Note Edited: 0008802 View Revisions
2013-07-28 23:36 gasche Note Added: 0009954
2013-07-29 10:36 frisch Note Added: 0009975
2014-05-13 16:46 frisch Note Added: 0011441
2014-09-23 16:57 frisch Note Added: 0012189
2014-10-02 18:11 frisch Status assigned => resolved
2014-10-02 18:11 frisch Fixed in Version => 4.02.1+dev
2014-10-02 18:11 frisch Resolution open => fixed
2014-10-12 18:42 gasche Note Added: 0012357
2014-10-13 09:24 frisch Note Added: 0012361
2016-12-07 11:36 xleroy Status resolved => closed
2017-02-23 16:36 doligez Category OCaml general => -OCaml general
2017-03-03 17:55 doligez Category -OCaml general => -(deprecated) general
2017-03-03 18:01 doligez Category -(deprecated) general => ~deprecated (was: OCaml general)
2017-03-06 17:04 doligez Category ~deprecated (was: OCaml general) => ~DO NOT USE (was: OCaml general)

Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker