Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-ppx could be made working in toplevel #5904

Closed
vicuna opened this issue Jan 23, 2013 · 8 comments
Closed

-ppx could be made working in toplevel #5904

vicuna opened this issue Jan 23, 2013 · 8 comments
Assignees

Comments

@vicuna
Copy link

vicuna commented Jan 23, 2013

Original bug ID: 5904
Reporter: meyer
Assigned to: @alainfrisch
Status: closed (set by @xavierleroy on 2016-12-07T10:36:48Z)
Resolution: fixed
Priority: low
Severity: feature
Fixed in version: 4.02.1+dev
Category: ~DO NOT USE (was: OCaml general)
Monitored by: @hcarty

Bug description

I'd like to test my -ppx code in toplevel. Currently -ppx works just offline, using the file system.

This is a feature wish, and probably requiring some effort, nevertheless I think it's worthwhile.

@vicuna
Copy link
Author

vicuna commented Jan 25, 2013

Comment author: @alainfrisch

Commit 13278 (trunk) adds support for -ppx to the toplevel. Each phrase (not directives) is passed to external rewriters. Not that when the toplevel is used to execute a script (passed on its command-line, or with #use), then each phrase is rewritten independently. It might be worth sending the whole script to the rewriter instead. It would also allow the rewriter to rewrite directives.

Actually, it would make sense to allow the rewriters to maintain their state across phrases. This could be done by letting the rewriters send (and read back) some extra custom data in addition to the ASTs; this requires the rewriter to maintain a clean notion of state, which can be easily serialized. Otherwise, we could keep the rewriters alive as external processes and talk with them for each phrase. But this requires more complex process control logic (from module Unix).

@vicuna
Copy link
Author

vicuna commented Jan 27, 2013

Comment author: meyer

That sounds awesome, thanks.

@vicuna
Copy link
Author

vicuna commented Jul 28, 2013

Comment author: @gasche

Is this issue resolved?

@vicuna
Copy link
Author

vicuna commented Jul 29, 2013

Comment author: @alainfrisch

Support for -ppx has been added to the toplevel, but it can be improved (see my note : "Actually, it would make sense to allow the rewriters to maintain their state across phrases...").

@vicuna
Copy link
Author

vicuna commented May 13, 2014

Comment author: @alainfrisch

A #ppx directive has been added, to add dynamically extra ppx rewriters during a toplevel session.

@vicuna
Copy link
Author

vicuna commented Sep 23, 2014

Comment author: @alainfrisch

Commit 15314 adds a notion of "persistent cookies", which can be set by ppx processors and retrieved on further invocations when called from the toplevel.
Cookies are identified by an arbitrary string and hold an arbitrary Parsetree expression.

It is the responsibility of each ppx processor to store a cookie that represent its state and then restore it (using Ast_mapper.get_cookie/set_cookie). Note that cookies are available when the function registered through Ast_mapper.register is executed, so it is possible to set some global variables there.

@vicuna
Copy link
Author

vicuna commented Oct 12, 2014

Comment author: @gasche

Sorry for reacting a bit late, but the current API for ppx
"environment" exposed in ast_mapper seems rather strange to me.

(** {2 Helper functions to call external mappers} *)

val add_ppx_context_str: tool_name:string -> Parsetree.structure -> Parsetree.structure
(** Extract information from the current environment and encode it
into an attribute which is prepended to the list of structure
items in order to pass the information to an external
processor. *)

val add_ppx_context_sig: tool_name:string -> Parsetree.signature -> Parsetree.signature
(** Same as [ad_ppx_context_str] for signatures. *)

val drop_ppx_context_str: restore:bool -> Parsetree.structure -> Parsetree.structure
(** Drop the ocaml.ppx.context attribute from a structure. If
[restore] is true, also restore the associated data in the current
process. *)

val drop_ppx_context_sig: restore:bool -> Parsetree.signature -> Parsetree.signature
(** Same as [drop_ppx_context_str] for signatures. *)

(** {2 Cookies} *)

(** Cookies are used to pass information from a ppx processor to
a further invocation of itself, when called from the OCaml
toplevel (or other tools that support cookies). *)

val set_cookie: string -> Parsetree.expression -> unit
val get_cookie: string -> Parsetree.expression option

I find this API quite indirect.

Here is what I understand it does:

  1. I can query and update some "information from the current environment"
  2. I can store and change the way this information is serialized as part of an AST

Aspects (1) and (2) are rather orthogonal, but the API only provide
direct actions for (2). If I want to change the current environment to
match a serialized one, I should do that by setting a an
optional ?drop parameter to true.

I would rather suggest an interface along the following lines:

  • an explicit type for "ppx contexts" (this is what is implemented
    under the hood) along a description of what it actually stores

  • functions to get and update that part of the currently-running
    compiler state

  • functions to serialize and deserialize that state as an attribute
    (in a standard way)

  • optionally, convenience functions to read and update this attribute
    from structure and signature items, as exposed here

Similarly, I would be interested in an immutable type for cookies with
pure get/set operations; the current {get,set}_cookie would still be
present as acting on a compiler-internal global reference.

As a concrete example of use for such immutable data structures,
Merlin has reimplemented the compiler Clflags module on top of
a pervasive record of booleans (src/ocaml_aux/clflags.mli) -- this is
used to handle having multiple Emacs/vim buffers with distinct
configurations talking to the same merlin daemon. The compiler exposes
overly imperative for historical reasons, but we shouldn't repeat that
mistake in new APIs.

@vicuna
Copy link
Author

vicuna commented Oct 13, 2014

Comment author: @alainfrisch

Hi Gabriel,

The API for cookies (set/get_cookie) is intended to be used by ppx rewriters.

The context function are "less public", and currently mostly intended to be used by the compiler itself or closely related tools. But don't hesitate to send a patch with a better API!

@vicuna vicuna closed this as completed Dec 7, 2016
avsm pushed a commit to avsm/ocaml that referenced this issue Jun 21, 2020
avsm pushed a commit to avsm/ocaml that referenced this issue Jun 21, 2020
…herry-picked from trunk, commit 15414.)

git-svn-id: http://caml.inria.fr/svn/ocaml/version/4.02@15415 f963ae5c-01c2-4b8c-9fe0-0dff7051ff02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants