New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When a plugin is loaded in the toplevel, Token.Filter.define_filter has no effect before the first syntax error #5579
Comments
Comment author: @gasche Note: I have reproduced the bug under 3.11.2, 3.12.1 and trunk. |
Comment author: @diml The reason is that the camlp4 toplevel module reuse the same filtered token stream for the same lexing buffer in Toploop.parse_toplevel_phrase. I think it is safe to restart the parsing from scratch at each invocation. And it seems even wrong not to do it since Toploop discard everything after the first phrase. |
Comment author: @diml I made the change. Commits 12336 and 12337. |
Comment author: @gasche Thanks for the quick bugfix! Looking at the patch, I'm wondering why there was a stream caching technique in place. Does this change affect the performances much (I don't know where would toplevel parsing be performance-sensitive; maybe in HOL-light, which uses the OCaml toplevel as its workspace?)? If that appeared to be a problem, you could restore the caching strategy, using something along the lines of (untested) let get_and_cache_stream lb = |
Comment author: @diml
The difference in term of performances is very minimal (creation of a camlp4 lexer + creation of a stream), so nothing perceptible. I looked in the history: in the old camlp4 (and in the current camlp5) there was no caching. The new camlp4 used only one stream (stored in a ref), leading to #4495 and #4593, the fix was to cache streams this way. Also not caching token streams fixes this kind of bugs: 1^CInterrupted. 1;;Characters 0-1: |
Original bug ID: 5579
Reporter: @gasche
Assigned to: @diml
Status: closed (set by @xavierleroy on 2015-12-11T18:04:31Z)
Resolution: open
Priority: normal
Severity: minor
Version: 4.00.0+dev
Fixed in version: 4.00.0+dev
Category: -for Camlp4 use https://github.com/ocaml/camlp4/issues
Related to: #4811
Monitored by: @diml @hcarty
Bug description
As a reduced test case, I have written an example filter that just prints the token stream. The filter is not active at first, and the first erroneous phrase activates it. Observe:
$ ocaml dynlink.cma camlp4o.cma
Objective Caml version 3.12.1
#load "test_plugin.cmo";;
test case loaded
let () = ();;
let () = ;;
Error: Parse error: [expr] expected after "=" (in [cvalue_binding])
let () = ();;
KEYWORD "let"
BLANKS " "
KEYWORD "("
KEYWORD ")"
BLANKS " "
KEYWORD "="
BLANKS " "
KEYWORD "("
KEYWORD ")"
KEYWORD ";;"
This behavior does not happen when invoking directly with
ocamlc dynlink.cma camlp4o.cma test_plugin.cmo
."test case loaded" is a debug statement that is executed when the plugin loads.
Steps to reproduce
(*
ocamlc -pp camlp4r -I +camlp4 -c test_plugin.ml
*)
open Camlp4;
module Id : Sig.Id = struct
value name = "filter bug test case";
value version = "0.1";
end;
module Make (Syntax : Sig.Camlp4Syntax) = struct
open Sig;
include Syntax;
value rec debug_filter = parser
[ [:
tok; rest :] -> let () = prerr_endline (Token.to_string (fst tok)) in [:
tok; debug_filter rest :] ];value () = Token.Filter.define_filter (Gram.get_filter ())
(fun filter stream -> filter (debug_filter stream));
value () = prerr_endline "test case loaded";
end;
let module M = Register.OCamlSyntaxExtension(Id)(Make) in ();
The text was updated successfully, but these errors were encountered: