When a plugin is loaded in the toplevel, Token.Filter.define_filter has no effect before the first syntax error #5579
Comments
|
Comment author: @gasche Note: I have reproduced the bug under 3.11.2, 3.12.1 and trunk. |
|
Comment author: @diml The reason is that the camlp4 toplevel module reuse the same filtered token stream for the same lexing buffer in Toploop.parse_toplevel_phrase. I think it is safe to restart the parsing from scratch at each invocation. And it seems even wrong not to do it since Toploop discard everything after the first phrase. |
|
Comment author: @diml I made the change. Commits 12336 and 12337. |
|
Comment author: @gasche Thanks for the quick bugfix! Looking at the patch, I'm wondering why there was a stream caching technique in place. Does this change affect the performances much (I don't know where would toplevel parsing be performance-sensitive; maybe in HOL-light, which uses the OCaml toplevel as its workspace?)? If that appeared to be a problem, you could restore the caching strategy, using something along the lines of (untested) let get_and_cache_stream lb = |
|
Comment author: @diml
The difference in term of performances is very minimal (creation of a camlp4 lexer + creation of a stream), so nothing perceptible. I looked in the history: in the old camlp4 (and in the current camlp5) there was no caching. The new camlp4 used only one stream (stored in a ref), leading to #4495 and #4593, the fix was to cache streams this way. Also not caching token streams fixes this kind of bugs: 1^CInterrupted. 1;;Characters 0-1: |
Original bug ID: 5579
Reporter: @gasche
Assigned to: @diml
Status: closed (set by @xavierleroy on 2015-12-11T18:04:31Z)
Resolution: open
Priority: normal
Severity: minor
Version: 4.00.0+dev
Fixed in version: 4.00.0+dev
Category: -for Camlp4 use https://github.com/ocaml/camlp4/issues
Related to: #4811
Monitored by: @diml @hcarty
Bug description
As a reduced test case, I have written an example filter that just prints the token stream. The filter is not active at first, and the first erroneous phrase activates it. Observe:
$ ocaml dynlink.cma camlp4o.cma
Objective Caml version 3.12.1
#load "test_plugin.cmo";;
test case loaded
let () = ();;
let () = ;;
Error: Parse error: [expr] expected after "=" (in [cvalue_binding])
let () = ();;
KEYWORD "let"
BLANKS " "
KEYWORD "("
KEYWORD ")"
BLANKS " "
KEYWORD "="
BLANKS " "
KEYWORD "("
KEYWORD ")"
KEYWORD ";;"
This behavior does not happen when invoking directly with
ocamlc dynlink.cma camlp4o.cma test_plugin.cmo."test case loaded" is a debug statement that is executed when the plugin loads.
Steps to reproduce
(*
ocamlc -pp camlp4r -I +camlp4 -c test_plugin.ml
*)
open Camlp4;
module Id : Sig.Id = struct
value name = "filter bug test case";
value version = "0.1";
end;
module Make (Syntax : Sig.Camlp4Syntax) = struct
open Sig;
include Syntax;
value rec debug_filter = parser
[ [:
tok; rest :] -> let () = prerr_endline (Token.to_string (fst tok)) in [:tok; debug_filter rest :] ];value () = Token.Filter.define_filter (Gram.get_filter ())
(fun filter stream -> filter (debug_filter stream));
value () = prerr_endline "test case loaded";
end;
let module M = Register.OCamlSyntaxExtension(Id)(Make) in ();
The text was updated successfully, but these errors were encountered: