Comment author: @diml

Unfortunately it is going to be very difficult to fix. The only simple fix i have in mind is going to degrade performances a lot.

By the way note that locations are correct after directives when you use the toplevel. And since directives are for the toplevel, I don't think it is a big problem. So you should avoid using them in programs. As a replacement you can use camlp4.macro:

INCLUDE "x.ml"

With this syntax there is no problem.

Comment author: Hendrik Tews

May I suggest to add a note to the documentation?

Comment author: @diml

Sure. But i don't know where it should go...

Comment author: Hendrik Tews

I would suggest to put it as ocamldoc comment into module Loc in file camlp4/Camlp4/Sig.ml. That's at least where I look when I want to know something about camlp4 locations.

Comment author: Hendrik Tews

After looking again at the problem I noticed that we might have a
misunderstanding: The problem is present with all directives (eg,
also with #load), even if I am using a dummy directive handler
for parsing, such as

let directive_handler x = Some x

The problem is obviously that

value stopped_at _loc =
Some (Loc.move_line 1 _loc) (* FIXME be more precise *);

in camlp4/Camlp4Parsers/Camlp4OCamlRevisedParser.ml does not move
the offsets.

Comment author: @bobzhang

I was bitten by this bug too.
there's a very useful directive, #default_quotation "blabla", which has parsing time effects, and
help me to remove a lot of duplicated code, <:expr< >> and <:patt< >> can share the same structure. The problem came the documentation generated by ocamldoc is totally wrong.
It would be very nice to fix this bug.

Comment author: @bobzhang

I defined my own add hoc syntax extension to work around this problem. I used syntax extension only for ad hoc usage.
EXTEND Gram GLOBAL: str_item ;
str_item:
[[ "<^"; quot=STRING; "^>"-> begin
Quotation.default.val:= quot;
<:str_item< >>
end
]];
END;

Comment author: @diml

After looking again at the problem I noticed that we might have a
misunderstanding: The problem is present with all directives (eg,
also with #load), even if I am using a dummy directive handler
for parsing, such as

That's what i understood.

The problem is obviously that

value stopped_at _loc =
Some (Loc.move_line 1 _loc) (* FIXME be more precise *);

in camlp4/Camlp4Parsers/Camlp4OCamlRevisedParser.ml does not move
the offsets.

It is a more complicated problem; the parser used by camlp4 for an implementation/interface file has type:

Loc.t -> char Stream.t -> Ast.(str_item/sig_item) * Loc.t option

Where Loc.t option is the the location after the current directive if any.

This parser internally creates a lexer, token stream, ... for this character stream, and when it stops after a directive, pending characters in the lexer or pending tokens in the token streams or any other stuff in internal data structures are just lost, so there is no way to tell what will be the next location.

Here is for example another bug with directives and camlp4:

$ cat > foo.ml
#use "bar.ml"
let x = 1
$ touch bar.ml
$ camlp4o foo.ml
x = 1

The "let" keyword has been dropped.

One way to fix this would be to change the API to use parsers that takes token streams but it will break programs.

If you really want to use directives you can have a look at the optcomp project, it handles them at the token level (before the parser) so it does not have this problem.

Comment author: @bobzhang

@dim, the bug you mentioned will makes ocamldoc generate a messy documentation?

Comment author: @diml

Not if you don't use directives.

Comment author: @bobzhang

FYI, this problem disappeared in my branch after I changed stopped_at, just pass the same position

Comment author: @damiendoligez

Transferred to camlp4/camlp4#72