Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005127OCamlCamlp4public2010-08-15 21:592014-08-21 11:24
ReporterHendrik Tews 
Assigned Todim 
PrioritynormalSeverityminorReproducibilityalways
StatusassignedResolutionopen 
PlatformOSOS Version
Product Version3.12.0 
Target Version4.02.1+devFixed in Version 
Summary0005127: wrong location after directive
DescriptionThe locations in the Camlp4 ast are completely wrong for the
material that follows a directive. For a two line file

#use "y.ml";;
let a = 5

the location in the StVal node for line 2 has

start_line = 3 (should be 2)
start_bol = 1 (should be 14)
start_off = 1 (should be 14)

the stop location is also wrong.
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
(0006512)
dim (developer)
2011-12-22 21:16

Unfortunately it is going to be very difficult to fix. The only simple fix i have in mind is going to degrade performances a lot.

By the way note that locations are correct after directives when you use the toplevel. And since directives are for the toplevel, I don't think it is a big problem. So you should avoid using them in programs. As a replacement you can use camlp4.macro:

  INCLUDE "x.ml"

With this syntax there is no problem.
(0006654)
Hendrik Tews (reporter)
2012-01-12 09:04

May I suggest to add a note to the documentation?
(0006661)
dim (developer)
2012-01-12 11:38

Sure. But i don't know where it should go...
(0007445)
Hendrik Tews (reporter)
2012-05-21 22:00

I would suggest to put it as ocamldoc comment into module Loc in file camlp4/Camlp4/Sig.ml. That's at least where I look when I want to know something about camlp4 locations.
(0007446)
Hendrik Tews (reporter)
2012-05-21 23:30

After looking again at the problem I noticed that we might have a
misunderstanding: The problem is present with all directives (eg,
also with #load), even if I am using a dummy directive handler
for parsing, such as

  let directive_handler x = Some x

The problem is obviously that

  value stopped_at _loc =
    Some (Loc.move_line 1 _loc) (* FIXME be more precise *);

in camlp4/Camlp4Parsers/Camlp4OCamlRevisedParser.ml does not move
the offsets.
(0007510)
hongboz (developer)
2012-06-04 04:57

I was bitten by this bug too.
there's a very useful directive, #default_quotation "blabla", which has parsing time effects, and
help me to remove a lot of duplicated code, <:expr< >> and <:patt< >> can share the same structure. The problem came the documentation generated by ocamldoc is totally wrong.
It would be very nice to fix this bug.
(0007515)
hongboz (developer)
2012-06-04 16:16

I defined my own add hoc syntax extension to work around this problem. I used syntax extension only for ad hoc usage.
EXTEND Gram GLOBAL: str_item ;
str_item:
  [[ "<^"; quot=STRING; "^>"-> begin
    Quotation.default.val:= quot;
    <:str_item< >>
  end
   ]];
END;
(0007522)
dim (developer)
2012-06-07 11:52

> After looking again at the problem I noticed that we might have a
> misunderstanding: The problem is present with all directives (eg,
> also with #load), even if I am using a dummy directive handler
> for parsing, such as

That's what i understood.

> The problem is obviously that
>
> value stopped_at _loc =
> Some (Loc.move_line 1 _loc) (* FIXME be more precise *);
>
> in camlp4/Camlp4Parsers/Camlp4OCamlRevisedParser.ml does not move
> the offsets.

It is a more complicated problem; the parser used by camlp4 for an implementation/interface file has type:

  Loc.t -> char Stream.t -> Ast.(str_item/sig_item) * Loc.t option

Where Loc.t option is the the location after the current directive if any.

This parser internally creates a lexer, token stream, ... for this character stream, and when it stops after a directive, pending characters in the lexer or pending tokens in the token streams or any other stuff in internal data structures are just lost, so there is no way to tell what will be the next location.

Here is for example another bug with directives and camlp4:

$ cat > foo.ml
#use "bar.ml"
let x = 1
$ touch bar.ml
$ camlp4o foo.ml
x = 1

The "let" keyword has been dropped.

One way to fix this would be to change the API to use parsers that takes token streams but it will break programs.

If you really want to use directives you can have a look at the optcomp project, it handles them at the token level (before the parser) so it does not have this problem.
(0007967)
hongboz (developer)
2012-08-22 12:33

@dim, the bug you mentioned will makes ocamldoc generate a messy documentation?
(0007972)
dim (developer)
2012-08-22 13:21

Not if you don't use directives.
(0008544)
hongboz (developer)
2012-11-28 21:38

FYI, this problem disappeared in my branch after I changed stopped_at, just pass the same position

- Issue History
Date Modified Username Field Change
2010-08-15 21:59 Hendrik Tews New Issue
2010-08-15 21:59 Hendrik Tews Status new => assigned
2010-08-15 21:59 Hendrik Tews Assigned To => ertai
2011-10-26 08:39 ertai Assigned To ertai => xclerc
2011-12-22 21:16 dim Note Added: 0006512
2011-12-22 21:17 dim Assigned To xclerc => dim
2011-12-22 21:17 dim Status assigned => feedback
2012-01-11 22:48 Hendrik Tews Note Added: 0006648
2012-01-11 22:48 Hendrik Tews Status feedback => assigned
2012-01-12 09:03 Hendrik Tews Note Deleted: 0006648
2012-01-12 09:04 Hendrik Tews Note Added: 0006654
2012-01-12 11:38 dim Note Added: 0006661
2012-05-21 22:00 Hendrik Tews Note Added: 0007445
2012-05-21 23:30 Hendrik Tews Note Added: 0007446
2012-06-04 04:57 hongboz Note Added: 0007510
2012-06-04 16:16 hongboz Note Added: 0007515
2012-06-07 11:52 dim Note Added: 0007522
2012-07-10 19:58 doligez Target Version => 4.01.0+dev
2012-07-31 13:36 doligez Target Version 4.01.0+dev => 4.00.1+dev
2012-08-22 12:33 hongboz Note Added: 0007967
2012-08-22 13:21 dim Note Added: 0007972
2012-09-19 13:55 doligez Target Version 4.00.1+dev => 4.01.0+dev
2012-11-28 21:38 hongboz Note Added: 0008544
2013-08-19 11:59 doligez Target Version 4.01.0+dev => 4.01.1+dev
2014-05-25 20:20 doligez Target Version 4.01.1+dev => 4.02.0+dev
2014-08-21 11:24 doligez Target Version 4.02.0+dev => 4.02.1+dev


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker