|Anonymous | Login | Signup for a new account||2014-07-25 20:20 CEST|
|Main | My View | View Issues | Change Log | Roadmap|
|View Issue Details|
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0006069||OCaml||OCamldoc||public||2013-07-09 06:27||2013-08-05 09:58|
|Priority||normal||Severity||minor||Reproducibility||have not tried|
|Target Version||4.01.0+dev||Fixed in Version|
|Summary||0006069: ocamldoc: lexing: empty token|
|Description||Just stumbled upon the case when ocamldoc aborts with unhelpful error message. Consider :|
$ cat q.ml
(** test *)
(** ok @@ *)
(** @deprecated @@ *)
(** ok [@@] *)
$ ocamldoc q.ml
/home/ygrek/work/arena/test/q.ml : lexing: empty token
1 error(s) encountered
In the real code it is hard to figure out what line causes an error. It appears all the code to track line numbers is in place, but not used when reporting errors. Simple patch below.
|Additional Information||diff --git a/ocamldoc/odoc_comments.ml b/ocamldoc/odoc_comments.ml|
index ce96f2a..61b8f9a 100644
@@ -90,7 +90,7 @@ module Info_retriever =
Failure s ->
incr Odoc_global.errors ;
- prerr_endline (file^" : "^s^"\n");
+ Printf.eprintf "%s : line %d : %s\n%!" file (!Odoc_lexer.line_number + 1) s;
| Odoc_text.Text_syntax (l, c, s) ->
incr Odoc_global.errors ;
|Tags||No tags attached.|
|Attached Files|| ocamldoc-better-error-message-for-Desc-token.-Fix-6069.patch [^] (2,941 bytes) 2013-07-23 13:17 [Show Content]
ocamldoc-better-error-message-for-Desc-token-v2.patch [^] (1,682 bytes) 2013-07-24 12:08 [Show Content]
|It's not only the error message, but also this "empty token" situation that needs to be fixed.|
|The attached patch introduces an explicit error message, saying: in arguments @-tags, others @-sign should be escaped.|
There are several different things in this patch:
(1) a way to report line errors for lexing errors in ocamldoc
(2) a new lexing rule to accept single "@" in inputs instead of having a lexer failure
(3) an error message for this particular error
I think (1) could be done better, but that would require more effort. The patch format is an improvement over the previous format, but it does not respect the usual OCaml convention (so no emacs location parsing for example), and in particular it does not report character positions. The nice way to fix that would be to call "Location.curr lexbuf" at error points inside the lexer (as done in the usual OCaml parser), or at least when catching the Failure exception in odoc_comments.ml (but then you need to change this code so that you know which "lexbuf" value provoked the error).
I'd say (2) can be easily improved: instead of turning @foo+ into @foo* and then raise an error on the empty foo* case, why not just handle @ alone specifically? That would be more readable. It seems to me that the longest-match precedence rule of lexers should make that work nicely.
I'm not sure I understand (3). Why does it say "in @-tags arguments"? I didn't know about the problem before reading your patch, but I'd say that there is an error for any isolated @ in the comment, not only after, say, a @raise keyword (which expects arguments). Besides, it would be helpful if the error message mentioned explicitly mentioned how to escape (rather than just the need for an escape). I would suggest something like "The character @ has a special meaning in ocamldoc comments, for commands such as @raise or @since. If you want to write a single @, you must escape it as \@.".
(Do we really need to fail when we read @ followed by something else than a lowercase ? I think it is prudent to do so, as it leaves the door open to extending the lexical structure of @-commands in the future.)
Well, my intention was just to track down the error. Hence, fixing point (2) was my sole intention. To improve readability, I've updated this part of the patch as suggested.
For point (3), I tried to be coherent with the ocamldoc manual:
As said in the manual, there is not an error for every isolated @-sign, it appears only after the first @-tags. Still, one may probably improve the error message.
> (Do we really need to fail when we read @ followed by something else than a lowercase ? I think it is prudent to do so, as it leaves the door open to extending the lexical structure of @-commands in the future.)
I remove point (1) from the modified patch.
> As said in the manual, there is not an error for every isolated @-sign
I'm not sure I understand the same thing from the documentation. If
you are referring to this part:
> The inside of documentation comments (**…*) consists of free-form text
> with optional formatting annotations, followed by optional tags giving
> more specific information about parameters, version, authors, … The
> tags are distinguished by a leading @ character. Thus, a documentation
> comment has the following shape:
> (** The comment begins with a description, which is text formatted
> according to the rules described in the next section.
> The description continues until the first non-escaped '@' character.
> @author Mr Smith
> @param x description for parameter x
Then my understanding is that the first non-escaped '@', followed by
a command *or otherwise*, delimits the end of the description
section. There is no mention that non-escaped '@' would be allowed,
when put alone, in the description section (and in fact the
documentation comment example below would be wrong according to this
rule, because of the explicitly mentioned '@'. It should read "until
the first non-escaped \@ character." to be consistent with the
Improved the error message, displaying also the unexpected character.
Rev 13974 in branch 4.O1.
Rev 13975 in trunk.
Oups, did not see the whole discussion before. Added the specific error message for not escaped '@'.
Rev 13976 in 4.01.
Rev 13977 in trunk.
|2013-07-09 06:27||ygrek||New Issue|
|2013-07-09 06:27||ygrek||Status||new => assigned|
|2013-07-09 06:27||ygrek||Assigned To||=> guesdon|
|2013-07-12 17:13||doligez||Note Added: 0009769|
|2013-07-12 17:13||doligez||Target Version||=> 4.01.0+dev|
|2013-07-23 13:16||hnrgrgr||Note Added: 0009832|
|2013-07-23 13:17||hnrgrgr||File Added: ocamldoc-better-error-message-for-Desc-token.-Fix-6069.patch|
|2013-07-23 21:19||gasche||Note Added: 0009837|
|2013-07-24 12:07||hnrgrgr||Note Added: 0009847|
|2013-07-24 12:08||hnrgrgr||File Added: ocamldoc-better-error-message-for-Desc-token-v2.patch|
|2013-07-24 18:19||gasche||Note Added: 0009857|
|2013-07-28 22:30||gasche||Relationship added||related to 0004518|
|2013-08-05 09:46||guesdon||Note Added: 0010102|
|2013-08-05 09:46||guesdon||Status||assigned => closed|
|2013-08-05 09:46||guesdon||Resolution||open => fixed|
|2013-08-05 09:58||guesdon||Note Added: 0010103|
|Copyright © 2000 - 2011 MantisBT Group|