Version française
Home     About     Download     Resources     Contact us    

This site is updated infrequently. For up-to-date information, please visit the new OCaml website at

Browse thread
Ocamllex newline counting...
[ Home ] [ Index: by date | by threads ]
[ Search: ]

[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: -- (:)
From: Berke Durak <berke.durak@e...>
Subject: Re: [Caml-list] Ocamllex newline counting...
Till Varoquaux a écrit :
> The title is pretty self explanetory: Ocamllex is able to keep track
> of positions automatically but it needs help with new lines ( you need
> to register new lines with a function like:
>  let newline lexbuf =
>     let pos = lexbuf.lex_curr_p in
>     lexbuf.lex_curr_p <-
>       { pos with pos_lnum = pos.pos_lnum + 1; pos_bol = pos.pos_cnum }
> ).
> This tends to pollute the code and require you to add additional rules
> and underlying machinery. I can see one easy workaround: pipe the
> function you build your lexer from trough an additional function that
> registers newlines. This seems a bit costly at run time but should be
> just fine in most cases.
> Is there any fundamental reason I am missing why newlines are not
> handled natively in the generated automaton?

Well, as you said, wrapping using Lexing.from_function to count newlines
or whatever you want is "just fine" in most cases; especially given that
you'll often want to build a byte-offset-to-line-number table while parsing.
Putting specialized logic in lexing doesn't seem very elegant...

My wild guess would be that the lexer_position record probably got those
extra lnum fields because they were needed in the compiler and they didn't
want to change the interface to insert a type parameter for the positions.
Am I right?