Version française
Home     About     Download     Resources     Contact us    

This site is updated infrequently. For up-to-date information, please visit the new OCaml website at

Browse thread
lexer library
[ Home ] [ Index: by date | by threads ]
[ Search: ]

[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: 2007-08-19 (10:13)
From: Nicolas Pouillard <nicolas.pouillard@g...>
Subject: Re: [Caml-list] lexer library
Excerpts from Jon Harrop's message of Sun Aug 19 00:34:44 +0200 2007:
> On Saturday 18 August 2007 19:39:52 skaller wrote:
> > I am interested in building a dynamically extensible lexer.. can
> > anyone recommend any libraries or tools that would help?
> >
> > I could use Dypgen (extensible GLR parser) for this, but it is
> > probably too slow.
> >
> > A system using combinators (rather than strings) would be preferred,
> > and Unicode support would be nice too. Ideally the lexer automaton
> > could be saved with Marshal, in which case the time to build the
> > automaton isn't important.
> The new camlp4 looks like it supports this but I'm damned if I can figure out 
> how you use it. There is a beautiful tutorial on the Wiki that fizzles out 
> just when it gets exciting.

Not  really,  with  the new camlp4 you can change the lexer and the token type
(thanks  to  the  system  module).  One can also dynamically add lexer filters
that  transforms  the  token stream. However the provided default lexer is not
extensible (it's an ocamllex one).

Nicolas Pouillard aka Ertai