Version française
Home     About     Download     Resources     Contact us    
Browse thread
ocamllex question
[ Home ] [ Index: by date | by threads ]
[ Search: ]

[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: -- (:)
From: Robert Muller <robert.muller2@g...>
Subject: ocamllex question
I am attempting to use ocamllex together with ocamlyacc to parse a  
subset of python. Python uses indentation to denote
statement blocks so a lexer is sometimes required to return a sequence  
of tokens without advancing the input pointer. In
particular, a lexer for python should return a sequence of so-called  
DEDENT tokens when indented fragments
end. E.g.,

def f(x):
	statement1;
	statement2;
		statement3;
		statement4;
A

the lexer should return two consecutive DEDENT tokens between the '\n'  
at the end of statement4 and the token for A.

Looking at the documentation and examples, it isn't clear how to  
convince the generated lexer to not advance the input pointer
so that two consecutive DEDENT tokens can be returned before the token  
for A is returned.

Any ocamllex perts out there?

Thanks,
Bob Muller