Version française
Home     About     Download     Resources     Contact us    
Browse thread
lexer function .
[ Home ] [ Index: by date | by threads ]
[ Search: ]

[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: -- (:)
From: Olivier Pons <opons@a...>
Subject: lexer function .


	hello !


I have a little problem with the lexer function generated by make_lexer
in the example below:

   let lexer = make_lexer ["("; ")"; "/\\"; "\//"; "~"; "->"];; 

   let a =  (lexer(stream_of_string"a/\~a"));; (* there is no blank*)
   a : token stream = <abstr>
   #let b = (lexer(stream_of_string"a/\ ~a"));;(* there is a blank*)
   b : token stream = <abstr>

and now I just inspect  the token stream generated

   #stream_next a;;                                              
   - : token = Ident "a"
   #stream_next b;;                            
   - : token = Ident "a"

but in the next call 

   #stream_next a;;
   - : token = Ident "/\\~"

   #stream_next b;;
   - : token = Kwd "/\\

   #stream_next a;;
   - : token = Ident "a"

   #stream_next b;;
   - : token = Kwd "~"
   .......

I don't understand why in the first case the keywords /\\ and ~  are not recognized ?

thank in advance


Olivier