Re: camlex/camlyacc + threads problem

From: Thierry Bravier (thierry.bravier@dassault-aviation.fr)
Date: Tue Sep 16 1997 - 16:25:38 MET DST

Message-Id: <341E96E2.6CE6@dassault-aviation.fr>
Date: Tue, 16 Sep 1997 16:25:38 +0200
From: Thierry Bravier <thierry.bravier@dassault-aviation.fr>
To: caml-list@inria.fr
Subject: Re: camlex/camlyacc + threads problem

ubject: Re: camlex/camlyacc + threads problem

Olivier Bouyssou (mailto:bouyssou@didac-mip.fr) wrote:
>
> J'ai un probleme lorsque j'utilise une fonction de parsing dans
> plusieurs threads
>

#==============================================================================
French summary:

Chers ocamliens,

A la suite du message d'O. Bouyssou j'ai essaye la version modifiee
de ocamlyacc que j'avais proposee et qui est deja disponible dans
les archives de cette 'mailing list' a
http://pauillac.inria.fr/caml/caml-list/0947.html
(Re: re-entrance of ocamlyacc parsers)

Sur son exemple, les limitations de la version actuelle de ocamlyacc
ne se font plus sentir avec la version modifiee.

Je pense que le probleme vient de l'utilisation combinee des threads
d'ocaml et d'une variable globale dans l'actuel stdlib/parsing.

Sur son exemple, les limitations de la version actuelle de ocamlyacc
ne se font plus sentir avec la version modifiee.

Je pense que le probleme vient de l'utilisation combinee des threads
d'ocaml et d'une variable globale dans l'actuel stdlib/parsing.

En esperant que ces modifications vous sembleront utiles
et/ou que certains voudront reagir a ce message.

La version anglaise est plus fournie ...

Thierry Bravier Dassault Aviation - DGT / DTN / ELO / EAV
78, Quai Marcel Dassault F-92214 Saint-Cloud Cedex - France
Telephone : (33) 01 47 11 53 07 Telecopie : (33) 01 47 11 52 83
E-Mail : mailto:thierry.bravier@dassault-aviation.fr

#==============================================================================
English version:

Dear ocamlers,

I have tried to compile and execute O. Bouyssou's example
(ocamlyacc + threads). Unexpected 'Parse_error' exception are raised
as he said.

In my opinion, the current implementation of stdlib/parsing.ml[i]
suffers from a limitation: in the current version, a global variable
is shared among all ocamlyacc-generated parsers.

In a recent mail about this topic, I stated that this could be a
limitation to parsers re-entrance but did not provide a convincing
example. See http://pauillac.inria.fr/caml/caml-list/0947.html
(Re: re-entrance of ocamlyacc parsers)

In the case of threads, what I understood, is that the ocaml
runtime library schedules threads from one allocation point to
another (in time).

This, I think, is (likely to be) incompatible with a global variable
subject to side-effects.

Anyway, I tried to compile the example with the modified versions of
stdlib/parsing.ml[i] and of ocamlyacc I recently sent to this list.
The executable works correctly (without mutex of course).

The modified ocamlyacc is just a perl-patch on top of the standard
ocamlyacc. I aggree that it would be much better to incorporate
its modifications into the regular ocamlyacc, I just needed to check
this was not a useless work to do.

The modified stdlib/parsing module only provides functions instead of
shared global variables, this solves the problem I think.

The code of the modified version of ocamlyacc is not included here
(it is already available on this list archive).

The example below is compiled by the script mail.build with the
standard or modified version of ocamlyacc.

During execution, a mutex is used iff $MUTEX is set.

Please note that this example does not demonstrate the additional
features of the modified version of ocamlyacc: recursive parser calls
and most of all: how to use a parsing environment (see %env directive).
This has already been presented in my last mail.

In the hope that some of you will find this useful
or will feel like answering this message.

Thierry Bravier Dassault Aviation - DGT / DTN / ELO / EAV
78, Quai Marcel Dassault F-92214 Saint-Cloud Cedex - France
Telephone : (33) 01 47 11 53 07 Telecopie : (33) 01 47 11 52 83
E-Mail : mailto:thierry.bravier@dassault-aviation.fr

# Listing of files:
#==============================================================================
$ ls ../parsing # modified version of stdlib/parsing.ml{,i}
parsing.cmi parsing.cmo parsing.ml parsing.mli

$ ls -ls mail_parser.mly mail_lexer.mll mail_main.ml mail.build
   4 -rwxr-xr-x 1 tb 1290 Sep 16 13:24 mail.build
   4 -rw-r--r-- 1 tb 669 Sep 16 12:43 mail_lexer.mll
   4 -rw-r--r-- 1 tb 502 Sep 16 12:41 mail_main.ml
   4 -rw-r--r-- 1 tb 1220 Sep 16 12:52 mail_parser.mly
 
$ head -9999 mail_parser.mly mail_lexer.mll mail_main.ml mail.build
==> mail_parser.mly <==
/*
============================================================================
 * File: mail_parser.mly
 * Language: camlyacc
 * Author: Thierry Bravier
 * Time-stamp: <97/09/16 12:53:53 tb>
 * Created: 97/09/16 12:44:35 tb
 *
=========================================================================
*/
%{
let below max n = if (n > max) then failwith "below"; n
%}

/*
=========================================================================
*/
%token <int> INT /* Int Literal */
%token COLON /* ":" */
%token HYPHEN /* "-" */
%token EOF /* eof */

/*
=========================================================================
*/
%start mail_yacc
%type <(int * int) list> mail_yacc

%%
/*
=========================================================================
*/
mail_yacc:
| intervals EOF { $1 }
;

intervals:
| interval intervals { $1 :: $2 }
| /* empty */ { [] }
;

interval:
| hour HYPHEN hour { $1, $3 }
;

hour:
| INT { (below 23 $1) * 60 }
| INT COLON INT { (below 23 $1) * 60 + (below 59 $3) }
;

/*
=========================================================================
*/
%%

(*
=========================================================================
*)

==> mail_lexer.mll <==
(*
============================================================================
 * File: mail_lexer.mll
 * Language: camllex
 * Author: Thierry Bravier
 * Time-stamp: <97/09/16 12:44:27 tb>
 * Created: 97/09/16 12:44:08 tb
 *
=========================================================================
*)
{
open Mail_parser
}

(*
=========================================================================
*)
rule mail_lex = parse
|['0'-'9'] ['0'-'9'] ? { INT (int_of_string (Lexing.lexeme lexbuf)) }
| ":" { COLON }
| "-" { HYPHEN }
| eof { EOF }

(*
=========================================================================
*)

==> mail_main.ml <==
let with_mutex =
  try Sys.getenv "MUTEX"; true
  with Not_found -> false

let mutex = Mutex.create ()

let task () =
  while true do
    let lexbuf = Lexing.from_string "0-23:59" in
    if with_mutex then Mutex.lock mutex;
    Mail_parser.mail_yacc Mail_lexer.mail_lex lexbuf;
    if with_mutex then Mutex.unlock mutex;
    if (Random.int 100) = 10 then ThreadUnix.sleep 1;
  done

let main () =
  for i = 0 to 100 do Thread.create task () done;
  while true do ThreadUnix.sleep 1 done
;;

main ()
;;

==> mail.build <==
#!/bin/sh -x
#==============================================================================
# File: mail.build
# Language: sh
# Author: Thierry Bravier
# Time-stamp: <97/09/16 13:25:34 tb>
# Created: 97/09/16 13:24:17 tb
#==============================================================================
STDLIB=/logiciel/ML/ocaml/stdlib
MODIFIED_YACC=/home/asf/tb/ESTEREL/ATLEAST/Root/OCAMLYACC/Ocamlyacc

#==============================================================================
if [ "$1" != 'standard' ]; then
   WHICH='modified';
   OCAMLYACC=${MODIFIED_YACC};
   PARSING_I='-I ../parsing';
   PARSING_F=parsing.cmo;
else
   WHICH='standard';
   OCAMLYACC=ocamlyacc;
   PARSING_I='';
   PARSING_F='';
fi

OCAMLC="ocamlc ${PARSING_I} -c"

#==============================================================================
${OCAMLYACC} mail_parser.mly
${OCAMLC} mail_parser.mli
${OCAMLC} mail_parser.ml

ocamllex mail_lexer.mll
${OCAMLC} mail_lexer.ml

${OCAMLC} -I ${STDLIB}/threads mail_main.ml

ocamlc -o main.${WHICH} \
   -thread -custom unix.cma threads.cma \
   ${PARSING_I} ${PARSING_F} \
   mail_parser.cmo mail_lexer.cmo mail_main.cmo \
   -cclib -lunix -cclib -lthreads

#==============================================================================
$

# Demo with the standard ocamlyacc:
#==============================================================================

# Compilation
#------------------------------------------------------------------------------
$ ./mail.build standard
STDLIB=/logiciel/ML/ocaml/stdlib
MODIFIED_YACC=/home/asf/tb/ESTEREL/ATLEAST/Root/OCAMLYACC/Ocamlyacc
+ [ standard != standard ]
WHICH=standard
OCAMLYACC=ocamlyacc
PARSING_I=
PARSING_F=
OCAMLC=ocamlc -c
+ ocamlyacc mail_parser.mly
+ ocamlc -c mail_parser.mli
+ ocamlc -c mail_parser.ml
+ ocamllex mail_lexer.mll
6 states, 267 transitions, table size 1104 bytes
+ ocamlc -c mail_lexer.ml
+ ocamlc -c -I /logiciel/ML/ocaml/stdlib/threads mail_main.ml
+ ocamlc -o main.standard -thread -custom unix.cma threads.cma
mail_parser.cmo mail_lexer.cmo mail_main.cmo -cclib -lunix -cclib
-lthreads

# Execution : without mutex: Parse_error as said in O. Bouyssou's
message
#------------------------------------------------------------------------------
$ ./main.standard
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Failure("below")
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Failure("below")
Uncaught exception: Parse_error
Uncaught exception: Failure("below")
Uncaught exception: Parse_error
Uncaught exception: Parse_error
Uncaught exception: Parse_error

Ctrl-C

# Execution : with a mutex: no Parse_error as said in O. Bouyssou's
message
#------------------------------------------------------------------------------
$ MUTEX=1 ./main

Ctrl-C

# Demo with the modified ocamlyacc:
#==============================================================================

# Compilation
#------------------------------------------------------------------------------
$ ./mail.build modified
STDLIB=/logiciel/ML/ocaml/stdlib
MODIFIED_YACC=/home/asf/tb/ESTEREL/ATLEAST/Root/OCAMLYACC/Ocamlyacc
+ [ modified != standard ]
WHICH=modified
OCAMLYACC=/home/asf/tb/ESTEREL/ATLEAST/Root/OCAMLYACC/Ocamlyacc
PARSING_I=-I ../parsing
PARSING_F=parsing.cmo
OCAMLC=ocamlc -I ../parsing -c
+ /home/asf/tb/ESTEREL/ATLEAST/Root/OCAMLYACC/Ocamlyacc mail_parser.mly
+ ocamlc -I ../parsing -c mail_parser.mli
+ ocamlc -I ../parsing -c mail_parser.ml
+ ocamllex mail_lexer.mll
6 states, 267 transitions, table size 1104 bytes
+ ocamlc -I ../parsing -c mail_lexer.ml
+ ocamlc -I ../parsing -c -I /logiciel/ML/ocaml/stdlib/threads
mail_main.ml
+ ocamlc -o main.modified -thread -custom unix.cma threads.cma -I
../parsing parsing.cmo mail_parser.cmo mail_lexer.cmo mail_main.cmo
-cclib -lunix -cclib -lthreads

# Execution : without mutex: no Parse_error any more
#------------------------------------------------------------------------------
$ ./main.modified
Ctrl-C

$

This archive was generated by hypermail 2b29 : Sun Jan 02 2000 - 11:58:12 MET