Version française
Home     About     Download     Resources     Contact us    
Browse thread
Faking concurrency using Unix forks and pipes
[ Home ] [ Index: by date | by threads ]
[ Search: ]

[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: -- (:)
From: Richard Jones <rich@a...>
Subject: Re: [Caml-list] Faking concurrency using Unix forks and pipes
On Wed, May 30, 2007 at 04:42:59AM +0100, Jon Harrop wrote:
> 
> Has anyone implemented a parallel map function in OCaml using Unix forks, 
> pipes and maybe marshalling?
> 
> This seems like an easy way to get concurrency in OCaml...

Don't particularly have any code to show but kind of: at Merjis we did
something like this using both MPI and the Ancient module.  (Not at
the same time).

MPI was used to parallelise a simple log parsing job across a cluster
of machines.  This wasn't very successful as it turned out because I
got bitten by Amdahl's law - the thing that was taking the time was
the serial parsing part of the job.

Second approach was to parse the logfiles overnight into a file-backed
memory heap using Ancient.  Then we used several processes to do the
analysis.  The processes were forked, but accessed the file-backed
shared memory directly (so no pipes).

Rich.

-- 
Richard Jones
Red Hat