Browse thread
Re: Threading and SharedMem (Re: [Caml-list] Re: Is OCaml fast?)
-
Stephan Houben
-
oliver@f...
- Eray Ozkural
- Gerd Stolpmann
-
oliver@f...
[
Home
]
[ Index:
by date
|
by threads
]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: | 2010-11-30 (14:09) |
From: | Gerd Stolpmann <info@g...> |
Subject: | Re: Threading and SharedMem (Re: [Caml-list] Re: Is OCaml fast?) |
Am Dienstag, den 30.11.2010, 13:55 +0100 schrieb oliver@first.in-berlin.de: > On Tue, Nov 30, 2010 at 09:10:36AM +0100, Stephan Houben wrote: > > On 11/29/2010 04:33 PM, Oliver Bandel wrote: > > >Zitat von "Gerd Stolpmann" <info@gerd-stolpmann.de>: > > > > > >>Am Montag, den 29.11.2010, 17:12 +0100 schrieb Oliver Bandel: > > >>>Zitat von "Gerd Stolpmann" <info@gerd-stolpmann.de>: > > >>> > > > > >>>You use shared mem(?), but you link only to *.ml files, > > >>>and I see no *.c there. > > > > >>>How can this be done? > > >>> > > >>>At least not via the libs that are shipped with OCaml?! > > > > Actually it can be done using the libs that ship with OCaml > > (Unix and Bigarray), although it is not 100% POSIX : > > > > let create_shared_genarray kind layout dims = > > let fd = Unix.openfile "/dev/zero" [Unix.O_RDWR] 0 > > in let ar = Bigarray.Genarray.map_file fd kind layout true dims > > in Unix.close fd; ar > > > > > > The resulting bigarray object is shared among subsequent forks. > > Hmhhh... we started talking about Threads and SharedMem. > You mean even fork.... hmhhh Independent processes are right now the only way to use several cores. You can organize shared memory between processes, but it is tricky. That's what I try to ease with my Netmulticore library. > > This relies on the fact that mmap-ing /dev/zero is equivalent > > to an anonymous mmap. > > > > http://en.wikipedia.org/wiki//dev/zero > > > > Well, at least it works on Linux. > > In APUE it's mentioned that memory mapped regions are inherited > by a child, when forking it. So it should work on all Unix-systems too. Yes, but is not defined by POSIX what mapping /dev/zero means. > There is one problem with this... when you have forked, then > you obviously have separated processes and also in each process > your own ocaml-program with it's own GC running... > > ..with such a mem-mapping trick (never used Bigarray, so I'm astouned it uses > mmap) Bigarrays can use any memory with fixed addresses. That's the essence here: Bigarrays are not moved around by the GC. > you then have independent processes, working on shared mem without > synchronisation. > > This is a good possibility to get corrupted data, and therefore unreliable behaviour. > > So, you have somehow to create a way of communicating of these processes. So you need inter-process synchronization primitives, like POSIX semaphores. > This already is easily done in the Threads-module, because synchronisation > mechanisms are bound there to the OCaml API and can be used easily. > > In the Unix module there is not much of ths IPC stuff... But in Ocamlnet's netsys module. > (A thread-specific GC for thread-specific variables would help here, > making global locks only necessary when accessing global used variables. > But I don't know if such a way would be possible without changing the GC-stuff > itself.) The global lock does not protect user variables, but the Ocaml runtime, e.g. the state of the memory manager/garbage collector. Also it eases code generation - the memory image needs not be in a consistent state all the time (i.e. all pointers meaningful), but only when the runtime gets a hand on it. Removing this lock has far-reaching consequences. The oc4mc (ocaml for multicore) project used a separate minor heap per thread, which actually eases the task a lot - memory is in most cases allocated in the minor heap anyway. Many variables keeping the state of the runtime are then thread-local. Gerd > > > Ciao, > Oliver > > _______________________________________________ > Caml-list mailing list. Subscription management: > http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list > Archives: http://caml.inria.fr > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > -- ------------------------------------------------------------ Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany gerd@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ------------------------------------------------------------