Version française
Home     About     Download     Resources     Contact us    

This site is updated infrequently. For up-to-date information, please visit the new OCaml website at

Browse thread
Shared data space: How to manage Ancient keys?
[ Home ] [ Index: by date | by threads ]
[ Search: ]

[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: 2010-03-19 (11:07)
From: Hugo Ferreira <hmf@i...>
Subject: Re: [Caml-list] Shared data space: How to manage Ancient keys?
Goswin von Brederlow wrote:
> Hugo Ferreira <> writes:
> How about a simple solution. Only one thread handles indexes.
> You said you have worker processes. Add one controling process on top of
> it that spawns all the worker processes and keeps a communication line
> open with them, e.g. a pipe. Each worker would then tell the controller
> to remove an index or request an unused index to store a result. Its
> probably a good idea to request unused indexes in bunches, say 100 at a
> time to cut down on latenzy.

To be honest I have already considered (but not discarded) this option
(which may provide other advantages). This however has several issues:

1) I still need to deal with problem of managing indexes (identifying
    and reusing slots).
2) Each worker has to scan all sequences that have been generated but
    not consumed. So I will also need to generate and send a list of
    available indexes. This may be a very long list.
3) Interaction between the controller and workers need to exchange
    several types of messages:
    a) Request for a sequence to process
    b) Request for a list of sequences to scan
    c) Request an index to use
    d) Acknowledge a used index (maybe)
    e) Request an index to free
    f) Acknowledge a freed index (maybe)
    This implies a complicated protocol with state (worker aware).

1) The workers scan the sequences asynchronously. Scans may have
    various processes access data in read only. Insertion/deletion
    requires locking for read/writes. Use of a controller will
    facilitate this.
2) The controller may avoid having the same sequences scanned
    by two or more workers.

Needless to say these problems are more than I bargained for. 8-(


> MfG
>         Goswin
> _______________________________________________
> Caml-list mailing list. Subscription management:
> Archives:
> Beginner's list:
> Bug reports: