On Sat, 2006-08-19 at 10:03 +0100, Richard Jones wrote:

> It sounds also like you shouldn't use select at all, but consider some
> of the more modern alternatives.  Here is a good place to start:
> 
> http://www.kegel.com/c10k.html#nb.select

Yeah but if you were doing that you'd probably want 
to use the demux library which is part of Felix ..
in which case you'd want to use it as part of the
context switching too .. in which case you might was
well just use Felix instead of Ocaml :)

I'd be thinking of Equeue if you do want to use 'modern'
alternatives, since it has generic support for 
different notification methods: looks like you could
use a select binding, and switch to some other method
later if you weren't happy with the performance.
Perhaps Gerd can confirm this?

Felix chooses the best of: epoll, kqueue, poll,
windows IOcompletion ports, Solaris iocompletion ports,
and select and abstracts the lot away. The end user
just does blocking I/O .. but it only blocks fibres,
not threads (unless there is nothing ready of course).
[Doesn't the VMthreads of bytecode do this too?]

The difficulty getting this right should not be underestimated.
For example, epoll is available in Linux 2.4 but it doesn't work:
the config script actually has to call it to find out it is
dysfunctional. Felix outperforms C10K by at least 2 decimal
orders of magnitude, without resorting to callbacks, and
without assembler hacks like stack swapping. It does this by
*translating* code using a thread model into code using a
callback model. The demux library demultiplexes notifications
and hooks the results into the scheduling algorithm.

I believe both Haskell and MLton can do this kind of thing
too (though I'm not sure): continuation passing is essential.
I don't think Ocaml uses continuation passing, however most
bytecode interpreters do (including for example Python)
because it's the only way to write a bytecode interpreter :)
Felix uses heaped continuations for procedures, but the
stack for functions.. a fairly novel and rather ugly mix
(required for C compatibility .. and also yielding the best
performance).

All these systems, either implicitly or, now in the case
of Felix quite explicitly .. are replacing poor operating
system designs like Unix. You have to remember Unix was
designed for batch processes with some time sharing:
its hardly surprising it doesn't work well with threads
and real time events .. it was designed to isolate the
end user from these features!

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net