[
Home
]
[ Index:
by date
|
by threads
]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: | 2005-05-24 (22:14) |
From: | Yaron Minsky <yminsky@g...> |
Subject: | Efficient I/O with threads |
We've been running into some interesting problems building highly efficient I/O routines in threaded code in ocaml, and I'm curious if anyone else has some thoughts on this. The basic problem seems to be that the locking and unlocking of the IO channels seems to take a large fraction of the execution time. A little bit of background first. The data type we're outputting is basically a simple s-expression, with the following type: type sexp = Atom of string | List of sexp list We write out an s-expression by writing a tag-byte to determine whether the s-expression is an atom or a string. If the s-expression is an atom, we then write a 4-byte int, which is the length of the string, and then the string. If the s-expression is a list, we write an atom which is the number of s-expression that are contained, and then write those s-expressions. It's very easy to write parsing and marshalling for this type of wire protocol, but that code turns out to be quite inefficient, because you end up making too many calls to the input and output functions, and each one of those calls requires releasing and acquiring locks. I just can't think of a clean way of implementing a reader for this kind of protocol. (a writer could be done by writing stuff to a buffer first, and then writing the whole buffer out at the socket at once.) Any thoughts? Yaron