On Tuesday 22 December 2009 13:11:58 Eray Ozkural wrote:
> On Tue, Dec 22, 2009 at 6:40 AM, Linas Vepstas <linasvepstas@gmail.com> 
wrote:
> > However, if you are  interested in merely using the system
> > to do your "real" work, then writing message-passing code
> > is an utter waste of time -- its difficult, time-consuming, error
> > prone, hard to balance and optimize & tune, works well only
> > for "embarrasingly parallel" code, etc.  Even the evil
> > slow-down of NUMA is often better than trying to
> > performance-tune a message-passing system.
>
> Message passing doesn't work well only for embarrassingly parallel
> code.

Message passing doesn't necessarily work well for embarrassingly-parallel 
problems either because you cannot use in-place algorithms and scatter and 
gather are O(n).

> For instance, you can implement the aforementioned parallel 
> quicksort rather easily,

But you cannot improve performance easily and performance is the *only* 
motivation for parallelism. So the fact that you can make naive use of 
message passing easily from OCaml is useless in practice.

> What message passing really is, it is the perfect match to a
> distributed memory architecture. Since, as you suggest, multicore
> chips have more or less a shared memory architecture, message passing
> is indeed not a good match.

Yes. Conversely, shared memory is effectively a hardware accelerated form of 
message passing.

> > Let me put it this way: suggesting that programmers can
> > write their own message-passing system is kind of like
> > telling them that they can write their own garbage-collection
> > system, or design their own closures, or they can go
> > create their own type system. Of course they can ... and
> > if they wanted to do that, they would be programming in
> > C or assembly, and would probably be designing new
> > languages.  Cause by the time you get done with message
> > passing, you've created a significant and rich programming
> > system that resembles a poorly-designed language... been
> > there, done that.
>
> For a functional language, am I right in expecting a high-level and
> clean interface for explicit parallelism?

I think that is a perfectly reasonable thing to expect but you still need to 
understand its characteristics and how to leverage them in order to make good 
use of the feature.

> I suppose a "spawn" directive would not be very hard to implement.

You cannot implement it with useful efficiency in OCaml.

> Message Passing/Distributed Memory can also be accommodated I suppose.

Sure but it is worth remembering that distributed parallelism across clusters 
is a tiny niche compared to multicores.

> OcamlP3l looks pretty cool. Parallel combinators? Definitely what I'm
> talking about, as usual the future is here with ocaml ;)
>
> http://ocamlp3l.inria.fr/eng.htm

Try solving some real problems with OCamlP3L and F#. I'm sure you'll agree 
that the OCaml approach is certainly not the future.

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e