On Sat, Dec 5, 2009 at 12:06 PM, Damien Doligez <damien.doligez@inria.fr> wrote:
>
> On 2009-12-04, at 20:09, Aaron Bohannon wrote:
>
>> So in a 15 second run (with no
>> idle time, as I said), it now does about 130 heap compactions instead
>> of 3 and gets better total performance because of it, utterly defying
>> my GC intuition.
>
> What is the size of your heap?  Have you tried compacting only once
> every 2, 3, 5, or 10 loops?

In a 15 sec run, the total allocated memory is about 8GB (from
Gc.allocated_bytes()).  The max heap size reached using one compaction
per loop is about 7MB (on a 64-bit machine, based on the
"top_heap_words" field).  The max heap size reached with no
compactions at all is only about 10MB.  These numbers may seem wierd,
but they are really not too surprising since the program is a
signal-processing pipeline that does a huge number of (purely
functional) Array.map and Array.init operations to build short-lived
intermediate results, and it does the same operatations in each
(sub-)cycle regardless of the input.

Yeah, I can squeeze out a bit more performance by using calling
Gc.compact slightly less often, as suggested, but since we're only
talking about around 5%, it doesn't make much difference.  In theory,
I could also optimize away some allocation by replacing functional
Array ops with imperative Array updates, but the throughput seems good
enough right now---I'm actually just concerned about how to keep the
the maximum GC pause time as low as possible.

 - Aaron