skaller <skaller@users.sourceforge.net> writes:

> On Tue, 2007-01-02 at 06:58 +1100, Erik de Castro Lopo wrote:
>> That file
>> is:
>> 
>>     96000 * 60 * 60 * 8 * 4 bytes => 
>>     11059.200 Mbytes =>
>>     11.059 Gbytes
>> 
>> Nobody is going to load the whole of that file into memory at once. 
>
> Why not? That's tiny compared to available address space on a 64
> bit machine, and personal computers have heaps
> of free address space.

I had to deal with big files and OCaml too and Erik's approach
sound good to me. 

On 64 bit machines you may mmap huge files, but you can't on 32-bit
machines. I run in troubles with files > 700MB.  Maybe you could mmap
smaller blocks, but this isn't possible with the current
implementation of bigarrays mmap (since you need to mmap with an
offset). Furthermore mmap is a bit different on different operation
system.

Measurements show that mmap doesn't mean a big (or any) speed
up. For the OS the advantage is, that no swap space needs to be
reservered.

Christoph Bauer