This is just a random idea...

Would it be possible in theory for "input_value" to work by
memory-mapping the file being read, rather than by immediately reading
the file?  The idea would be that the structured value would then only
actually be realised in physical memory as it is touched by execution
and the corresponding pages of the memory-mapped file dragged in by the
virtual memory mechanism.  (To be honest, I haven't actually checked if
this is how input_value currently works, though I'm certain it can't
be.)

This technique would certainly require some modification to the GC, and
I'm not even sure if the relocation of internal pointers in the data
structure could be made to work (do any memory mapping primitives
provide that functionality?).  But if it could work, then that could
make for one of the very best and easiest ways of persisting data
structures - easier than moving to a relational database, and directly
related to the programming model.  In addition, the layout of data
structures on disk could be then be optimized to take into account the
access pattern at runtime.  With a page-fault costing something in the
order of a million cycles these days that could be very valuable...

Cheers,
Don

-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr