> Pierre Weis <pierre.weis@inria.fr> writes:
> 
> >   As far as I know the best (and simpler) way to do this for reasonable
> >   number of URLs bindings (say thousands but not millions) is to create
> >   a Hashtlbl.t or Map.t and dump it to file using output_value (then
> >   read it back with input_value).
> 
> Is there a recommended data structure in case one needs tables for
> reasonably fast access to millions or tens of millions of values?
> Probably hash tables are no longer providing nearly-constant access
> time at those sizes.  Is there something better in the standard
> library?   
> 
> Thanks --
> 
>         Joshua 

You need to try :) I think that hash table and maps can handle
tens of millions of values if you have enough memory available. If
your hash tables are big enough and if your keys are reasonably
different strings, hash table will still give you nearly-constant
access time.

If you have not enough memory, you should consider mmap facilities
from the Bigarray module (memory mapping of files, accessed from the
disk by need when a given page is indeed accessed by the
application).

All the best for the next year!

Pierre Weis

INRIA, Projet Cristal, Pierre.Weis@inria.fr, http://pauillac.inria.fr/~weis/


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners