| Anonymous | Login | Signup for a new account | 2013-05-21 23:19 CEST |  |
| Main | My View | View Issues | Change Log | Roadmap |
| View Issue Details [ Jump to Notes ] | [ Issue History ] [ Print ] | ||||||||||
| ID | Project | Category | View Status | Date Submitted | Last Update | ||||||
| 0004868 | OCaml | OCaml otherlibs | public | 2009-09-17 23:34 | 2012-06-30 16:26 | ||||||
| Reporter | elehack | ||||||||||
| Assigned To | doligez | ||||||||||
| Priority | normal | Severity | feature | Reproducibility | N/A | ||||||
| Status | assigned | Resolution | open | ||||||||
| Platform | OS | OS Version | |||||||||
| Product Version | 3.11.1 | ||||||||||
| Target Version | Fixed in Version | ||||||||||
| Summary | 0004868: Request: make CAML_BA_MAX_MEMORY runtime-configurable | ||||||||||
| Description | Currently, Bigarray uses a compiled-in constant CAML_BA_MAX_MEMORY, set to 256MiB, as the maximum size for allocating bigarray custom blocks. While find for low- to medium-memory machines, it causes excessive garbage collector thrashing when using large bigarrays on high-memory machines (e.g. arrays of several gigabytes). I propose the following changes: - change CAML_BA_MAX_MEMORY from a constant to a variable - provide a function to set this variable to a different value if it would help program performance I have made this change on my local copy of OCaml, set the maximum memory to 12GiB, and the GC is substantially less active in the same code (which does not do heavy allocations, but primarily manipulates the bigarrays). | ||||||||||
| Tags | No tags attached. | ||||||||||
| Attached Files |  available_ram.sh [^] (594 bytes) 2012-06-30 16:26 | ||||||||||
 Relationships |
|
 Relationships |
|
Notes |
|
|
(0007627) gerd (reporter) 2012-06-30 16:25 |
Supporting this. I'm also thinking about auto-configuring this. CAML_BA_MAX_MEMORY could be set to 3/4 of the available RAM, where the amount of RAM is calculated by a number of methods: (1) if there is a ulimit, take this (getrlimit for RLIMIT_DATA); (2) otherwise use an OS-specific method (see the attached script available_ram.sh to get an idea). Also thinking that mmaped files (and other memory not allocated by Bigarray directly) should not count here at all. There could be another mechansim here, e.g. giving the GC a one-time boost when the user indicates that the file should be unmapped asap (or just auto-boost whenever a dead mapping is found, to get some automatic speed regulation). Rationale: memory-mapped files do not allocate memory, but only address space. You can nowadays map giant files, and it does not make any sense to include this space into OCaml's memory management calculations. |
|
Notes |
|
Issue History |
|||
| Date Modified | Username | Field | Change |
| 2009-09-17 23:34 | elehack | New Issue | |
| 2011-05-31 16:32 | doligez | Status | new => assigned |
| 2011-05-31 16:32 | doligez | Assigned To | => doligez |
| 2012-06-21 20:16 | frisch | Category | OCaml general => OCaml otherlibs |
| 2012-06-30 16:25 | gerd | Note Added: 0007627 | |
| 2012-06-30 16:26 | gerd | File Added: available_ram.sh | |
|
Issue History |
| Copyright © 2000 - 2011 MantisBT Group |
 |