Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0004108OCamlotherlibspublic2006-09-11 18:352017-03-10 11:24
Assigned Todoligez 
PlatformOSOS Version
Product Version3.09.3 
Target VersionlaterFixed in Version 
Summary0004108: Memory-mapping of bigarrays may exhaust address space
DescriptionThis seems to be a bug in the bigarray implementation: when files are memory-mapped to bigarrays, the GC does not work hard enough deallocating and (thus, more importantly here, unmapping) the files. On 32bit architectures this will exhaust the address space if a program maps/forgets large files very quickly, leading to a [Sys_error "Cannot allocate memory"] exception.

The bug seems to be in alloc_bigarray in otherlibs/bigarray/bigarray_stubs.c. The function tests "data == NULL", which is not the case for memory-mapped files, and "size" will therefore not be set to the size of the mapped array. Because "size" has been set to zero before, the alloc_custom-call will therefore assume that memory-mapped files are "for free", which is not quite true: address space is a scarce resource, too. I thus suggest that the "size"-parameter in the call to alloc_custom be initialized correctly for memory-mapped files, too, to make the GC work hard enough.
TagsNo tags attached.
Attached Files

- Relationships
related to 0006294assignedgarrigue Poor tracking of extra heap resources 
related to 0006962acknowledged bigarray lacks a free-like call. 

-  Notes
xleroy (administrator)
2012-04-09 12:42

Controlling the rate of the incremental GC is a difficult issue that the current API for custom allocation doesn't fully address. In 4.00 I added "release" functions to the Bigarray module that support immediate, explicit reclamation of resources (memory and file mappings) associated with a bigarray, just like finalization does but at predictable times. Using these functions wisely should resolve the issue described in this PR, even though it takes more work from the programmer than letting the GC do the finalization.
xleroy (administrator)
2012-05-06 10:26

There were objections to the "release" functions in Bigarray, which were therefore removed. Reopening this PR.

- Issue History
Date Modified Username Field Change
2006-09-11 18:35 mottl New Issue
2006-09-12 14:29 doligez Status new => acknowledged
2012-04-09 12:42 xleroy Note Added: 0007300
2012-04-09 12:42 xleroy Status acknowledged => resolved
2012-04-09 12:42 xleroy Resolution open => suspended
2012-04-09 12:42 xleroy Fixed in Version => 4.00.0+dev
2012-05-06 10:26 xleroy Note Added: 0007415
2012-05-06 10:26 xleroy Status resolved => confirmed
2012-05-06 10:26 xleroy Resolution suspended => open
2012-07-06 16:41 doligez Target Version => 4.01.0+dev
2012-07-31 13:37 doligez Target Version 4.01.0+dev => 4.00.1+dev
2012-09-07 12:52 frisch Target Version 4.00.1+dev => 4.00.2+dev
2013-06-03 22:28 doligez Fixed in Version 4.00.0+dev =>
2013-06-06 23:07 frisch Target Version 4.00.2+dev => 4.01.0+dev
2013-06-14 14:05 frisch Target Version 4.01.0+dev => 4.02.0+dev
2013-07-12 18:15 doligez Target Version 4.02.0+dev => 4.01.1+dev
2014-05-25 20:20 doligez Target Version 4.01.1+dev => 4.02.0+dev
2014-07-17 16:21 doligez Target Version 4.02.0+dev => 4.02.1+dev
2014-09-04 00:25 doligez Target Version 4.02.1+dev => undecided
2014-09-25 17:07 doligez Target Version undecided => 4.02.2+dev / +rc1
2015-02-24 23:15 doligez Target Version 4.02.2+dev / +rc1 => 4.03.0+dev / +beta1
2015-07-25 09:02 xleroy Relationship added related to 0006294
2015-08-18 19:13 xleroy Relationship added related to 0006962
2015-11-27 18:16 frisch Target Version 4.03.0+dev / +beta1 => later
2016-12-07 18:53 shinwell Category OCaml general => OCaml otherlibs
2017-02-23 16:42 doligez Category OCaml otherlibs => otherlibs
2017-03-10 11:24 shinwell Assigned To => doligez
2017-03-10 11:24 shinwell Status confirmed => assigned

Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker