Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory-mapping of bigarrays may exhaust address space #4108

Closed
vicuna opened this issue Sep 11, 2006 · 3 comments
Closed

Memory-mapping of bigarrays may exhaust address space #4108

vicuna opened this issue Sep 11, 2006 · 3 comments

Comments

@vicuna
Copy link

vicuna commented Sep 11, 2006

Original bug ID: 4108
Reporter: @mmottl
Assigned to: @damiendoligez
Status: assigned (set by @mshinwell on 2017-03-10T10:24:19Z)
Resolution: open
Priority: normal
Severity: major
Version: 3.09.3
Target version: later
Category: otherlibs
Related to: #6294 #6962 #7676
Monitored by: @ygrek till @mmottl

Bug description

This seems to be a bug in the bigarray implementation: when files are memory-mapped to bigarrays, the GC does not work hard enough deallocating and (thus, more importantly here, unmapping) the files. On 32bit architectures this will exhaust the address space if a program maps/forgets large files very quickly, leading to a [Sys_error "Cannot allocate memory"] exception.

The bug seems to be in alloc_bigarray in otherlibs/bigarray/bigarray_stubs.c. The function tests "data == NULL", which is not the case for memory-mapped files, and "size" will therefore not be set to the size of the mapped array. Because "size" has been set to zero before, the alloc_custom-call will therefore assume that memory-mapped files are "for free", which is not quite true: address space is a scarce resource, too. I thus suggest that the "size"-parameter in the call to alloc_custom be initialized correctly for memory-mapped files, too, to make the GC work hard enough.

@vicuna
Copy link
Author

vicuna commented Apr 9, 2012

Comment author: @xavierleroy

Controlling the rate of the incremental GC is a difficult issue that the current API for custom allocation doesn't fully address. In 4.00 I added "release" functions to the Bigarray module that support immediate, explicit reclamation of resources (memory and file mappings) associated with a bigarray, just like finalization does but at predictable times. Using these functions wisely should resolve the issue described in this PR, even though it takes more work from the programmer than letting the GC do the finalization.

@vicuna
Copy link
Author

vicuna commented May 6, 2012

Comment author: @xavierleroy

There were objections to the "release" functions in Bigarray, which were therefore removed. Reopening this PR.

@github-actions
Copy link

This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants