You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Original bug ID: 4108 Reporter:@mmottl Assigned to:@damiendoligez Status: assigned (set by @mshinwell on 2017-03-10T10:24:19Z) Resolution: open Priority: normal Severity: major Version: 3.09.3 Target version: later Category: otherlibs Related to:#6294#6962#7676 Monitored by:@ygrek till @mmottl
Bug description
This seems to be a bug in the bigarray implementation: when files are memory-mapped to bigarrays, the GC does not work hard enough deallocating and (thus, more importantly here, unmapping) the files. On 32bit architectures this will exhaust the address space if a program maps/forgets large files very quickly, leading to a [Sys_error "Cannot allocate memory"] exception.
The bug seems to be in alloc_bigarray in otherlibs/bigarray/bigarray_stubs.c. The function tests "data == NULL", which is not the case for memory-mapped files, and "size" will therefore not be set to the size of the mapped array. Because "size" has been set to zero before, the alloc_custom-call will therefore assume that memory-mapped files are "for free", which is not quite true: address space is a scarce resource, too. I thus suggest that the "size"-parameter in the call to alloc_custom be initialized correctly for memory-mapped files, too, to make the GC work hard enough.
The text was updated successfully, but these errors were encountered:
Controlling the rate of the incremental GC is a difficult issue that the current API for custom allocation doesn't fully address. In 4.00 I added "release" functions to the Bigarray module that support immediate, explicit reclamation of resources (memory and file mappings) associated with a bigarray, just like finalization does but at predictable times. Using these functions wisely should resolve the issue described in this PR, even though it takes more work from the programmer than letting the GC do the finalization.
This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc.
Original bug ID: 4108
Reporter: @mmottl
Assigned to: @damiendoligez
Status: assigned (set by @mshinwell on 2017-03-10T10:24:19Z)
Resolution: open
Priority: normal
Severity: major
Version: 3.09.3
Target version: later
Category: otherlibs
Related to: #6294 #6962 #7676
Monitored by: @ygrek till @mmottl
Bug description
This seems to be a bug in the bigarray implementation: when files are memory-mapped to bigarrays, the GC does not work hard enough deallocating and (thus, more importantly here, unmapping) the files. On 32bit architectures this will exhaust the address space if a program maps/forgets large files very quickly, leading to a [Sys_error "Cannot allocate memory"] exception.
The bug seems to be in alloc_bigarray in otherlibs/bigarray/bigarray_stubs.c. The function tests "data == NULL", which is not the case for memory-mapped files, and "size" will therefore not be set to the size of the mapped array. Because "size" has been set to zero before, the alloc_custom-call will therefore assume that memory-mapped files are "for free", which is not quite true: address space is a scarce resource, too. I thus suggest that the "size"-parameter in the call to alloc_custom be initialized correctly for memory-mapped files, too, to make the GC work hard enough.
The text was updated successfully, but these errors were encountered: