Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Array.make uses twice the required memory #4773

Closed
vicuna opened this issue Apr 19, 2009 · 2 comments
Closed

Array.make uses twice the required memory #4773

vicuna opened this issue Apr 19, 2009 · 2 comments
Labels

Comments

@vicuna
Copy link

vicuna commented Apr 19, 2009

Original bug ID: 4773
Reporter: Camarade_Tux
Status: closed (set by @xavierleroy on 2015-12-11T18:07:13Z)
Resolution: not a bug
Priority: normal
Severity: minor
Platform: x86_64
OS: Linux x86_64
Version: 3.11.0
Category: ~DO NOT USE (was: OCaml general)
Monitored by: @glondu jm @yakobowski

Bug description

Currently calling 'Array.make (1610241024) 0' uses approximately twice the required memory. This doesn't show up immediately though : on 64bit, resident size climbs to (a bit more than) 128MB while virtual size goes up to (roughly) 230MB.
With linux's overcommit this isn't a problem at first but when compacting, the resident size increases to match the virtual one and I run out of memory.

I tried to pinpoint the problem myself but couldn't manage.

Is it possible to reduce the memory usage ?

Steps to reproduce

let a = Array.make (1610241024) 0 in
let () = Gc.compact () in
read_line ()

@vicuna
Copy link
Author

vicuna commented Apr 19, 2009

Comment author: Camarade_Tux

I had forgotten about Bigarrays which have let me reduce memory usage (has been divided by three). Now, should int arrays take less memory or should the documentation be extended to warn about memory usage and point to bigarrays ?

@vicuna
Copy link
Author

vicuna commented May 26, 2009

Comment author: @damiendoligez

This is (more or less) done on purpose. Because of the incremental garbage collector, for every 10 bytes that are in use by your data, there are another 8 bytes allocated to hold the dead data that will be reclaimed by the current (or the next) GC cycle. This 80% overhead is the "space_overhead" parameter of the GC. By lowering this parameter, you can reduce memory consumption at the cost of more time spent in the GC. In your case, if you have a few very large arrays with a very long lifetime, you should be able to reduce the space overhead quite a lot without paying too much in terms of run time.

Unfortunately, I don't know of a simple way to do this automatically.

@vicuna vicuna closed this as completed Dec 11, 2015
@vicuna vicuna added the bug label Mar 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant