Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Gc.major () Gc.full_major () bug? #3392

Closed
vicuna opened this issue Dec 21, 2004 · 3 comments
Closed

Gc.major () Gc.full_major () bug? #3392

vicuna opened this issue Dec 21, 2004 · 3 comments
Labels
bug

Comments

@vicuna
Copy link

vicuna commented Dec 21, 2004

Original bug ID: 3392
Reporter: administrator
Status: closed
Resolution: fixed
Priority: normal
Severity: minor
Category: ~DO NOT USE (was: OCaml general)

Bug description

Hello

The following code fragment (attached at the end) has drastically
different behaviour when
the Gc.major call in the loop is commented out. As it is written, with
the call to Gc.major,
the programs memory usage tends to grow without bounds and eventually
crash with an Out_of_memory exception ("Raised by primitive operation
at unknown location"). On my
machine, it uses up to 2.5G. When this call is commented out, the
program runs without problems. Still uses a lot of memory 500M, but
the memory usage is bounded.

A call to Gc.full_major () behaves similarly. Gc.minor () and
Gc.compact () do not exhibit this behaviour.

What could be the cause of this bug?

leonid

let protect ~f ~finally =
try
let rval = f () in finally (); rval
with e -> finally (); raise e

let load_value fname =
let f = open_in_bin fname in
protect ~f:(fun () -> input_value f)
~finally:(fun () -> close_in f)

let save_value fname value =
let f =open_out_bin fname in
protect ~f:(fun () -> output_value f value)
~finally:(fun () -> close_out f)

let load_string_list = (load_value : string -> string list)
let save_string_list = (save_value : string -> string list -> unit)

let tname = "/tmp/random_file"

let () =
let list_length = 2500 in
let string_length = 10240 in
let huge_string_list =
Array.to_list (
Array.init list_length (fun i -> String.create string_length))
in
save_string_list tname huge_string_list;
Random.init (int_of_float (Unix.time ()));
for i = 0 to 100000 do
Gc.major ();
let sl = load_string_list tname in
let len = List.length sl in
let c =
String.get
(List.nth sl (Random.int list_length))
(Random.int string_length)
in
Printf.printf "%c %!" c
done;
@vicuna
Copy link
Author

vicuna commented Dec 22, 2004

Comment author: administrator

thank you.

On Wed, 22 Dec 2004 17:06:34 +0100 (MET), Damien Doligez
caml-bugs@pauillac.inria.fr wrote:

Hello

The following code fragment (attached at the end) has drastically
different behaviour when
the Gc.major call in the loop is commented out. As it is written, with
the call to Gc.major,
the programs memory usage tends to grow without bounds and eventually
crash with an Out_of_memory exception ("Raised by primitive operation
at unknown location"). On my
machine, it uses up to 2.5G. When this call is commented out, the
program runs without problems. Still uses a lot of memory 500M, but
the memory usage is bounded.

A call to Gc.full_major () behaves similarly. Gc.minor () and
Gc.compact () do not exhibit this behaviour.

What could be the cause of this bug?

The problem is automatic triggering of compaction. This is normally done
using data collected from the incremental collection. Gc.major() doesn't
satisfy this requirement of incremental collection, so it doesn't compute
the conditions for automatic compaction. I will add some code to Gc.major()
to compute a lower bound on fragmentation, and make it trigger a
compaction if that lower bound is over the limit. I will use the same code
for Gc.full_major(), where it will work in the same way, except that the
computed "lower bound" will be the exact value.

This fix will be available in 3.08.3. With the fix, your program takes a bit
more than 300M on my machine.

Thanks for the bug report.

-- Damien

@vicuna
Copy link
Author

vicuna commented Dec 22, 2004

Comment author: administrator

Hello

The following code fragment (attached at the end) has drastically
different behaviour when
the Gc.major call in the loop is commented out. As it is written, with
the call to Gc.major,
the programs memory usage tends to grow without bounds and eventually
crash with an Out_of_memory exception ("Raised by primitive operation
at unknown location"). On my
machine, it uses up to 2.5G. When this call is commented out, the
program runs without problems. Still uses a lot of memory 500M, but
the memory usage is bounded.

A call to Gc.full_major () behaves similarly. Gc.minor () and
Gc.compact () do not exhibit this behaviour.

What could be the cause of this bug?

The problem is automatic triggering of compaction. This is normally done
using data collected from the incremental collection. Gc.major() doesn't
satisfy this requirement of incremental collection, so it doesn't compute
the conditions for automatic compaction. I will add some code to Gc.major()
to compute a lower bound on fragmentation, and make it trigger a
compaction if that lower bound is over the limit. I will use the same code
for Gc.full_major(), where it will work in the same way, except that the
computed "lower bound" will be the exact value.

This fix will be available in 3.08.3. With the fix, your program takes a bit
more than 300M on my machine.

Thanks for the bug report.

-- Damien

@vicuna
Copy link
Author

vicuna commented Dec 22, 2004

Comment author: administrator

fixed 2004-12-22 DD

@vicuna vicuna closed this as completed Dec 22, 2004
@vicuna vicuna added the bug label Mar 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vicuna