Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007798OCamlback end (clambda to assembly)public2018-05-24 19:482018-05-25 12:21
Assigned To 
PlatformOSOS Version
Product Version4.06.1 
Target VersionFixed in Version 
Summary0007798: minor_words in gc stats sometimes double counts allocations
DescriptionThe symptom is simple: sometimes you allocate n words and the gc stats increase by 2n.

Here is a program that shows this behavior [1]:
$ ocamlopt && ./a.out
minor heap size: 1048576 = (2 * 524288)
allocated: 524289, gc stats error: 3
allocated: 1048575, gc stats error: 6
allocated: 1572861, gc stats error: 9
allocated: 2097147, gc stats error: 12
allocated: 2621433, gc stats error: 18

This has been causing some very hard to understand spurious failures in allocation tests where the reported allocation is too high by a few words, despite measuring allocation like so (no signals/ocaml finalizers in this context):

let minor_words_allocated f =
  Gc.minor ();
  let before = truncate (Gc.minor_words ()) in
  Sys.opaque_identity (f ());
  let after = truncate (Gc.minor_words ()) in
  after - before

This is because if you're unlucky and the call to Gc.minor finishes the major cycle, the major gc goes into Phase_idle and call caml_request_minor_gc (). Which causes the next minor allocation to trigger a minor gc and be double counted because of the bug above.

The double counting does not happen all the time. caml_alloc is correct. The inline assembly is wrong. The assembly functions used with -compact are also wrong.

After having spent hours figuring out what the problem is, the workaround is easy enough: do one allocation after Gc.minor () to set off the requested minor gc, if any.
I also see that the documentation for minor_words says "this is an approximation in native code", but who knows what that emcompasses? Given that experimentally Gc.minor_words () works perfectly (or so it seemed until now), and that allocation tests are reasonable code to wrote, it's only natural to rely on Gc.minor_words ().

So I'd like to ask for the stats to be fixed (and the documentation to be updated, unless it's referring to something else).


The fix doesn't seem too hard: before calling caml_call_gc, undo the subtraction to the minor heap frontier. In the non inlined-assembly, I think this is not contentious. In the inlined assembly, code size might be a concern? But even that seems fixable (shouldn't matter for big allocations, and for small allocations, one can use a family of caml_call_gc_X functions that change back the heap frontier by X bytes and jump to caml_call_gc).


let rec zero_alloc_bytes_of_int buf n i =
  Bytes.set buf i (Char.unsafe_chr (Char.code '0' + (n mod 10)));
  if n / 10 = 0
  then (
    for j = 0 to i / 2; do
      let tmp = Bytes.get buf j in
      Bytes.set buf j (Bytes.get buf (i - j));
      Bytes.set buf (i - j) tmp;
    i + 1
  else zero_alloc_bytes_of_int buf (n / 10) (i + 1)

let zero_alloc_string_of_int buf n =
  zero_alloc_bytes_of_int buf n 0
let minor_words () = truncate (Gc.minor_words ())

let () =
  let gc = Gc.get () in
  Printf.printf "minor heap size: %d = (2 * %d)\n%!"
    gc.minor_heap_size (gc.minor_heap_size / 2)
let l = ref []
let buf = Bytes.create 100
let () = Gc.compact ()
let () = Gc.compact ()
let () = Gc.compact ()
let () = Gc.minor ()

let () =
  let minor_words_start = minor_words () in
  let minor_words_allocated = ref 0 in
  let stats_error = ref 0 in
  while true do
    l := () :: !l;
    minor_words_allocated := !minor_words_allocated + 3;
    let minor_words_from_stats = minor_words () - minor_words_start in
    let stats_error_now = minor_words_from_stats - !minor_words_allocated in
    if stats_error_now <> !stats_error
    then (stats_error := stats_error_now;
          output_string stdout "allocated: ";
          let len = zero_alloc_string_of_int buf !minor_words_allocated in
          output_substring stdout (Obj.magic (buf : Bytes.t) : string) 0 len;
          output_string stdout ", gc stats error: ";
          let len = zero_alloc_string_of_int buf !stats_error in
          output_substring stdout (Obj.magic (buf : Bytes.t) : string) 0 len;
          output_string stdout "\n";
          flush stdout;
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
stedolan (developer)
2018-05-25 12:10

We ran into this bug on the multicore branch a while ago, where it caused crashes since we assume that the minor heap does not have holes in it. I didn't realise that this affected trunk as well! The fix is as you say (see commit 8ceec49 on

By the way, Alloc_small in caml/memory.h (used for allocations in the bytecode interpreter) already undoes the subtraction before entering the GC, which is why this bug only affects native code.
gasche (developer)
2018-05-25 12:21

For reference: the commit mentioned by stedolan above can be found at the URL [^]

- Issue History
Date Modified Username Field Change
2018-05-24 19:48 sliquister New Issue
2018-05-25 12:10 stedolan Note Added: 0019142
2018-05-25 12:21 gasche Note Added: 0019143
2018-05-25 12:21 gasche Status new => confirmed

Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker