Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007228OCamlruntime system and C interfacepublic2016-04-15 12:532016-04-18 16:49
Reporterfrisch 
Assigned To 
PrioritynormalSeveritymajorReproducibilityhave not tried
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target Version4.03.0+dev / +beta1Fixed in Version4.03.0+dev / +beta1 
Summary0007228: GC not aggressive enough, leads to OOM in simple conditions
DescriptionThe following program:

let () =
  let h = Hashtbl.create 1024 in
  for k = 1 to 1000 do
    Printf.printf "=======================\nROUND %d\n" k;
    flush stdout;

    Hashtbl.clear h;
    for i = 1 to 10000000 do
      Hashtbl.add h i (string_of_int i);
    done
  done


eats more and more memory and crashes after a few rounds (number is not deterministic).

Tests:

MSVC 32-bit:
 trunk : crash after 25 or 19 rounds.
 4.03 : crash after 25 or 19 rounds.
 4.02 : memory usage oscillating between 150Mb and 350Mb; no crash until round 140.

Linux 64-bit:
 trunk : memory usage oscillating between 600Mb and 1Gb, but no failure until round 35.
 4.02 : memory usage oscillating between 500Mb and 1Gb, but no failure until round 140.


Another example:

let () =
  let l = ref [] in
  for k = 1 to 1000 do
    Printf.printf "=======================\nROUND %d\n%!" k;
    l := [];
    for i = 1 to 10000000 do
      l := Bytes.create 20 :: !l
    done
  done


Tests:

MSVC 32-bit:
 trunk : crash after 5 rounds.
 4.02 : memory usage oscillating between 200Mb and 550Mb; no .


Both native and bytecode are affected.

With the last example, here is the output of the test with OCAMLRUNPAM=v=1:

$ OCAMLRUNPARAM=v=1 ./eatmem.exe
=======================
ROUND 1
Starting new major GC cycle
Starting new major GC cycle
Starting new major GC cycle
Starting new major GC cycle
Starting new major GC cycle
Starting new major GC cycle
Starting new major GC cycle
Starting new major GC cycle
Starting new major GC cycle
Starting new major GC cycle
=======================
ROUND 2
=======================
ROUND 3
=======================
ROUND 4
=======================
ROUND 5
Fatal error: out of memory.


Setting OCAMLRUNPARAM=o=50 only delays the problem (crash at round 26).

Adding a call to Gc.major before the inner loop avoids the crash.


With OCAMLRUNPARAM=v=64 and grepping for "computed work", I observe that with 4.02, this amount remains around 1474537, while with 4.03, it oscillates a lot. Here is output of this grep piped to "uniq -c":

      1 computed work = 128574 words
      1 computed work = 284671 words
      1 computed work = 433664 words
      1 computed work = 574463 words
      1 computed work = 550947 words
      1 computed work = 728868 words
      1 computed work = 838521 words
      1 computed work = 1157120 words
      2 computed work = 1330687 words
      1 computed work = 1228898 words
      2 computed work = 1228883 words
      4 computed work = 1474537 words
      3 computed work = 1228848 words
      1 computed work = 1228839 words
      5 computed work = 1474537 words
      1 computed work = 1228831 words
      4 computed work = 1228825 words
      1 computed work = 1228819 words
      8 computed work = 1474537 words
      1 computed work = 1228814 words
      6 computed work = 1228810 words
      2 computed work = 1228806 words
     11 computed work = 1474537 words
      2 computed work = 1228803 words
      9 computed work = 108938 words
     11 computed work = 255035 words
     13 computed work = 382057 words
     10 computed work = 492524 words
     24 computed work = 1474537 words
     15 computed work = 115340 words
     22 computed work = 260583 words
     25 computed work = 386874 words
     28 computed work = 130639 words
     33 computed work = 273877 words
     35 computed work = 121659 words
     65 computed work = 1474537 words
      1 computed work = 4 words
      1 computed work = 182366 words
     27 computed work = 182364 words
      1 computed work = 695 words
      1 computed work = 181672 words
      3 computed work = 182364 words
     58 computed work = 136869 words
     66 computed work = 121051 words
     76 computed work = 127938 words
     88 computed work = 32216 words
     88 computed work = 84244 words
      1 computed work = 321 words
      1 computed work = 83928 words
      1 computed work = 84241 words
     11 computed work = 84244 words
    116 computed work = 52577 words
    134 computed work = 48643 words
    118 computed work = 65748 words
      1 computed work = 250 words
      1 computed work = 65501 words
      1 computed work = 65746 words
     33 computed work = 65748 words
    177 computed work = 38979 words
    169 computed work = 38981 words
      1 computed work = 148 words
      1 computed work = 38834 words
      1 computed work = 38979 words
     32 computed work = 38981 words


It seems that when the heap reaches a certain size, the GC reduces too much the amount of work to do in each slide.
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
(0015768)
frisch (developer)
2016-04-15 17:02

I think I've found the problem. Computing "caml_stat_heap_wsz * 250" in major_gc.c overflows and returns bogus results. Casting caml_stat_heap_wsz to double seems to fix the problem. But there might be other similar places to fix.

Damien: does casting to double seem like a correct fix?
(0015779)
gasche (developer)
2016-04-17 23:01

Alain proposed a fix on github:

  https://github.com/ocaml/ocaml/pull/546 [^]
(0015788)
doligez (administrator)
2016-04-18 16:48

Fixed by GPR#546.

- Issue History
Date Modified Username Field Change
2016-04-15 12:53 frisch New Issue
2016-04-15 12:53 frisch Status new => assigned
2016-04-15 12:53 frisch Assigned To => doligez
2016-04-15 12:55 frisch Description Updated View Revisions
2016-04-15 12:55 frisch Steps to Reproduce Updated View Revisions
2016-04-15 14:00 frisch Description Updated View Revisions
2016-04-15 14:00 frisch Steps to Reproduce Updated View Revisions
2016-04-15 14:55 frisch Description Updated View Revisions
2016-04-15 15:07 frisch Description Updated View Revisions
2016-04-15 15:15 frisch Description Updated View Revisions
2016-04-15 15:16 frisch Description Updated View Revisions
2016-04-15 16:02 frisch Description Updated View Revisions
2016-04-15 16:07 frisch Description Updated View Revisions
2016-04-15 16:11 frisch Description Updated View Revisions
2016-04-15 16:13 frisch Description Updated View Revisions
2016-04-15 16:35 frisch Description Updated View Revisions
2016-04-15 16:35 frisch Description Updated View Revisions
2016-04-15 17:02 frisch Note Added: 0015768
2016-04-17 23:01 gasche Note Added: 0015779
2016-04-18 16:48 doligez Note Added: 0015788
2016-04-18 16:49 doligez Assigned To doligez =>
2016-04-18 16:49 doligez Status assigned => closed
2016-04-18 16:49 doligez Resolution open => fixed
2016-04-18 16:49 doligez Fixed in Version => 4.03.0+dev / +beta1
2017-02-23 16:43 doligez Category OCaml runtime system => runtime system
2017-03-03 17:45 doligez Category runtime system => runtime system and C interface


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker