Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007433OCamlruntime system and C interfacepublic2016-12-16 02:572017-02-19 18:08
Reporterjoris 
Assigned To 
PrioritynormalSeverityminorReproducibilitysometimes
StatusresolvedResolutionsuspended 
PlatformlinuxOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0007433: Minor Gc can takes several minutes to complete
DescriptionWhen the heap becomes quite large, compaction can be really slow to run, taking several minutes. One solution is to increase Gc.max_overhead or periodically trigger compaction when stalling the program for several minutes won't be an issue.
In this case though, another issue emerge. In some cases, minor Gc can takes several minutes to complete.

For instance, this is an example of heap where the issue arise :

[2016-12-16T01:38:41.4598] 39191:0 [memory:info] GC: Heap: 66.4GB (max 66.4GB, chunks 8414) Counters(mi,pr,ma): 54.0TB 2.0TB 3.2TB Collections(mv,ma,mi): 7 1607 14680433
[2016-12-16T01:38:42.9726] 39191:0 [memory:info] VM: rss 68.7GB, vsz 72.5GB, swap 0B, maps 332. MALLOC: size 72.1GB, used 68.9GB, free 1.8GB

In this case, the process became stuck for 6 minutes performing a minor collection:

      7 caml_fl_allocate caml_alloc_shr caml_oldify_one caml_oldify_mopup caml_empty_minor_heap caml_minor_collection caml_make_vect Array.init Tableq.check Tableq.setup_writer List.iter Tableq.start Supertable.on_new_txn Supertable.master_handle Messaging.callback Messaging.0001645
      3 caml_fl_add_blocks caml_alloc_shr caml_oldify_one caml_oldify_mopup caml_empty_minor_heap caml_minor_collection caml_make_vect Array.init Tableq.check Tableq.setup_writer List.iter Tableq.start Supertable.on_new_txn Supertable.master_handle Messaging.callback Messaging.0001645

With the following parameters :
 Minor heap size 64MB
 Compact disabled, performed once an hour manually
 Space overhead 50
Steps To ReproduceThe informations provided here are probably not enough to reproduce, if you have suggestions on how to find more detailed information i run some tests.
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
(0017013)
gasche (developer)
2016-12-16 03:44

If you use 4.03 or above, you can enable GC instrumentation by compiling your program with the "-runtime-variant i" parameter and running it with the environment variable OCAML_INSTR_FILE being set to the path of a log file in which to write instrumentation logs.
(0017015)
lefessan (developer)
2016-12-16 10:17

If you disable compaction, the major heap is going to be fragmented, with lot of small unusable blocks in the free list. As a consequence, when a block is promoted from minor to major, it takes more time to find a block in the free list. That might explain this problem.

I don't remember if Damien implemented the idea of having several free lists sorted by size in 4.04. Did he ?
(0017142)
joris (reporter)
2017-01-10 13:19

Thank you for your suggestion. Sorry I haven't had time to look at this issue yet, and i need to find a proper way to test this without killing production machines.
(0017350)
xleroy (administrator)
2017-02-19 18:08

Also, if compaction is disabled, switching the allocation policy to first-fit could help. (See module Gc, record type "control", field "allocation_policy".)

Until you've gathered more data, I'm setting this issue to "suspended".

- Issue History
Date Modified Username Field Change
2016-12-16 02:57 joris New Issue
2016-12-16 03:44 gasche Note Added: 0017013
2016-12-16 10:17 lefessan Note Added: 0017015
2017-01-10 13:19 joris Note Added: 0017142
2017-02-19 18:08 xleroy Note Added: 0017350
2017-02-19 18:08 xleroy Status new => resolved
2017-02-19 18:08 xleroy Resolution open => suspended
2017-02-23 16:43 doligez Category OCaml runtime system => runtime system
2017-03-03 17:45 doligez Category runtime system => runtime system and C interface


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker