Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0006133OCamlruntime system and C interfacepublic2013-08-20 15:282017-12-20 21:24
Assigned To 
PlatformOSOS Version
Product Version4.00.1 
Target VersionFixed in Version 
Summary0006133: Wish: make max size of custom space runtime-configurable
DescriptionSetting the used and max parameters of caml_alloc_custom is always a problem. If you try to use values other than 0, there is always the risk that the GC is running way too often, reducing the runtime speed heavily. For used=0, RAM gets sometimes out of control. (After writing quite a number of C bindings, my experience is that it is never correct.)

IMO, a better way to control this is to only pass "used" to caml_alloc_custom, and let the user set a limit for max at runtime (i.e. for the sum of all used bytes). This makes it easier to fine-tune the setting for a concrete workload. Also, we can provide other types of limits than just an absolute maximum. E.g. the maximum could be specified as a factor of the ocaml heap. Also, the user could control parameters in self-written code (e.g. read it from a config file, or even develop an own policy that is controlled from GC timers).

The suggestion is to provide

caml_alloc_custom2(ops, size, used)

and to let the user control this with new functions in the Gc module:

type custom_ctrl =
  { custom_abs_max : int; (* absolute maximum in bytes; 0=disable *)
    custom_rel_max : float; (* number 0-1: relative maximum; 0=disable *)

val get_custom_ctrl : unit -> custom_ctrl
val set_custom_ctrl : custom_ctrl -> unit

val incr_custom_space : int -> unit
  (* To be used at initialization time of bindings: This number is added
     to the current absolute maximum

val get_custom_sum : unit -> int
  (* Get the current sum of [used], as figured out in the last major

val get_custom_rate : unit -> int
  (* The sum of all custom allocations in the last major cycle. Intended
     for custom policies that are controlled by GC timers
Additional Informationrelated: 0004868

Attached Files

- Relationships
related to 0006294assignedgarrigue Poor tracking of extra heap resources 
related to 0007676acknowledged Allocating custom blocks become expensive with large major heaps 

-  Notes
gerd (reporter)
2013-08-20 15:52

The old caml_alloc_custom could still be supported: Either by having two limits in effect, or by reinterpreting used and max relative to the current effective maximum, e.g.

let caml_alloc_custom used max =
  caml_alloc_custom2 ~used:(used / max * eff_maximum)

where eff_maximum is the lower limit imposed by custom_abs_max and custom_rel_max at the time of calling.
ygrek (reporter)
2017-12-20 21:24

I imagined that such limit should be per-resource type, because they are written by diferent authors who will use different units for `used`, also sometimes the important resource is not memory but some limited system resource, so they cannot have common denominator.

- Issue History
Date Modified Username Field Change
2013-08-20 15:28 gerd New Issue
2013-08-20 15:52 gerd Note Added: 0010210
2013-08-26 16:57 doligez Status new => acknowledged
2014-01-17 17:50 doligez Relationship added related to 0006294
2014-01-17 17:50 doligez Tag Attached: patch
2017-02-23 16:43 doligez Category OCaml runtime system => runtime system
2017-03-03 17:45 doligez Category runtime system => runtime system and C interface
2017-12-20 18:04 xleroy Relationship added related to 0007676
2017-12-20 21:24 ygrek Note Added: 0018764

Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker