Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wish: make max size of custom space runtime-configurable #6133

Closed
vicuna opened this issue Aug 20, 2013 · 3 comments
Closed

Wish: make max size of custom space runtime-configurable #6133

vicuna opened this issue Aug 20, 2013 · 3 comments

Comments

@vicuna
Copy link

vicuna commented Aug 20, 2013

Original bug ID: 6133
Reporter: gerd
Status: acknowledged (set by @damiendoligez on 2013-08-26T14:57:54Z)
Resolution: open
Priority: normal
Severity: feature
Version: 4.00.1
Category: runtime system and C interface
Tags: patch
Related to: #6294 #7676
Monitored by: @ygrek @hcarty

Bug description

Setting the used and max parameters of caml_alloc_custom is always a problem. If you try to use values other than 0, there is always the risk that the GC is running way too often, reducing the runtime speed heavily. For used=0, RAM gets sometimes out of control. (After writing quite a number of C bindings, my experience is that it is never correct.)

IMO, a better way to control this is to only pass "used" to caml_alloc_custom, and let the user set a limit for max at runtime (i.e. for the sum of all used bytes). This makes it easier to fine-tune the setting for a concrete workload. Also, we can provide other types of limits than just an absolute maximum. E.g. the maximum could be specified as a factor of the ocaml heap. Also, the user could control parameters in self-written code (e.g. read it from a config file, or even develop an own policy that is controlled from GC timers).

The suggestion is to provide

caml_alloc_custom2(ops, size, used)

and to let the user control this with new functions in the Gc module:

type custom_ctrl =
{ custom_abs_max : int; (* absolute maximum in bytes; 0=disable )
custom_rel_max : float; (
number 0-1: relative maximum; 0=disable *)
}

val get_custom_ctrl : unit -> custom_ctrl
val set_custom_ctrl : custom_ctrl -> unit

val incr_custom_space : int -> unit
(* To be used at initialization time of bindings: This number is added
to the current absolute maximum
*)

val get_custom_sum : unit -> int
(* Get the current sum of [used], as figured out in the last major
cycle
*)

val get_custom_rate : unit -> int
(* The sum of all custom allocations in the last major cycle. Intended
for custom policies that are controlled by GC timers
*)

Additional information

related: #4868

@vicuna
Copy link
Author

vicuna commented Aug 20, 2013

Comment author: gerd

The old caml_alloc_custom could still be supported: Either by having two limits in effect, or by reinterpreting used and max relative to the current effective maximum, e.g.

let caml_alloc_custom used max =
caml_alloc_custom2 ~used:(used / max * eff_maximum)

where eff_maximum is the lower limit imposed by custom_abs_max and custom_rel_max at the time of calling.

@vicuna
Copy link
Author

vicuna commented Dec 20, 2017

Comment author: @ygrek

I imagined that such limit should be per-resource type, because they are written by diferent authors who will use different units for used, also sometimes the important resource is not memory but some limited system resource, so they cannot have common denominator.

@xavierleroy
Copy link
Contributor

I believe the new caml_custom_alloc_mem API, as introduced by #1738, answers the need. Holler if it is not the case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants