Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ref_table grows without bound #7128

Closed
vicuna opened this issue Jan 18, 2016 · 7 comments
Closed

ref_table grows without bound #7128

vicuna opened this issue Jan 18, 2016 · 7 comments

Comments

@vicuna
Copy link

vicuna commented Jan 18, 2016

Original bug ID: 7128
Reporter: @stedolan
Status: acknowledged (set by @damiendoligez on 2016-02-03T11:00:36Z)
Resolution: open
Priority: normal
Severity: minor
Version: 4.02.3
Target version: later
Category: back end (clambda to assembly)
Monitored by: @gasche @hcarty

Bug description

The ref_table is added to whenever a pointer is created from the old to the young generation. Unfortunately, this can happen an unbounded number of times between two safepoints, since safepoints are only at allocations in the native compiler.

The attached program runs using less than 1MB of memory with the bytecode interpreter. However, when compiled with ocamlopt, it allocates 4GB of ref_table and then crashes, as ocamlopt must wait for a safepoint to clear the ref_table.

File attachments

@vicuna
Copy link
Author

vicuna commented Dec 8, 2016

Comment author: @stedolan

The easiest solution here is to emit safepoints (e.g. a zero-byte allocation) inside loops and at function entry, so that the amount of time between safepoints can be bounded. Such extra safepoints are also useful for multicore, which needs to bound the time between safepoints to handle inter-thread synchronisation.

@vicuna
Copy link
Author

vicuna commented Dec 8, 2016

Comment author: @mshinwell

I'm going to move this to the code generation category, which seems more appropriate.

@vicuna
Copy link
Author

vicuna commented Feb 20, 2017

Comment author: @xavierleroy

This is a known issue. It can also postpone indefinitely signal handling and thread preemption.

The insertion of "safe points" is the solution indeed, but done naively it can be quite costly in run-time. You shoud read "Polling efficiently on stock hardware" by Marc Feeley (FPCA'93, oldie but goldie).

@dc-mak
Copy link
Contributor

dc-mak commented Jul 28, 2019

Just wanted to link a few related issues around the repos here:
Relevance to multi-core ocaml-multicore/ocaml-multicore#187
Report is here
Code (horribly organised) trunk...dc-mak:trunk

@gasche
Copy link
Member

gasche commented Jul 29, 2019

I went to read the nice PDF report and I thought it could be helpful to summarize a bit more information for people ending up on the present PR:

  • @dc-mak did an advanced-compiler-class project to implement Marc Feeley's algorithm for OCaml -- as a transformation on the Cmm code. The result does work as intended, and it seems to have been done properly: it takes into account subtleties such as being careful not to insert safepoints at places where non-traceable data is live in registers.
  • The report contains benchmarks; the code size doesn't increase much, but on micro-benchmarks with tight loops there is a marked decrease in performance (Feeley observed that trivial loop could run 80% slower; here similar 70-90% have been observed in the worst case). On realistic programs the slowdowns should be much smaller, but not much serious measurement has been made there (nobody really knows how to do that well).

@github-actions
Copy link

github-actions bot commented Feb 5, 2021

This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc.

@xavierleroy
Copy link
Contributor

This issue appears to be fixed in 4.14 thanks to the new, proper "safepoints" implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants