Comment author: @alainfrisch

It seems we could indeed scan the minor heap linearly, detect blocks which have been copied to the major heap in order to copy them back. The hard part might be to undo the rewriting of pointers from other blocks into these moved blocks, since the information is not kept, AFAICT. Perhaps one could instead scan from the roots again.

Damien: do you think it is somehow doable?

While the problem is most noticeable on 32-bit architectures, it is also applicable to other systems with constraints on memory usage.

Comment author: @jhjourdan

Another solution, I think, would be to pre-allocate a block of memory for the major heap that is never touched, except when getting out of memory.

When encountering an out of memory, we:
1- use this "emergency" block to empty the minor heap
2- unwind the stack to find the exception handler, thus releasing local roots
3- re-run the major GC (and the compactor), until freeing memory
4- re-allocate the emergency region
5- launch the exception handler

Comment author: @alainfrisch

This is a very interesting approach. Damien: do you see any obstacle or downside to this approach (except that one "waste" the equivalent of the minor heap size)?

Comment author: @lpw25

What happens if you can't reallocate the emergency buffer in step 4?

Comment author: @alainfrisch

What happens if you can't reallocate the emergency buffer in step 4?

I can see several variants:

• Immediately try to reallocate the buffer in step 4; if it fails, simply continue unwinding the stack until the next handler, and iterate. The downside is that this can silently drop handlers that would for instance log the error, or restore some invariants. Of course, if they need to allocate, the fact that they cannot be guaranteed to be executed in case of OOM condition is clear enough, but in case they manage to do their job without allocating, one can do better:

• Instead of reallocating the buffer immediately, keep the allocation pointer to the top of the minor heap (even though it is empty) so that the next allocation will trigger the GC. Only reallocate at this point (and re-raise OOM if not possible). This preserves the expected semantics that the OOM exception is only raised at allocation points; and it guarantees that e.g. a try..finally block that does not allocate can always do its job.

• Immediately try to reallocate the buffer in step 4; if it fails, abort the process as of today.

• Immediately try to reallocate the buffer in step 4; if it fails, split the minor heap in two equal halves (the new minor heap, and the new emergency buffer). Back to previous step if the minor heap becomes too small.

Comment author: @mshinwell

As far as I recall we were considering turning all out-of-memory errors into fatal errors to reduce the number of possibilities for asynchronous exceptions. (cf. #852).

@doligez Can you comment?

This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc.