OK I've found out where it's going wrong and I have a theory, ill-informed
probably, about the problem.

Firstly when I run the compiler with OCAMLRUNPARAM=v=0x1ff, just
before the error I see:

  Growing heap to 4320k bytes
  Growing page table to 34359705221 entries
  No room for growing page table
  Fatal error: out of memory.

and running it under gdb seems to indicate that the problem is a
massive malloc() request, the second call to malloc in the function
caml_add_to_heap.

Now my uninformed theory: The preceeding mmap (from caml_aligned_mmap)
returned a very high memory address:

  mmap(0x722020, 495648, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ffff7f83000

and the runtime is trying to construct a page table (bitmap, right?)
to track both very low and very high addresses.  That's failing,
badly.

Does this sound right?

Unfortunately I have no idea how to solve this, certainly it seems
there is no simple fix ...  (I tried to set vm.overcommit_memory
policy, but that doesn't work because the page table is initialized
right after allocation).

Rich.

-- 
Richard Jones
Red Hat