Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005626OCamlOCaml backend (code generation)public2012-05-29 17:302014-05-30 11:51
ReporterRichard Jones 
Assigned To 
PrioritylowSeverityminorReproducibilityalways
StatusresolvedResolutionsuspended 
PlatformOSOS Version
Product Version3.12.1 
Target VersionFixed in Version 
Summary0005626: Stack overflow in ocamlopt.opt: Comballoc.combine
DescriptionIf you have a file with a large number of lines:

  let () = ...
  let () = ...
  let () = ...

then ocamlopt.opt will give a stack overflow trying to compile
it. The stack trace on ppc64 is:

#2222 0x00000000100372b4 in .camlComballoc__combine_restart_1107 ()
#2223 0x0000000010037740 in .camlComballoc__combine_1106 ()
#2224 0x00000000100376a4 in .camlComballoc__combine_1106 ()
#2225 0x00000000100376a4 in .camlComballoc__combine_1106 ()
0002226 0x00000000100376a4 in .camlComballoc__combine_1106 ()
#2227 0x00000000100376a4 in .camlComballoc__combine_1106 ()
#2228 0x00000000100376a4 in .camlComballoc__combine_1106 ()
#2229 0x00000000100376a4 in .camlComballoc__combine_1106 ()
0002230 0x00000000100376a4 in .camlComballoc__combine_1106 ()
#2231 0x00000000100376a4 in .camlComballoc__combine_1106 ()
#2232 0x00000000100376a4 in .camlComballoc__combine_1106 ()
#2233 0x00000000100372b4 in .camlComballoc__combine_restart_1107 ()

repeated for thousands of frames.
Steps To ReproduceOn x86-64 you need to use a very large number of lines. The
following causes the stack overflow for me on x86-64 [note:
original problem manifested on ppc64]:

echo "let h = Hashtbl.create 0 let add = Hashtbl.add h" > test.ml
for f in `seq 1 50000`; do echo "let () = add \"$f\" \"bar\"" >> test.ml; done
echo "let () = Gc.compact ()" >> test.ml
ocamlopt.opt test.ml -o test
Additional InformationThe reproducer code looks strange, but this actually hit in
real code on ppc64 where the stack limits are much
smaller, so you get this with only about 5000 lines of
code (hand written, not generated!)
TagsNo tags attached.
Attached Files

- Relationships
related to 0005844resolvedgasche stack overflow compiling medium size list constants 
related to 0005925resolvedgasche Stack overflow in compiler 
related to 0006364resolved ocamlopt crashes with "Fatal error: exception Stack_overflow" 

-  Notes
(0007508)
gasche (developer)
2012-06-03 19:40

As a simple workaround, you can chunk your code in smaller functions.

Basically, ocamlopt works on functions one by one, and some compilation passes are not tail-recursive, but this is not an issue because they run at must on the inner code of a function, not on the whole program or even whole module.

But "let () = ..."-style toplevel expressions are collected in a single "entry point" function that is handled like all other user-defined function. In your example, there are too many of them, and you reach your stack limit.
By splitting these into smaller functions, you can avoid exhausting your stack. I tested for example with the following code adapted from yours, which packs your 50_000 phrases into 500 functions of 100 phrases each, and doesn't raise a Stack Overflow:


  echo "let h = Hashtbl.create 0 let add = Hashtbl.add h" > test.ml
  
  for f in `seq 1 500`
  do
      echo "let f () =" >> test.ml
      for i in `seq 1 100`
      do
          echo "let () = add \"$f$i\" \"bar\" in" >> test.ml
      done
      echo "()" >> test.ml
  
      echo >> test.ml
      echo "let () = f ()" >> test.ml
      echo >> test.ml
  done
  
  echo "let () = Gc.compact ()" >> test.ml


Of course, this is only a workaround. A "better" solution would be to rewrite the involved functions to be tail-recursive, but that is a dubious enterprise: the different passes of the compiler are not tail-recursive, and what fails at the Comballoc step today may raise a Stack Overflow somewhere else tomorrow. Maintainers are not fond of chasing such changes through the compiler, when they degrade code readability (and performance) to fix problems that arise only in exceptional cases that can be relatively easily worked around.
(0007509)
gasche (developer)
2012-06-03 19:43

(bug triaging hat on) I mark the bug "acknowledged" as the issue is real and easy to reproduce thanks to the reporter script, but my gut feeling is that this is too low-priority.
If no developer stands up with an opinion on this, I'll probably move it into the "resolved > suspended" realm at some point in the future.
(0008139)
doligez (administrator)
2012-09-21 14:30

Setting to "confirmed" because I can reproduce the problem.
Setting the priority to "low" because this impacts so few users.
No idea whether we should spend time fixing this.
(0008935)
smimram (reporter)
2013-02-28 11:12

A patch has been proposed in http://caml.inria.fr/mantis/view.php?id=5925 [^]
(0009808)
xleroy (administrator)
2013-07-19 11:19

I am marking this PR as "suspended" for the reasons given in 0005925, namely: making the native-code generator tail-recursive is a very invasive change.

For hand-written code, gasche gave good advice on how to circumvent the issue.

- Issue History
Date Modified Username Field Change
2012-05-29 17:30 Richard Jones New Issue
2012-05-29 18:22 mehdi Note Added: 0007471
2012-05-29 18:24 mehdi Note Deleted: 0007471
2012-06-03 19:40 gasche Note Added: 0007508
2012-06-03 19:43 gasche Note Added: 0007509
2012-06-03 19:43 gasche Status new => acknowledged
2012-07-09 14:54 doligez Target Version => 4.01.0+dev
2012-07-31 13:36 doligez Target Version 4.01.0+dev => 4.00.1+dev
2012-09-21 14:30 doligez Note Added: 0008139
2012-09-21 14:30 doligez Priority normal => low
2012-09-21 14:30 doligez Status acknowledged => confirmed
2012-09-21 14:30 doligez Target Version 4.00.1+dev => 4.01.0+dev
2012-12-05 11:34 gasche Relationship added related to 0005844
2013-02-18 11:27 gasche Relationship added related to 0005925
2013-02-28 11:12 smimram Note Added: 0008935
2013-07-19 11:19 xleroy Note Added: 0009808
2013-07-19 11:19 xleroy Status confirmed => resolved
2013-07-19 11:19 xleroy Resolution open => suspended
2013-07-19 11:19 xleroy Target Version 4.01.0+dev =>
2014-05-30 11:51 gasche Relationship added related to 0006364


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker