Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0006047OCamlOCaml backend (code generation)public2013-06-22 18:232013-06-24 10:10
Reporteredwin 
Assigned To 
PrioritynormalSeverityminorReproducibilityalways
StatusresolvedResolutionnot fixable 
Platformx86_64OSDebian GNU/LinuxOS VersionWheezy
Product Version4.00.1 
Target VersionFixed in Version 
Summary0006047: Stack trace missing class methods with tail-recursive call (both byte/native-code)
DescriptionWhen a class method calls a tail-recursive function the stack-trace of an exception is missing both the method call and the tail-recursive inner function.
This happens both for both bytecode and native code.
Steps To Reproduce$ ocamlbuild ./z.byte -tag debug
$ ./z.byte
Fatal error: exception Failure("Exception raised but object method missing from backtrace")
Raised at file "pervasives.ml", line 22, characters 22-33
Called from file "z.ml", line 23, characters 2-19
$ ocamlbuild ./z.native -tag debug
$ ./z.native
Fatal error: exception Failure("Exception raised but object method missing from backtrace")
Raised at file "pervasives.ml", line 22, characters 22-33
Called from file "z.ml", line 23, characters 2-19
Additional InformationIf I make the recursive call a non-tail call:
      if pos < len then begin
        handle (pos+1); ()
      end

then I get a better stacktrace, it has the recursive calls, but is still missing
the actual failure location (z.ml:5).
Also bytecode shouldn't inline so I should see the z.ml:19 and z.ml:15 calls too.

$ ./z.byte
Fatal error: exception Failure("Exception raised but object method missing from backtrace")
Raised at file "pervasives.ml", line 22, characters 22-33
Called from file "z.ml", line 11, characters 8-22
Called from file "z.ml", line 11, characters 8-22
Called from file "z.ml", line 11, characters 8-22
Called from file "z.ml", line 11, characters 8-22
Called from file "z.ml", line 24, characters 2-19

I have noticed this a few times before, but it was always for complicated code, it is only now that I found a simple testcase by accident.
TagsNo tags attached.
Attached Files? file icon z.ml [^] (484 bytes) 2013-06-22 18:23 [Show Content]

- Relationships

-  Notes
(0009599)
edwin (reporter)
2013-06-22 18:33

If the run_test invocation is moved to another file (lets say y.ml), then the resulting backtrace contains no lines from z.ml, and this makes hard to track exceptions in z.ml (especially if in a real situation the exception can be raised from multiple places).

$ ./y.byte
Fatal error: exception Failure("Exception raised but object method missing from backtrace")
Raised at file "pervasives.ml", line 22, characters 22-33
Called from file "y.ml", line 5, characters 2-19
(0009600)
gasche (developer)
2013-06-22 22:28
edited on: 2013-06-22 22:31

I don't think there is anything specifically related to methods here. Both the call sites you describe as missing are tail calls, and it is an important semantic property of OCaml that tail calls allow to drop the caller frame and consume no additional memory (besides what's needed to store their arguments).

Storing debug information for function calls that ended in a tail call would change that property, and break the reasoning technique we use on tail calls, that would de-facto change the correctness properties of various coding patterns.

If I understood this situation properly, I would consider this a "not fixable" issue; there is no way to "guess" which call sites should be tail-call for correctness or performance, and which are accidental and should not be optimized to get better debugging information.

What could be useful would be a way to explicitly mark some calls as non-tail (to get more debug information; and conversely ensure that some calls are indeed tail-calls). What you can at least do, currently, is query the .annot file to check tail-call-ness of function calls (in the caml-mode of Emacs that is the "caml-types-show-call" function, to be called when your pointer is on a function call expression).

As you noted, some very simple program transformation currently allow to artificially make a call site non-tail to get better debugging behavior. I'm a bit worried about what the effect of more effective optimizations would be on these coding tricks; I think they should be considered fragiles (eg. just relied upon for the time of a debugging session, and not left in released code and expected to keep working), and therefore no real substitute to explicit annotations -- for example under the form promoted by Alain Frisch in his extension_points branch.

(0009601)
edwin (reporter)
2013-06-22 23:59
edited on: 2013-06-22 23:59

I think there are actually 3 issues here, solving #1 would already be very useful, and no. 2 would be nice to have (no. 3 is wishlist).

Problem no. 1: raising an exception might be a tail-call.

Since they unwind the stack anyway it is not strictly required for them to be tail-calls, isn't it?
Making them non-tail-calls would already improve the situation by allowing me to see where the exception was raised.

Problem no. 2: have the tail-call show up *once* in the stacktrace (i.e. first one) for bytecode.

Even if we have mutually-recursive functions the bytecode interpreter doesn't use the system's stack, so it won't crash. For safety this could be enabled only by an OCAMLRUNPARAM=... flag.

Problem no. 3:

As for tail-calls for recursive and mutually recursive functions for native code I see why that is a problem if we want to preserve stack-traces.
One possibility might be that when frame-pointers are enabled you do not adjust the SP before a tail-call, and restore the proper SP from the frame pointer before a ret, but OCaml doesn't support frame-pointers yet.

(0009602)
gasche (developer)
2013-06-23 20:17

> Problem no. 1: raising an exception might be a tail-call.

The point where the exception was raised does appear in the stack trace: it is in the code of function "failwith" defined in Pervasives. If you had used (raise (Failure "foo")) instead of a tail-call (failwith "foo"), you would have seen this raise-point in the trace.


I don't really understand the details of point 2 and 3, but it seems that any suggestion that would prevent a loop of mutually recursive tail-call of executing in constant memory space is an unacceptable deviation from the expected semantics of functional languages -- unless those calls are explicitly marked as memory-consuming.
(0009606)
xleroy (administrator)
2013-06-24 10:10

gasche's analysis is correct: the three calls that edwin expected to see in the backtrace are tail calls.

For suggestion #1, the compiler has no way to know, when it generates a tail call, that the function being called (i.e. "failwith") will always raise an exception. Suggestions #2 and 0000003, as far as I can see, amount to turn tailcall optimization off, which we cannot afford.

So, this is a "not fixable" situation indeed.

- Issue History
Date Modified Username Field Change
2013-06-22 18:23 edwin New Issue
2013-06-22 18:23 edwin File Added: z.ml
2013-06-22 18:33 edwin Note Added: 0009599
2013-06-22 22:28 gasche Note Added: 0009600
2013-06-22 22:28 gasche Status new => feedback
2013-06-22 22:31 gasche Note Edited: 0009600 View Revisions
2013-06-22 23:59 edwin Note Added: 0009601
2013-06-22 23:59 edwin Status feedback => new
2013-06-22 23:59 edwin Note Edited: 0009601 View Revisions
2013-06-22 23:59 edwin Note Edited: 0009601 View Revisions
2013-06-23 20:17 gasche Note Added: 0009602
2013-06-24 10:10 xleroy Note Added: 0009606
2013-06-24 10:10 xleroy Status new => resolved
2013-06-24 10:10 xleroy Resolution open => not fixable


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker