Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0004703OCamlOCaml generalpublic2009-01-26 19:002013-09-04 18:08
Reporterdb 
Assigned Toxclerc 
PrioritynormalSeverityfeatureReproducibilityalways
StatusacknowledgedResolutionopen 
PlatformOSOS Version
Product Version3.11.0 
Target VersionFixed in Version 
Summary0004703: Debugging embedded Ocaml applications
DescriptionOcaml now supports ocamldebug under Windows (also due to some my effort); that's quite good. But there is another feature that I am missing: ability to debug embedded Ocaml apps (initialized via caml_startup()). Seems that not only I - discussion with Alain Frish from Lexifi shows that Lexifi people would also be happy if it becomes possible.

So I dug into Ocaml sources and now I can say it's doable with not very much effort. Actually I already have working implementation (tell me if you like to see the patch). Initial idea was the following: to modify bytecode linker so that

ocamlc -g -output-obj -o foo.obj ...

also generates foo.cds (abbreviation: Caml debug & symbols), that is an ordinary bytecode file containing DBUG and SYMB sections. I also modified startup.c to allow debugger initialization in caml_startup_code(). Now then I run

ocamldebug foo.cds

ocamldebug sees there is no CODE section (minor change) and switches into loadingmode=manual. Then I start the debugged app, Ocaml runtime initializes, connects to ocamldebug and everything seems to be working.

So far so good, but there another part of Ocaml runtime where debug info is needed: uncatched exception stack backtrace. Normally Ocaml runtime searches OS path for the bytecode executable, loads debug info and reports backtrace. But now foo.cds is not related to the running process (e.g. Ocaml app can be packed into foo.dll which is loaded by bar.exe). How to locate it? Currently in my prof-of-the-concept implementation I just strip extension from the process executable name and add .cds extension (works under Linux and Windows).

Alain Frish thinks I should get rid of any additional .cds files and just pack DBUG section as .c data. Then ocamldebug loads debug info via additional command in the low-level communication protocol; as for backtraces I should just unpack the .c data.

Of course it's also doable but with much more efforts. All Ocaml debug info reading/writing is channel-oriented (starting from caml_getword/input_binary_int), so some sort of resizable memory buffer structure + primitives should be added. Plus ocamldebug would behave slightly differently. Compare

(ocd) run
Loading program... (...)
has no debugging info.

with

(ocd) run
Loading program...
Waiting for connection...(the socket is 127.0.0.1:11980)
has no debugging info.

"no debug info" status could be reported only after the connection with the debuggee is established. I am afraid that can confuse some existing IDEs (OcaIDE, Emacs mode, etc.)

So I would like to know your opinion. What is the best approach from your point of view that would allow to integrate the patch into the distribution?
Tagspatch
Attached Filesdiff file icon embedded-caml-debug.diff [^] (13,950 bytes) 2009-04-02 19:32 [Show Content]
diff file icon embedded-caml-debug2.diff [^] (14,036 bytes) 2009-04-03 11:35 [Show Content]
diff file icon embedded-caml-debug3.diff [^] (14,834 bytes) 2009-05-05 11:50 [Show Content]

- Relationships

-  Notes
(0004884)
xleroy (administrator)
2009-03-28 18:19

My apologies for taking so long to reply: I read your message with interest when you posted it, then got sidetracked by a number of other things.

Globally, I like the approach you propose. It is certainly a much smaller change than trying to embed debug info in the C object file as Alain proposed. Moreover, I'm a bit nervous about putting lots of debug info as C initialized data in a C executable.

In the meantime, you probably had the opportunity to experiment more with your approach. Does it work well in practice? If you're happy with it, I'm willing to integrate this change in OCaml. Feel free to send a patch if you have one ready.
(0004901)
db (reporter)
2009-04-02 19:32

OK, the patch is now sort of been finalized. I have implemented the backtrace support the following way:
1) When Ocaml linker generates .cds (debug info) file it also stores its full path as a C initialized data.
2) caml_startup() initializes the global caml_cds_name variable with that data.
3) When backtrace machinery searches for a bytecode file it tries caml_cds_name full path first.
4) If not found, caml_cds_name base name is searched in the current process' directory.
5) If not found, caml_cds_name base name is searched in the path specified by CAML_SYMBOL_PATH environment variable.

As for stability/usability - well, it seems to work OK for me (on Win32 platform). I did my best to not break anything Unix-specific, but testing is probably needed. If someone decides to try this any feedback will be appreciated.

The patch is against Ocaml 3.11.
(0004902)
db (reporter)
2009-04-03 11:36

Last minute change. Replace the previous patch with this one.
(0004950)
db (reporter)
2009-05-05 11:51

Fixed a bug. Replace the previous patch with this one (embedded-caml-debug3.diff)
(0004999)
xclerc (developer)
2009-06-18 13:18

Support for debugging of embedded applications has been committed in branch "release311".
It is quite similar to the proposed patch, except that the name of the cds file is not saved in
the executable. Instead, one should set the "CAML_DEBUG_FILE" environment variable
(name chosen to be consistent with the "CAML_DEBUG_SOCKET" variable) to point to the
aforementioned cds file.
(0005000)
db (reporter)
2009-06-20 10:04
edited on: 2009-06-20 10:05

Thanks, but what is wrong with the name of cds file in the debug executable? Just as a hint when CAML_DEBUG_FILE is not set? I proposed that as Microsoft Visual C++ under Windows uses similar approach: the name of .pdb (debug info) is hard-coded into the executable and used when .pdb is not found in the same directory as the executable module.

Using only CAML_DEBUG_FILE is error-prone (you will realize that it is wrong set only when an uncaught exception happen)

(0005713)
frisch (developer)
2010-11-23 08:18

What's the status of this issue? It seems the feature has been implemented but is left undocumented (no mention of it in the manual).
(0006233)
frisch (developer)
2011-12-12 08:51

Reminder sent to: xclerc, xleroy

Shouldn't this entry be marked as fixed (and documented)?
(0007386)
xclerc (developer)
2012-04-23 09:06

Here is a short note, after we reached consensus ; the support for embedded
debugging will be enhanced in the following way.

Besides the use of the "CAML_DEBUG_FILE" environment variable, we will
fallback to the name of the cds file as stored in the executable. Both names
will be interpreted relatively to the path of the executable. Finally, the
executable will refuse to run if there is a MD5 mismatch between itself and
the cds file.

I am in charge of implementing this.
(0007556)
frisch (developer)
2012-06-13 20:03

Any update on this front?
(0007561)
xclerc (developer)
2012-06-14 09:41

I did not have very much time to work on this issue but,
as of today, I encountered no showstopper.

As this does not involve any semantic change, I think it
could be safely committed at any point in development.

- Issue History
Date Modified Username Field Change
2009-01-26 19:00 db New Issue
2009-03-28 18:19 xleroy Note Added: 0004884
2009-03-28 18:19 xleroy Assigned To => xclerc
2009-03-28 18:19 xleroy Status new => acknowledged
2009-04-02 19:32 db File Added: embedded-caml-debug.diff
2009-04-02 19:32 db Note Added: 0004901
2009-04-03 11:35 db File Added: embedded-caml-debug2.diff
2009-04-03 11:36 db Note Added: 0004902
2009-05-05 11:50 db File Added: embedded-caml-debug3.diff
2009-05-05 11:51 db Note Added: 0004950
2009-06-18 13:18 xclerc Note Added: 0004999
2009-06-20 10:04 db Note Added: 0005000
2009-06-20 10:05 db Note Edited: 0005000
2010-11-23 08:18 frisch Note Added: 0005713
2011-12-12 08:51 frisch Note Added: 0006233
2012-04-23 09:06 xclerc Note Added: 0007386
2012-06-13 20:03 frisch Note Added: 0007556
2012-06-14 09:41 xclerc Note Added: 0007561
2013-09-04 18:08 doligez Tag Attached: patch


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker