0005318: segfault on stack overflow when reading marshaled data

(0006049)
lefessan (developer)
2011-07-21 22:42

Unfortunately, your data structure is too deep for Marshal.from_string which triggers a stack overflow. Marshal.from_string is optimized for trees, i.e. depth in log(n), and simple lists. You should consider either changing the representation of your doubly linked list, or save it into an array first, save the array with Marshal.to_string, and then do the reverse when unmarshaling.

Fixing this would require implementing another marshal/unmarshal couple, more expensive in memory, but without recursion on the stack.

(0006050)
eugenz (reporter)
2011-07-21 22:47

Still, the same issue arises when outputting into/inputting from a file.

(0006051)
lefessan (developer)
2011-07-21 22:52

Of course, Marshal.from/to_string share the C code with input/output_value, so all of them should trigger the same problem. You should consider saving the doubly linked list in an array first, before outputting it.

(0006052)
eugenz (reporter)
2011-07-21 23:30

OK, thank you.

(0006397)
xleroy (administrator)
2011-12-20 09:47

Feature wish: a non-recursive implementation of byterun/extern.c, perhaps following the same pattern as byterun/compare.c.

(0006417)
pascal_cuoq (reporter)
2011-12-20 15:14

I am relatively sure that the implementation of unmarshal for OCaml 3.12 available in http://frama-c.com/download/frama-c-Nitrogen-20111001.tar.gz [^] (files external/un*.ml*) and documented at http://blog.frama-c.com/public/unmarshal.pdf [^] does not have this issue.

(0006420)
xleroy (administrator)
2011-12-20 15:43

The problem here is with marshaling. But, yes, the unmarshaler in byterun/intern.c is also recursive and vulnerable to stack overflows.

(0007388)
meyer (developer)
2012-04-23 23:41

This has been fixed now (the recursion has been unrolled in both marshaller and un-marshaller).

Please see SVN trunk revisions: 12390, 12250, 12248 and 12247, and also merged into 4.0.

(0007389)
meyer (developer)
2012-04-25 02:52

Merged from trunk to 4.0 as SVN revisions r12392 and r12394.

Issue History
Date Modified	Username	Field	Change
2011-07-21 20:56	eugenz	New Issue
2011-07-21 20:56	eugenz	File Added: issue.ml
2011-07-21 22:42	lefessan	Note Added: 0006049
2011-07-21 22:47	eugenz	Note Added: 0006050
2011-07-21 22:52	lefessan	Note Added: 0006051
2011-07-21 23:30	eugenz	Note Added: 0006052
2011-12-20 09:47	xleroy	Note Added: 0006397
2011-12-20 09:47	xleroy	Status	new => acknowledged
2011-12-20 09:50	xleroy	Summary	sigfault when reading marshaled data => segfault on stack overflow when reading marshaled data
2011-12-20 15:14	pascal_cuoq	Note Added: 0006417
2011-12-20 15:43	xleroy	Note Added: 0006420
2012-02-20 03:00	meyer	Assigned To	=> meyer
2012-02-20 03:00	meyer	Status	acknowledged => assigned
2012-04-23 23:41	meyer	Note Added: 0007388
2012-04-23 23:42	meyer	Status	assigned => resolved
2012-04-23 23:42	meyer	Fixed in Version	=> 4.00.0+dev
2012-04-23 23:42	meyer	Resolution	open => fixed
2012-04-25 02:52	meyer	Note Added: 0007389

Notes
(0006049) lefessan (developer) 2011-07-21 22:42	Unfortunately, your data structure is too deep for Marshal.from_string which triggers a stack overflow. Marshal.from_string is optimized for trees, i.e. depth in log(n), and simple lists. You should consider either changing the representation of your doubly linked list, or save it into an array first, save the array with Marshal.to_string, and then do the reverse when unmarshaling. Fixing this would require implementing another marshal/unmarshal couple, more expensive in memory, but without recursion on the stack.

(0006050) eugenz (reporter) 2011-07-21 22:47	Still, the same issue arises when outputting into/inputting from a file.

(0006051) lefessan (developer) 2011-07-21 22:52	Of course, Marshal.from/to_string share the C code with input/output_value, so all of them should trigger the same problem. You should consider saving the doubly linked list in an array first, before outputting it.

(0006052) eugenz (reporter) 2011-07-21 23:30	OK, thank you.

(0006397) xleroy (administrator) 2011-12-20 09:47	Feature wish: a non-recursive implementation of byterun/extern.c, perhaps following the same pattern as byterun/compare.c.

(0006417) pascal_cuoq (reporter) 2011-12-20 15:14	I am relatively sure that the implementation of unmarshal for OCaml 3.12 available in http://frama-c.com/download/frama-c-Nitrogen-20111001.tar.gz [^] (files external/un.ml) and documented at http://blog.frama-c.com/public/unmarshal.pdf [^] does not have this issue.

(0006420) xleroy (administrator) 2011-12-20 15:43	The problem here is with marshaling. But, yes, the unmarshaler in byterun/intern.c is also recursive and vulnerable to stack overflows.

(0007388) meyer (developer) 2012-04-23 23:41	This has been fixed now (the recursion has been unrolled in both marshaller and un-marshaller). Please see SVN trunk revisions: 12390, 12250, 12248 and 12247, and also merged into 4.0.

(0007389) meyer (developer) 2012-04-25 02:52	Merged from trunk to 4.0 as SVN revisions r12392 and r12394.

Relationships