Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005049OCamlOCaml generalpublic2010-05-18 10:292013-08-31 12:46
Reporterglondu 
Assigned Tofrisch 
PrioritynormalSeverityfeatureReproducibilityalways
StatusclosedResolutionno change required 
PlatformARMOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0005049: natdynlink is broken on arm
DescriptionHello,

While rebuilding all Debian packages (armel port) with ocamlopt enabled (using trunk r10396), ssreflect failed to build. The full build log is available at [1].

The following command crashes with a segmentation fault:

  /usr/bin/coqc -dump-glob theories/ssreflect.glob -q -R src Ssreflect -R theories Ssreflect theories/ssreflect

It uses (nat)dynlink. The bytecode counterpart (by adding "-byte") doesn't segfault. Generating a Coq toplevel with ssreflect statically linked in (with "coqmktop -opt -o ssrcoq src/ssreflect.cmx"), and then using it (by adding "-image ./ssrcoq" to the command above) doesn't segfault either.

Running ocsigen in native code on arm also segfaults:
> root@daribow:/tmp# ocsigen.opt -V
> -- Dependencies of ocsigen.ext.staticmod: ocsigen.commandline, camlp4, ocsigen.ext.staticmod
> -- Needed: /usr/lib/ocaml/ocsigen/parsecommandline.cmxs, /usr/lib/ocaml/ocsigen/staticmod.cmxs
> Loading extension /usr/lib/ocaml/ocsigen/parsecommandline.cmxs
> Loading extension /usr/lib/ocaml/ocsigen/staticmod.cmxs
> Segmentation fault

These errors don't happen on amd64.


Cheers,

--
Stéphane
Additional Information[1] http://ocaml.debian.net/debian/ocaml3120dev23r10396/failures/ssreflect_1.2%2Bdfsg-4%2B3.12.0%2Bdev23%2B10396%2B1_armel.build [^]
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
(0005498)
frisch (developer)
2010-05-25 11:01

Natdynlink is indeed supported only a limited number of platforms. We simply plan to declare ARM not to be in this list (and, accordingly, it will not be installed).
(0005499)
glondu (reporter)
2010-05-25 11:26

What criterion determines whether natdynlink is supported on a given architecture or not? So far, all architectures that support native code generation supported by Debian support natdynlink as well, so I assumed Linux and FreeBSD as a whole did. Could you summarize the technical difficulties? (al least for future reference)

FWIW, armel is currently the third most popular architecture in Debian [1] (after i386 and amd64). I (and I guess many other people) consider it more valuable that e.g. sparc.

[1] http://popcon.debian.org/ [^]
(0005500)
frisch (developer)
2010-05-25 11:38

It turns out that natdynlink is (was) compiled on all platforms that support dynamic linking, but it was never supported on all these platforms.

The problem is that there are also some constraints on the code generated by ocamlopt in order to have a working natdynlink.

I don't think anyone on the caml development team is going to spend energy trying to make natdynlink work for arm, but if you can identify the problem and submit a patch, it will most likely be considered for inclusion.


Would it make things simpler for you if a version of natdynlink were always installed, even when not functional (trying to use it would raise a clean runtime exception)? As opposed to, say, not install dynlink.cmxa at all.
(0005502)
xleroy (administrator)
2010-05-25 13:36

For the record, here are the conditions to be met by a platform for natdynlink to work:

- Either non-PIC code can be put in shared libraries. Then, the ocamlopt code generator need no changes. This is the case for x86-32 bits under Linux, at least. (Recent versions of MacOS X broke this. Under Windows, this works thanks to FlexDLL magic.)

- Or the ocamlopt code generator was changed specifically to generate PIC code. This is the case for x86-64 bits, where the changes and the performance hit were minimal.

- Or the ocamlopt code generator just happens to generate PIC code. I don't think this is the case of any of the other code gens, but Alpha might come very close.

A quick look at ARM code generated by gcc -fPIC suggests that significant work is needed to make ocamlopt produce PIC code on this platform, and the run-time overhead of PIC code is large. So, don't expect a working natdynlink on ARM any time soon.

> FWIW, armel is currently the third most popular architecture in Debian [1] (after i386 and amd64).

... at 1% of the installed base...

> I (and I guess many other people) consider it more valuable that e.g. sparc.

I certainly agree with you on that; witness my efforts to switch ARM to EABI. That doesn't mean that making natdynlinked-ssreflect work on ARM is a priority.
(0005514)
glondu (reporter)
2010-05-25 17:49

frish wrote:
> Would it make things simpler for you if a version of natdynlink were always installed, even when not functional (trying to use it would raise a clean runtime exception)? As opposed to, say, not install dynlink.cmxa at all.

This should probably be asked to a wider audience...

My personal opinion as a programmer would be not to install dynlink.cmxa... as in the OCaml spirit: prefer compile-time errors (such as inexistent dynlink.cmxa) rather than runtime errors. Either way, software relying on dynlink will have to deal with situations where there is ocamlopt, but no natdynlink (the possibilities I see are providing a static counterpart or falling back to bytecode, the latter being the simplest one).

From a maintainer PoV, installing dynlink.cmxa and failing at run-time could have worked if dynlink was never used during the build process... but this approach seems fragile and not very satisfactory.

xleroy wrote:
> the run-time overhead of PIC code is large. So, don't expect a working natdynlink on ARM any time soon.

Does that mean that a patch for this would be rejected, as opposed to Alain's invitation to provide such a patch? I don't say that I would write such a patch myself, but other people might have interest...

> ... at 1% of the installed base...

I didn't hide it :-)

> witness my efforts to switch ARM to EABI

Of course I do, and thank you for that.
(0005515)
Julien Signoles (reporter)
2010-05-25 19:36

=====
Glondu wrote:
frish wrote:
> Would it make things simpler for you if a version of natdynlink were always installed, even when not functional (trying to use it would raise a clean runtime exception)? As opposed to, say, not install dynlink.cmxa at all.

This should probably be asked to a wider audience...
=====

I agree with Stéphane: it is better to have no dynlink.cmxa that a broken one.

Currently the 3 situations already occur anyway:
- no dynlink.cmxa (architectures that do not support natdynlink)
- a broken dynlink.cmxa generating exceptions at runtime (some versions of Mac OS X at least)
- a working dynlink.cmxa

Thus we have to deal with all of these (that is what Frama-C did for instance).
(0005516)
glondu (reporter)
2010-05-26 00:26

I forgot this one:

> - Or the ocamlopt code generator just happens to generate PIC code. I don't think this is the case of any of the other code gens, but Alpha might come very close.

As far as Debian is concerned, ocamlopt is not installed on alpha, but it is on powerpc and sparc and at least ssreflect happens to work there.
(0005519)
frisch (developer)
2010-05-26 08:08

> Either way, software relying on dynlink will have to deal with situations where there is ocamlopt, but no natdynlink (the possibilities I see are providing a static counterpart or falling back to bytecode, the latter being the simplest one).

I believe it is not uncommon to have applications that depends on dynlink only when the user explicitly ask to load some kind of addins, but are still fully functional without dynlink. Having the natdynlink module always available with a way to check whether it works would simplify the code organization and build system for these programs. It is always possible to check the configuration statically if needed (there will be a NATDYNLINK variable in the OCaml config file). There are still cases where dynlink.cmxa is not installed, but this could easily be addressed.

I'm still not 100% sure about the right solution.
(0005520)
Julien Signoles (reporter)
2010-05-26 09:50

Anyway if you want to be compatible with old versions of ocaml (< 3.11) as well as many architectures, you have to handle the case where dynlink.cmxa is not installed. Furthermore you have to handle the incompatibility of Dynlink interface.

Thus, from a developer point of view, I believe that the simplest way to handle all the possible behaviors is to write an unique front-end to Dynlink and to statically generate the implementation according to the configuration. That a work done by any reasonable build system.

But it is still better to have no dynlink.cmxa that a broken one since the error is detected earlier (as Stéphane already said). Moreover, for a build system, it is easier to detect a missing file that a buggy one whatever is the ocaml config file/version.

Above all, I believe the most important think is to be consistent: either you provide dynlink.cmxa on **all** architectures with an easy and documented way to check whether it is working, either you don't provide it on architectures without an usable natdynlink.
(0005526)
frisch (developer)
2010-05-28 12:19

Ok, you convinced me. dynlink.cmxa (and related files) are no longer installed when native dynlink is not supported.

The list of platforms where natdynlink is supported is given in the configure script. Currently, the list is quite conservative, but it should be extended with feedback from users who can test natdynlink on other platforms:

  case "$host" in
    *-*-cygwin*) natdynlink=true;;
    i[3456]86-*-linux*) natdynlink=true;;
    x86_64-*-linux*) natdynlink=true;;
  esac
(0005530)
mehdi (reporter)
2010-05-28 15:20

Maybe I'm not too late here… But, I think that providing dynlink.cmxa on *all* architectures is a better approach for the following reasons:

- It allows to have the same code to compile on any architecture. Otherwise, if we do use natdynlink in some application, we have to modify the code before compiling depending on natdynlink-less native architectures.

- "unix.cmxa" is another example where you have a completely broken module on some architectures (e.g. Windows). Nonetheless, it's installed on Windows but fails gracefully with an exception.

It would be really nice to have a natdynlink module even when dynlink is broken and raise an exception when broken (and even add a "val has_dynlink : bool" (like the test "is_native") to exploit constant inlining…).

> ... at 1% of the installed base...
Keep in mind that on this kind of architecture, popcon is usually disabled (s390 is also very under estimated).
(0005531)
frisch (developer)
2010-05-28 17:12

I'm afraid you're too late, I've already reverted the changes that implemented a Dynlink.is_supported flag, and I don't feel to re-revert it, unless there is some overwhelming popular demand.

(As a side node, I wouldn't say Unix is completely broken on Windows: a lot of its functions work pretty-well.)
(0005532)
frisch (developer)
2010-05-28 17:13

Natdynlink is indeed supported only on some specific platforms, which does not include ARM. Third party packages should be prepared to deal with other platforms and degrade gracefully.
(0006878)
meurer (developer)
2012-02-04 11:19

Natdynlink should now work for ARM starting with revision 12124 (trunk).

- Issue History
Date Modified Username Field Change
2010-05-18 10:29 glondu New Issue
2010-05-25 11:01 frisch Note Added: 0005498
2010-05-25 11:26 glondu Note Added: 0005499
2010-05-25 11:38 frisch Note Added: 0005500
2010-05-25 13:36 xleroy Note Added: 0005502
2010-05-25 13:36 xleroy Status new => feedback
2010-05-25 17:49 glondu Note Added: 0005514
2010-05-25 19:36 Julien Signoles Note Added: 0005515
2010-05-26 00:26 glondu Note Added: 0005516
2010-05-26 08:08 frisch Note Added: 0005519
2010-05-26 09:50 Julien Signoles Note Added: 0005520
2010-05-28 12:19 frisch Note Added: 0005526
2010-05-28 12:20 frisch Severity block => feature
2010-05-28 12:20 frisch Status feedback => acknowledged
2010-05-28 12:20 frisch Platform => ARM
2010-05-28 15:20 mehdi Note Added: 0005530
2010-05-28 17:12 frisch Note Added: 0005531
2010-05-28 17:13 frisch Note Added: 0005532
2010-05-28 17:13 frisch Status acknowledged => resolved
2010-05-28 17:13 frisch Resolution open => no change required
2010-05-28 17:13 frisch Assigned To => frisch
2012-02-04 11:19 meurer Note Added: 0006878
2013-08-31 12:46 xleroy Status resolved => closed


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker