Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005516OCamlOCaml otherlibspublic2012-02-25 04:022012-09-27 17:57
Reportermottl 
Assigned To 
PrioritynormalSeverityminorReproducibilityalways
StatusresolvedResolutionfixed 
PlatformApple MacOSMac OS XOS Version10.7.3
Product Version3.12.1 
Target VersionFixed in Version3.13.0+dev 
Summary0005516: "struct hack" for bigarrays clashes with clang array bounds checks
DescriptionThe most recent release of clang (part of Xcode) for Mac OS X Lion reportedly fails to compile C-bindings accessing bigarrays due to the "struct hack" used to declare the "dim" field in the caml_ba_array declaration. This hack can trigger bogus array bounds errors in user code.

Would it be possible to make this declaration more general? It seems that using "CAML_BA_MAX_NUM_DIMS" instead of "1" as declared size (+ accordingly adapting caml_ba_alloc) would work. Or using the C-preprocessor to generate different code if C99 mode is detected as described here (would also require adapting caml_ba_alloc):

  http://bugs.call-cc.org/attachment/ticket/778/0001-Use-flexible-array-member-in-C99-mode-silences-clang.patch [^]
TagsNo tags attached.
Attached Files

- Relationships
related to 0005761resolvedxleroy Incorrect bigarray custom block size 

-  Notes
(0006974)
xleroy (administrator)
2012-02-27 18:47

Tentative fix in SVN trunk, revision 12188. Use a "flexible" array type if C99 or GCC. Tested using GCC under Linux; let us know if it doesn't fix the issue with Clang.
(0006984)
mww (reporter)
2012-02-27 22:47
edited on: 2012-02-27 22:50

Does not seem to work: OCaml itself builds and installs fine;
bin-prot compiles but it's test fails:


I: Running test 'test_runner'
I: Changing directory to 'lib_test'
I: Running command '/opt/local/var/macports/...../ocaml-bin-prot/work/bin-prot-2.0.3/_build/lib_test/test_runner'
..F...............FFFFFFF..........F.....W: Test 'test_runner' fails: Command '/opt/local/var/macports/...../ocaml-bin-prot/work/bin-prot-2.0.3/_build/lib_test/test_runner' terminated with error code 255
I: Changing directory to '/opt/local/var/macports/...../ocaml-bin-prot/work/bin-prot-2.0.3'
I: Running test 'mac_test'
I: Changing directory to 'lib_test'
I: Running command '/opt/local/var/macports/...../ocaml-bin-prot/work/bin-prot-2.0.3/_build/lib_test/mac_test'
msgs: 100000 msg length: 461
write time: 0.198s write rate: 503912.91 msgs/s write throughput: 221.54 MB/s
 read time: 0.276s read rate: 362862.97 msgs/s read throughput: 159.53 MB/s
I: Changing directory to '/opt/local/var/macports/..../ocaml-bin-prot/work/bin-prot-2.0.3'
E: Tests had a 50.00% failure rate
make: *** [all] Error 1


This is OS X 10.7.3/x86_64/clang-llvm 3.0/ocaml 3.12.1 with the bigarray.h patch

(0006988)
mottl (reporter)
2012-02-28 04:42

@mww

please use the latest version of bin-prot (2.0.7), which can be downloaded from here: http://forge.ocamlcore.org/projects/bin-prot [^]

The latest version should fix some portability problems, which are the likely reason for the observed errors.

The next release of bin-prot will be part of the Jane Street Core library at bitbucket, which should be finalized any day now. Until then, I'm afraid, there will still be some version confusion due to multiple download sites.
(0006993)
mww (reporter)
2012-02-28 18:23

bin-prot 2.0.7 also has a test failure rate of 50%; the crash-reporter gives me something like this:

Exception Type: EXC_BAD_ACCESS (SIGBUS)
Exception Codes: KERN_PROTECTION_FAILURE at 0x000000010055cff0

VM Regions Near 0x10055cff0:
    MALLOC guard page 000000010055b000-000000010055c000 [ 4K] ---/rwx SM=NUL
--> MALLOC metadata 000000010055c000-000000010055d000 [ 4K] r--/rwx SM=PRV
    MALLOC_LARGE 000000010055d000-0000000100600000 [ 652K] rw-/rwx SM=PRV

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libsystem_c.dylib 0x00007fff9214fe7f memmove$VARIANT$sse42 + 450
1 test_runner 0x000000010043446e caml_blit_string + 30
2 test_runner 0x0000000100400b6a .L193 + 37
(0006994)
xleroy (administrator)
2012-02-28 18:52

After rereading the change, I still can't see anything wrong. It doesn't mean nothing is wrong, just that I need help finding what's wrong :-)

I downloaded bin-prot 2.0.7 and tried to test it on my Linux x86-64 box with the development version of OCaml. I gave up because I'm too lazy to set up ocamlfind and the dependencies of bin-prot with the devel version.

To mww: does bin-prot 2.0.7 work on your machine with a stock OCaml 3.12.1 ?
(0006995)
mottl (reporter)
2012-02-28 19:10

I have tried to reproduce the problem on my Mac using clang for compilation (using the -cc flag with the OCaml compiler), but have not managed to trigger any problems. My configuration seems to be the same as yours. The resulting code seems to be running somewhat faster than with gcc, though this library is highly sensitive to code placement, etc.

@mww: please send me an email detailing how exactly you are using clang with bin-prot on your Mac.

@xleroy: it's usually easiest to install Godi first (takes a few human minutes only) and just select the bin-prot package. This should build and install everything you need for testing. Then you can toy with the library source as usual.
(0006996)
mww (reporter)
2012-02-28 21:29

bin-prot works w/o the bigarray patch (though I had to patch bin-prot to silence the array-out-of-bound warnings and to remove all "inline" directives);
if I only patch bigarray.h after compiling ocaml, everything works fine;
(0006998)
doligez (administrator)
2012-03-01 15:54
edited on: 2012-03-01 15:55

Another data point: bin-prot 2.0.3 works with the current trunk (revision 12189) on Mac OS 10.6.

(modulo a trivial change to bin-prot to silence C compiler warnings).

(0006999)
mottl (reporter)
2012-03-01 17:50

I'm pretty certain the observed problem was due to an incorrect patch file applied to the MacPorts OCaml package. It did not contain changes to bigarray_stubs.c, which would make allocated arrays too short with clang.

@mww, if updating the patch accordingly solves your problem, please also let us know here so that this issue can be closed. Thanks!
(0007012)
mww (reporter)
2012-03-06 18:46

ok, I've added also the corresponding changes from bigarray_stubs.c (complete patch is here [1]) and bin-prot seems to compile and work flawless.

Regarding the changes: Don't they break for gcc? At least for Apple's gcc 4.2 on 10.6 this seems to break -- the preprocessor macro that checks for clang/llvm also says gcc is _always_ fine:

#if (__STDC_VERSION__ >= 199901L) || defined(__GNUC__)

This does not seem to work for Apple's XCode on 10.6: gcc 4.2; I changed the patch so the check only reads:

#if (__STDC_VERSION__ >= 199901L)

which seems to work for XCode/gcc 4.2 on 10.6 and also for XCode/clang/llvm 4.3 on 10.7.

[1] https://trac.macports.org/browser/trunk/dports/lang/ocaml/files/patch-otherlibs-bigarray.diff [^]
(0007271)
xleroy (administrator)
2012-04-03 15:52

Thanks for the testing. Normally, a compiler that defines __GNUC__ should implement all GNU extensions to the C language, which include flexible arrays at end of structs, predating C99. But, to be on the safe side, I removed the "|| defined(__GNUC__)". (Commits 12311 and 12312.)

- Issue History
Date Modified Username Field Change
2012-02-25 04:02 mottl New Issue
2012-02-27 18:47 xleroy Note Added: 0006974
2012-02-27 18:47 xleroy Status new => resolved
2012-02-27 18:47 xleroy Resolution open => fixed
2012-02-27 18:47 xleroy Fixed in Version => 3.13.0+dev
2012-02-27 22:47 mww Note Added: 0006984
2012-02-27 22:47 mww Note Edited: 0006984 View Revisions
2012-02-27 22:47 mww Note Edited: 0006984 View Revisions
2012-02-27 22:48 mww Note Edited: 0006984 View Revisions
2012-02-27 22:50 mww Note Edited: 0006984 View Revisions
2012-02-28 04:42 mottl Note Added: 0006988
2012-02-28 18:23 mww Note Added: 0006993
2012-02-28 18:52 xleroy Note Added: 0006994
2012-02-28 19:10 mottl Note Added: 0006995
2012-02-28 21:29 mww Note Added: 0006996
2012-03-01 15:54 doligez Note Added: 0006998
2012-03-01 15:55 doligez Note Edited: 0006998 View Revisions
2012-03-01 17:50 mottl Note Added: 0006999
2012-03-06 18:46 mww Note Added: 0007012
2012-04-03 15:52 xleroy Note Added: 0007271
2012-09-27 17:57 doligez Relationship added related to 0005761


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker