Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005737OCamlOCaml internal build/install (Makefiles, configure)public2012-08-24 00:262014-01-21 13:58
Assigned To 
PlatformmxeOSlinuxOS Version
Product Version3.12.1 
Target VersionFixed in Version 
Summary0005737: adapt build system for cross-compilation
I am using mxe on linux to cross-compile my application for windows. It uses camlimages, lablgtk2, lablgl, cairo, xml-light (and ocamlbuild+findlib)
Mxe is a platform for cross-compilation, it includes patches and make commands for hundreds of packages. [^] (main site) [^] (sources) [^] (fork being integrated)

To build ocaml I adapted debian patches maintained by Romain Beauxis, to make it usable with many different targets (i686-pc-mingw32, i686-w64-mingw32, x86_64-w64-mingw32 for 32/64bits targets).

Though it works fine with ocaml 3.12.1, I would like :
- to get your feed back if possible, to know if the process could be simplified (I do not really understand the patch)
- to know if you could integrate part of the code for next versions

Best regards,
Attached Filespatch file icon 0001-various-patches-to-build-an-ocaml-cross-compiler.patch [^] (21,616 bytes) 2012-08-24 00:26 [Show Content]
patch file icon 0001-config-Makefile.mingw64-remove-redundant-mms-bitfiel.patch [^] (1,386 bytes) 2012-12-26 22:23 [Show Content]
patch file icon [^] (486 bytes) 2012-12-26 22:23 [Show Content]
patch file icon 0003-configure-use-stderr-for-all-messages-targetted-at-t.patch [^] (35,508 bytes) 2012-12-26 22:23 [Show Content]
patch file icon 0004-build-fix-filename-case.patch [^] (647 bytes) 2012-12-26 22:23 [Show Content]
patch file icon 0005-build-prefer-forward-slashes-to-backward-slashes.patch [^] (1,487 bytes) 2012-12-26 22:23 [Show Content]
patch file icon 0006-configure-fix-detection-of-non-working-C-compiler.patch [^] (1,581 bytes) 2012-12-26 22:24 [Show Content]
patch file icon [^] (2,419 bytes) 2012-12-26 22:24 [Show Content]
patch file icon [^] (1,364 bytes) 2012-12-26 22:24 [Show Content]
patch file icon 0009-configure-add-target-and-use-target-instead-of-host-.patch [^] (6,539 bytes) 2012-12-26 22:24 [Show Content]
patch file icon 0010-configure-when-cross-compiling-set-TOOLPREF-and-use-.patch [^] (6,731 bytes) 2012-12-26 22:24 [Show Content]
patch file icon 0011-configure-support-cross-compilation-with-mingw.patch [^] (9,166 bytes) 2012-12-26 22:24 [Show Content]
patch file icon 0012-windows-makefiles-remove-unused-DO-variable.patch [^] (1,347 bytes) 2012-12-26 22:24 [Show Content]
patch file icon 0013-windows-makefiles-use-dll-for-the-SO-extension.patch [^] (1,033 bytes) 2012-12-26 22:25 [Show Content]
gz file icon cross-patches-rev2-2012-12-29.tar.gz [^] (22,118 bytes) 2012-12-29 13:27
gz file icon patches_dump.tar.gz [^] (49,294 bytes) 2013-01-23 22:25
gz file icon cross-patches-rev2-2013-02-17.tar.gz [^] (54,029 bytes) 2013-02-17 22:48
gz file icon cross-patches-rev4-2013-02-24.tar.gz [^] (46,908 bytes) 2013-02-24 12:49
gz file icon cross-patches-rev4-2013-06-27-non-specific.tar.gz [^] (11,523 bytes) 2013-06-27 20:39
gz file icon cross-patches-rev4-2013-06-28-safe2.tar.gz [^] (15,547 bytes) 2013-06-28 21:32
patch file icon 0001-build-avoid-an-unneeded-gnu-style-conditional-in-Mak.patch [^] (1,137 bytes) 2013-07-01 09:14 [Show Content]
gz file icon cross-patches-rev4-2013-07-01-safe3.tar.gz [^] (5,112 bytes) 2013-07-01 20:28
gz file icon fix-build-failures-2013-07-01.tar.gz [^] (4,165 bytes) 2013-07-01 23:11
patch file icon 0001-Do-not-run-tests-when-cross-compiling.patch [^] (998 bytes) 2013-07-10 01:54 [Show Content]
gz file icon cross-patches-2013-07-27-configure-to-disable-ocamldebug-ocamlbuild-ocamldoc.tar.gz [^] (7,544 bytes) 2013-07-27 23:31
patch file icon 0001-build-pass-f-Makefile.nt-to-sub-make-processes-of-Ma.patch [^] (1,816 bytes) 2013-07-28 17:56 [Show Content]
patch file icon 0001-build-typo-that-recursively-called-the-wrong-make-ru.patch [^] (946 bytes) 2013-07-28 19:29 [Show Content]
patch file icon 0001-build-fix-build-on-Windows.patch [^] (2,616 bytes) 2013-07-28 22:53 [Show Content]
patch file icon 0001-build-replace-build-mk-config-myocamlbuild_config-.s.patch [^] (12,471 bytes) 2013-08-26 22:46 [Show Content]
gz file icon cross-build_20130829.tar.gz [^] (5,979 bytes) 2013-08-29 00:48
patch file icon 0001-build-replace-build-mk-config-myocamlbuild_config-.s.modified.patch [^] (12,501 bytes) 2013-08-31 02:26 [Show Content]
patch file icon 0001-build-replace-build-mk-config-myocamlbuild_config-.s-2.patch [^] (12,796 bytes) 2013-08-31 17:30 [Show Content]
gz file icon cross-patches-2013-09-07.tar.gz [^] (880,640 bytes) 2013-09-07 22:03
gz file icon cross-patches-2013-09-07-fixed.tar.gz [^] (92,160 bytes) 2013-09-14 14:09
patch file icon 0001-build-fix-make-clean-failure-when-.-configure-hasn-t.patch [^] (2,705 bytes) 2013-09-20 22:15 [Show Content]
patch file icon 0002-build-don-t-use-make-s-C-it-s-not-available-everywhe.patch [^] (1,252 bytes) 2013-09-20 22:15 [Show Content]
patch file icon 0001-build-skip-IFLEXDIR-in-mk_shell_and_ocamlbuild_confi.patch [^] (2,306 bytes) 2013-09-21 11:44 [Show Content]
patch file icon 0001-build-ocamlmklib-on-Windows-expect-a-Windows-style-p.patch [^] (2,831 bytes) 2013-09-23 00:08 [Show Content]
patch file icon 0002-build-typo-on-config-Makefile.mingw64-msvc-and-msvc6.patch [^] (2,037 bytes) 2013-09-23 00:08 [Show Content]
patch file icon [^] (3,274 bytes) 2013-10-02 22:40 [Show Content]
patch file icon 0002-build-always-build-lex-and-tools-with-the-compiler-f.patch [^] (2,384 bytes) 2013-10-02 22:40 [Show Content]
patch file icon 0001-build-always-build-lex-tools-and-ocamlc-with-the-com-fixed.patch [^] (3,304 bytes) 2013-10-03 08:47 [Show Content]

- Relationships

-  Notes
gasche (developer)
2012-09-04 16:18
edited on: 2012-09-04 16:18

I have no authority on the question of whether such patches could be accepted in a future state. I just had a look at the patches related to the OCaml compiler proper (very quick, so feel free to correct misunderstandings): I must they they're not really that convincing.

- some part of them amount to hardcode that the target is windows (see for example [^] and below)

- some part of them are terribly hackish ("Give up and just define _finddata_t explicitly here")

- some part of them seems to be bugfixes or code changes that are independent of the cross-compilation question (I suppose you integrated all debian patches, not only those that straighten the build system). That seems to be the case of the ocamlbuild changes for example [^]

I would understand their integration in a source tree designed solely for unix-to-windows compilation, with the understanding that they will remain as a fork forever. Even then, I would be a bit wary of this integration of all those independent changes in a single patch, making potential errors/divergences more difficult to track (that seems unrelated to your work and inevitable under the current project organization, though; they should consider moving to a structure with multiple patches per project).
But integrating them upstream is clearly impossible in their current state.

If I were a maintainer I wouldn't touch this until the changes are sorted out and given more structure and justification.

I also noticed that a lot of the "ocaml"-related patches do not concern the ocaml compiler proper: cairo, findlib, camlimages, xml-light, etc. are third-party libraries. If you have not yet contacted the upstream of those projects, you should consider doing so. A post on the caml-list may also interest people that would learn about how to make their programs portable with respect to this specific situation and, one can always hope, motivate a few charitable souls to help you getting the patches in better shape?

william (reporter)
2012-09-04 22:46

Thanks for this analysis. My questions were only targeting this: [^]

Indeed, I tried to submit other patches to related people, but did not get many answers...

I would appreciate any hint that would improve the ocaml related patch : [^]
which is the patch given above : [^]

Changes to ocamlbuild were made mainly in order to make ocamlbuild usable for cross targets with ocamlfind integration.

Camarade_Tux (reporter)
2012-12-26 22:32
edited on: 2012-12-26 22:32


Based on William's patch, I made a series of smaller patches.

First one is: 0001-config-Makefile.mingw64-remove-redundant-mms-bitfiel.patch (and last one is 13th). The patch 0002 is something I forgot (I should have moved it out and not submit it). The patchs have some details if you select "show content" so I won't repeat it here. Some patches are trivial but for others, I have some questions on the approach.

* 0003-configure-use-stderr-for-all-messages-targetted-at-t.patch
Some things were printed to stdout and some others to stderr; I now send everything to stderr.

* 0010, 0011, 0013:
Are they a "right" approach? These changes need some polishing but is it ok to add special cases for cross-compilation with hardcoded settings?

Adrien Nader

meyer (developer)
2012-12-28 22:48
edited on: 2012-12-28 22:50


Thanks for the patches. It looks like it's only a build system.

I reviewed them all, my comments below, however haven't attempted to cross compile OCaml yet. I might try it for ARM this weekend.
I think we need at least a second pair of eyes to look at the patches because they look really promising!


* patch 1

looks good apart from the part it would be good to:

- document why we are disabling the -mms-bitfields and what kind of
implications it has

- maybe making it configurable, like a variable for cross compilation
that is empty

* patch 2; ignored, spurious .gitignore/vim stuff

* patch 3

I admire simplicity and portability here, but we should use automatic redirection or custom command. Then if somebody adds another echo, she would be forced to use 'msg' command instead of echo and redirection I can see however that we use it in all the places, so looks Ok.

* patch 4
Looks OK.

* patch 5
Why do we need to touch Makefile.nt?

but looks OK. With a reservation it will be tested on Windows.

* patch 6
Looks OK.

* patch 7

I trust this cases remained the same, so looks ok.

* patch 8

Same as above.

* patch 9
Looks a bit hairy.
First of all why just not substitute host for target before it's been used?

It's certainly improvement to use it but increases a risk, of breaking some exotic builds.

* patch 10
We should initialise TOOLPREF in the begining if we plan to use it.

We should be able to replace these patterns:
with something like:
thus avoiding a lot of redundant pattern matches

it might cause troubles only when something like 'mygcc' was used which is fairly unrelastic.

I think better way of using TOOLPREF would be to, once it's defined, redefine the interesting tools (as & gcc) to ${AS} & ${GCC} with TOOLPREF prefix and then use these variables.

* patch 11


+ x86_64-*-mingw*) echo "Wow! A 64 bit architecture!" 1>&2
+ echo "#undef ARCH_SIXTYFOUR" >> m.h

doesn't sound like a win situation.

It will be more consistent to use variables like ${TOOLCHAIN}. Instead of ${toolchain} but that's just cosmetic.

I've gone through all the mingw cases, but can't guarantee they are correct until I would see myself ocaml crosscompiling.

* Patches 12,13
Looks OK.

Camarade_Tux (reporter)
2012-12-29 13:26
edited on: 2012-12-29 13:42

Thanks for the review.

So far I'm not cross-compiling OCaml successfully. By posting them here I'm trying to get the overall approach validated. There are also some changes that can be merged independently of the others and doing so will reduce the size of subsequent patches (including out-of-tree ones in the case cross-compilation support cannot be included easily).

I've updated the set of patches and I'm going to post them inside a tarball now or this page will soon be unreadable. I've reorganized the patches to move the least intrusive and least dangerous ones first so the patch numbers have changed but in this comment I'll use the same numbers as you did.

The file is named "cross-patches-rev2-2012-12-29.tar.gz".

I've dropped the patch about -mms-bitfields. This switch is now enabled by default for *-w64-mingw*. However this changed only happened in gcc 4.7.

I've introduced inf()/wrn()/err() functions. inf() writes its arguments to stderr; wrn() also writes "WARNING:\n" before; err() also "exit 2".

Forward slashes work just as well on windows. I've changed Makefile.nt to use forward slashes in advance. I'm not sure yet whether I'll be able to use the regular Makefiles or if I'll have to rely on Makefile.nt ones. I've moved the patch down in the list; we'll see a bit later.

Regarding "host" vs. "target" in the configure file, I simply started and went over all the cases using "$host". It turned out that all of them actually want the "target". However it's quite likely "host" will also be needed (probably for ocamlobjinfo for instance: it runs on $host and needs to link to a C library).

> I think better way of using TOOLPREF would be to, once it's defined, redefine the interesting tools (as & gcc) to ${AS} & ${GCC} with TOOLPREF prefix and then use these variables.

I've tried to do that but it touches areas for which I'm not sure I understand the intent. The biggest use of ${TOOLPREF}gcc is around line 830 and starts with:
  case "$arch,$system" in
    amd64,macosx) as="${TOOLPREF}as -arch x86_64"
                    aspp="${TOOLPREF}gcc -arch x86_64 -c";;
As far as I understand, ocaml requires aspp to come from gcc in this setup. Was that the intent or is it because the only compiler available was gcc? And what about clang/llvm?

I need some more input on that.

I've fixed the "#undef ARCH_SIXTYFOUR" issue and I've replaced "toolchain" with "TOOLCHAIN".

edit: I've fixed two issues but I'm not uploading another archive right now:
# if target_type is "unknown", the script will try to cross-compile for the "unknown" architecture

inf/wrn/err functions need to invoke echo with "-e" and need to quote the arguments

meyer (developer)
2012-12-30 04:04

Thanks for the patches, here are my further comments:

* patch 1

I like the cleanup you have made for DBM. I've applied this patch.

* patch 2

Looks good. Applied, thanks.

- patch 3

Not applied.

What kind of problems compatibilty.h gives?

If there are problems we should fix it in the header.

- patch 4

I like the attempt you've made to improve the ./configure script.

I'm also quite reserved about redirecting everything to stderr. Your
patch probably just changes how the user will interact with the script
as instead the user will filter the messages by string which might be
better or not.

By any means, I am tempted to apply these changes, once I know if there
are mandatory, thanks.

- Patch 5

I am resistant to warn the user, would be better to support both
notations? If you decide to do this, please do it for all the flags.

- patch 6

Is incorrect.

# cd config/auto-aux/
# cc=gcc sh ./runtest sizes.c
4 4 4 2

This will print to stdout the sizes of the primitive types.

Quotation will make them spliced into set builtin shell command, thus
setting the arguments to each of the sizes.

However you have issued:
# ret=$?
# set $ret

which just sets the first argument to the return code of the command.

* Patch 7

I'm resistant, looking closely I think it would be a premature

It's a small improvement that buys nothing, and might be reverted in a
future. Is there any particular reason for this cleanup?

* Patch 8

target_type is initially set to "unknown" therefore we shouldn't check
it with test -n. Also, we should print un-coditionaly information about
target, consistency here is more important. I also think we shouldn't
check-in mechanical renaming changes from $target to $host before we
have working cross-compilation so I'd refrain from checking-in the patch

* patch 9

Follow up of 4, so skipped for time being.

* patch 10

I wouldn't drop "support" for old buggy gcc versions, however in general
I am in favour of some cleanups.

Follow up of 4 so not applied yet.

* patch 11

Looks all right, applied.

* patches 12, 13

Don't have mingw system to test it, so I would rather not apply it
before I know it's important to change.

Overall, thank you for your efforts,

Camarade_Tux (reporter)
2012-12-30 16:53

- patch 3

The renaming from stat_alloc() to caml_stat_alloc() comes from William's patch. I haven't had an issue myself and thinking more about it, I'm not sure what would have caused an issue for William. I saw it as an opportunity for a slight update. I'm going to remove it for now; if "namespaced" function names are preferred, it will probably be better to change all of them and not only stat_alloc(), and do it at once. But for now, removed.

- patch 4

I started redirecting messages to stderr because there were some blocks of code that I wanted to use as functions (and therefore have them write to stdout (or stderr or anything)) but which were already writing to stdout. Usage between stderr and stdout was inconsistant but stderr seemed to prevail so I settled for that.

I think that I don't have the original issue anymore and I don't mind where the messages are sent as long as it's consistent. Progress messages are probably better on stdout but if warnings and errors go to stderr, they will probably in the wrong place or out of context. So, everything to stdout?

I think I'll try to redirect inf/wrn/err to "3" or something like that and use the "exec" command to redirect 3 to stdout. That should free stdout and stderr for use in other places and display properly in the terminal. (I'll see how it goes)

It could be good to use colour too for wrn and err but doing it in a portable fashion might be difficult. For later.

- patch 5

I added the check for "-option=val" vs. "-option val" because I made the mistake myself. The check is easy to implement but I'm not sure the support for "=" is. I'll see if '=' can be added to $IFS. However, it seems standard not to use '=' for ocaml

- patch 6

Very good catch. I'm wondering if the issue can be solved with "set". If not, I'll probably introduce size_short/size_long/size_... variables.

- patch 7

The unused variable in the case/in/esac constructs did not cause me any issue so far. I'll drop it.

- patch 8

I caught the issue with target initially being "unknown" but only after making the archive. I've fixed it by making the default "" instead of "unknown".

I'll make it always print the target.

- patch 10

I haven't dropped support for gcc 2.96/2.95. I've simply not touched them. This means the warnings won't appear if you cross-compile with gcc 2.96/29.5, which I consider an unlikely scenario.

Thanks a lot.
Camarade_Tux (reporter)
2013-01-23 22:26


I've attached a new file. This is close to a code dump but I that way I'm somehow creating a backup for the patches. It's not really worth looking at them right now since they need cleaning.
meyer (developer)
2013-01-29 02:05

Dear Adrien,

Sorry to keep you waiting, I'm going to look at your cross-compilation patches this week.
Camarade_Tux (reporter)
2013-02-17 22:57

I've add "cross-patches-rev2-2013-02-17.tar.gz" (still "rev2" by mistake). I still need to clean things a bit but...

ocamlopt.opt works!

There's one thing I'm not verry happy with: I had to do changes asmcomp/ because it uses nativeints for many integers and I build on a 64bit system for a 32bit one; I ended up with several bad integer litterals in the generated assembly. This will definitely need proofreading.

Also, at first I had tried to cross-compile things like ocamldoc while building the cross-compiler. That didn't make much sense (that would only make sense if I were cross-compiling the compiler). In the end I added the ability to disable building ocamldoc and a few others like it is doable for camlp4.
So I did something that will improve cross-compiling the compiler but that is not needed for this bug report. What should I do about these patches?

Try to have them in? Try to have them but after the other ones? Forget about them (for now)?
Depending on the answer I'll reorganize the order of my patches to match so wait for rev4 before reading these patches.

Thanks. :-)
meyer (developer)
2013-02-18 03:10
edited on: 2013-02-23 17:43

Thanks for all the patches. My initial comments below, I said I applied
- but just realised after reviewing 31 small patches that there are 40 of
them, so I'll postpone it for tomorrow. The general comment is that it's
getting into the right track. Some of the patches needs a really
thorough reviewing before we commit them. The ones I marked with a
star are the ones that I think needs some improvements. I'm updating
this in-place as it goes along with applying and testing them. Short
note: did you try your cross-compile patches on a different
architecture, i.e ARM? Also we might try porting them to OPAM.
The reason why I ask is that I do have an ARM device, and would like to
build packages for it, and probably many other people would like.

Patch 1/
I don't think we should keep flexdll.h it in the triangle quotes - it's
not a system header.
However I think it's a part of OCaml installation so we should apply this patch.

Patch 2/
I trust it works, however I am not able to test it.

Patch 3/
Looks fine, thanks. I am not sure if we should prefix it with
underscore, my initial thought was that we shouldn't, but it's a header
not used anywhere else so maybe not worth to change.

Patch 4/
Looks OK to me, applied.

Patch 5/*
The echo -e command prints on my machine also leading -e.

$ ./configure --prefix $PWD
-e Configuring for host i686-pc-linux-gnu ...
-e Configuring for target i686-pc-linux-gnu ...
-e Using compiler gcc.

also the message prefix breaks the line:

./configure -help
-e Unknown option "-help".

The script continues to run even with error:

-e Using compiler arm-linux-gnueabi-gcc.
./tst: 1: Syntax error: word unexpected (expecting ")")
-e Unable to compile the test program.
-e This failure is expected for cross-compilation: we will assume the C compiler is ANSI-compliant.

-e Checking the sizes of integers and pointers...

Patch 6/
It's always best to inform the user about his failure. Applied.

Patch 7/
Applied. Thanks.

Patch 8/
I don't think we need $model anymore, so thanks for cleaning this up. I've applied this patch too.

Patch 9/
This needs to be documented. Probably additional entries to the manual
needs to be done. Especially that we are able to cross compile now at all.

Patch 10/
We should use conventional lowercase variable names in the Caml build
system. TOOLPREF and TOOLCHAIN all are not obeying the convention.

I suspect TOOLPREF/TOOLCHAIN are a mingw specific variable, but can't
find it anywhere. If we decide to expose it, then it needs to be
documented - I propose in the configure script itself as a commentary. But I'd leave it as it is.

My worry here is that we are hardcoding the sizes, instead of detecting

I'll apply this patch however, because can't think about anything better
at the moment.

Patch 11/
I like this patch, applied. Did you try to test it on a different shell than bash?

Patch 12/*
I don't like this: '# XXX' prefix for the comment.

I'd prefer nicer prefix, like TODO or FIXME. However it's not a show
stopper, patch looks complicated so need to make a sweep over this

Patch 13/
I don't have mingw system to test it, and there is 32 how about 64? If
there is another flavour of the library we need to cover that. I
tentatively applied.

Patch 14/
Looks OK.

Patch 15/*
ocaml -version can print other characters than specified in the sed command.

It would be much safer if the you matched for example like this:
sed 's/.*version \(.*\)/\1/g'

if test x"$target" != x"$host"; then
we should be using ==.

Patch 16/
Makes sense, applied.
It looks like a continuation of the previous patch, it would be easier to collate them.

Patch 17/
Looks OK, applied.

Patch 18/
Looks good to me, applied.

Patch 19/
Looks OK, applied.

Patch 20/
Fine. Applied.

Patch 21/
Apart from visible whitespaces it, looks OK.

EDIT: Sorry, I notice that the file have a lot leading whitespaces, I'll
clean them up as they polluting (not necessarily) the file.

Patch 22/
Thank you, applied.

Patch 23/
Don't put comma after `if' condition. Otherwise applied.

Patch 24/*
To be reviewed.

Patch 25/*
Yet to be reviewed again.

Patch 26/
Looks good to me. Thanks.

Patch 27/*
Needs to be reviewed.

Patch 28/*
I like it, but needs more looking.

Patch 29-40
Needs to be reviewed.

meyer (developer)
2013-02-18 04:32

> Try to have them in? Try to have them but after the other ones? Forget about them (for now)?
> Depending on the answer I'll reorganize the order of my patches to match so wait for rev4 before reading these patches.

We could consider them as an extra set of patches, if you separate them nicely, it would be a nice addition, but be careful to not over complicate what is now.
Camarade_Tux (reporter)
2013-02-18 23:18

Thanks for the comments.

I'll reorganize the commits so that the patches that could benefit cross-compilation of the compiler appear last.
I'll need a few more days to do that. Probably this week-end.

Btw, for TOOLPREF, it's not specific to anything. It's either arbitrary or already somewhere in the ocaml sources.
meyer (developer)
2013-02-18 23:52

Absolutely no rush, I'm going to look at it again in few days time - I have a fully booked week too.
william (reporter)
2013-02-19 11:42

about Camarade_Tux comment from 2013-02-17 22:57, regarding ocamldoc and camlp4 :

There are two categories of executables :

the ones that need to be cross compiled :
ocamlc ocamlrun ocamlcp ocamldep ocamlmklib ocamlmktop ocamlopt ocamlprof

the ones that do not need to be cross compiled :
camlp4prof camlp4boot camlp4 camlp4oof camlp4of camlp4o camlp4rf camlp4r camlp4orf ocamldoc ocamllex ocamlyacc

For all of them, it is important to be able to have them "prefixed", like
i686-w64-mingw32-ocamlc, even if this is for a program such as ocamldoc that is not really cross compiled. Because when building other libraries and programs, it is easier to use prefixed tools no matter if it is ocamldoc or ocamlc.

So would it be possible to have a build system that preserves building of those tools ?
Camarade_Tux (reporter)
2013-02-19 19:55

Well, it's difficult to have them build during the same build.

I believe it should be a requirement that if you want to build a cross ocaml X.Y.Z, you need to have a native ocaml X.Y.Z on your system.
The reason is fairly simple: there's no guarantee that it'll work otherwise. You might try with a different version of the compiler but you'll need a native ocaml compiler that is still fairly close version-wise. (plus you need the native compiler for camlp4 and ocamlbuild to work along with a working Dynlink)

That means that in order to build the cross-compiler, you will have to have a matching native toolchain. In that case you will be able to reuse most of it to get the right tools.

If you need to have specific names for these tools too, simply 'cp' or 'ln -s' them. They will exist for sure during build: they're really needed.

It might be doable to have them built as native tools during the build of the cross-compiler but it's really much more work. That will basically require that it is possible to cross-compile a native compiler. For now, 'cp' or 'ln -s' are very good ways to do it and actually the configure of the cross-compiler will fail if the version of the native compiler and the version of the cross-compiler you're building don't match.
meyer (developer)
2013-02-19 20:09

Dear William and Adrien,

Maybe a small chime in: building a cross-compiler does not preclude requirement of not having the compiler on the path. It does however simplify a lot the build process.

In general the problem could be solved by a separate recursive step for Makefile, so in normal world OCaml is built like this:

- bootstrap compiler builds bytecode compiler
- bytecode compiler builds bytecode compiler and the native one
- native compiler builds a native compiler compiled natively

in the cross-compilation world additional recursive invokation of Makefile would happen with all the variables set up to met the requirement building the cross-compiler.

In practice, it's not that simple however and I don't think it's needed at a cost of making it more complex. So I'd assume we already have the native toolchain already in place.
Camarade_Tux (reporter)
2013-02-20 07:47

I forgot to mention that both gfortran and gnat have the same requirement for cross-compilers and that's why I stopped trying to not require a native toolchain.
william (reporter)
2013-02-20 22:18

Indeed you are right.
In mxe, I ended up building first native ocaml then cross ocaml, copying native version of ocamldoc and camlp4* with right prefix.
Do not make things more complex, it is fine like that.
meyer (developer)
2013-02-23 17:27
edited on: 2013-02-23 17:29

I tried your patches with arm toolchain on x86:

$ ./configure --target arm-linux-gnueabi 
-e Configuring for host i686-pc-linux-gnu ...
-e Configuring for target arm-linux-gnueabi ...
-e Using compiler arm-linux-gnueabi-gcc.
./tst: 1: Syntax error: word unexpected (expecting ")")
-e Unable to compile the test program.
-e This failure is expected for cross-compilation: we will assume the C compiler is ANSI-compliant.

-e Checking the sizes of integers and pointers...
./tst: 1: Syntax error: word unexpected (expecting ")")
-e Unable to compile the test program.
 Make sure the C compiler 'arm-linux-gnueabi-gcc -O ' is properly installed.

2 things:
- tag prefix breaks lines - I included this comment in the summary
- the configure script continues running even though it failed - this was included in the previous-previous comments: exit will return from the shell function not from the script.

Also I would suggest testing on both on mingw and any other popular architecture (ARM for instance).

william (reporter)
2013-02-24 00:52

could you please explain how to patch sources with cross-patches-rev2-2013-02-17.tar.gz ?
I know the command :
patch -p1 < /path/to/patch
but how to use it for 30 patches ? I tried using "find /path/to/patch-dir -exec patch -p1 < {} \;", but does not work
meyer (developer)
2013-02-24 01:41

if you are sitting in the git repo, using git svn, then this command just works:

git am *.patch

if not than:

for f in *.patch; do patch -p1 < $f; done

(not that < {} will not work as it will splice two halves to redirection)
william (reporter)
2013-02-24 11:44

based on the git directory :
git clone [^]
I do :
git am ../cross-patches-rev2-2013-02-17/*.patch

and get :
Applying: byterun/win32.c: use < > to #include flexdll.h instead of "".
Applying: windows: don't define lseeki64 and lseek since it already exists.
/home/performance/src/mxe/gits/ocaml/.git/rebase-apply/patch:16: space before tab in indent.
     debugger.o meta.o dynlink.o
warning: 1 line adds whitespace errors.
Applying: yacc: "ocamlyacc$(EXE)" rule produced "ocamlyacc" (no trailing $(EXE)).
Applying: .gitignore: ignore vim temp files and .cm* stuff.
Applying: STASH
error: patch failed: build/
error: build/ patch does not apply
Patch failed at 0037 STASH

same problem with git version of 17/02/2013
Camarade_Tux (reporter)
2013-02-24 12:49
edited on: 2013-02-24 12:50

Here's a new version of the patches. It's against HEAD~ (damn people commiting on sundays!) but with the patch in 0005887. The file is "cross-patches-rev4-2013-02-24.tar.gz".

I've moved commits around and I've squashed some of them together and got 31 commits (down from 40 or so).

There are three directories inside the archive:
* 01-non-specific/ : 10 commits
* 02-safe/ : 8 commits
* 03-wip/ : 13 commits

Their names should be fairly explicit. The 03-wip directory is included for completeness but is not mergeable.

If moved the error messages from "echo -e" to "printf '%b\n'" since echo isn't portable: [^]
  "It is not possible to use echo portably across all POSIX systems unless both -n (as the first argument) and escape sequences are omitted."

The fact that there is a newline after "WARNING:" and "ERROR!" is on purpose: I've found it to look nicer and be easier to stop. Actually, I've added a new line before the tag and another one after the error message.

I've improved some messages. In particular, the error you had with the arm toolchain is because the patches have a hardcoded list of integers/pointers/shorts/longs sizes when cross-compiling since they cannot be guessed. Currently, only windows 32bit and 64bit are there. I prefer to leave someone else add the entries for other toolchains.

I've documented -target in the INSTALL file.

The TOOLPREF/TOOLCHAIN variables are mostly arbitrary: they appear like that in some other projects but there is absolutely no rule about them. I haven't changed their name in these patches but there is no reason to not do it if it is wished.

I've tried the configure script with ash too (instead of bash invoked as sh) and had no issue.

About the use of the "ws2_32" library, the 32 is both for 32bit and 64bit. Windows 64 provides the "win32" API. It's a name, they could also have named it "windows-api-after-dos" and it's not going away nor changing in incompatible ways any time soon.

I've replaced the invocation of "ocamlrun -version" with "ocamlc -version":
  % ocamlrun -version
      The Objective Caml runtime, version 3.12.1
  % ocamlc -version

That made it possible to replace sed with cut (in order to strip the +dev... part in the version).

For patch 15, you said:
    if test x"$target" != x"$host"; then
  we should be using ==.
I don't understand: what is the issue?

For patch 23, you said:
  Don't put comma after `if' condition. Otherwise applied.

I've replaced:
  ifeq ($(var1), $(var2))
  ifeq "$(var1)" "$(var2)"

Is it what you had in mind?

Camarade_Tux (reporter)
2013-02-24 13:00

Actually I've borked the change in byterun/Makefile.

I wrote:
  ifeq $(SYSTEM) mingw
but it should have been:
  ifeq "$(SYSTEM)" "mingw"

The corresponding file is 02-safe/0014-build-select-win32-variants-of-unix-and-graph-for-mi.patch .
meyer (developer)
2013-02-25 03:17

Adrien, thanks for the next round of patches.

My answers for your questions:

> For patch 23, you said:
> Don't put comma after `if' condition. Otherwise applied.

My bad typo, I actually meant: don't put space after the comma. The space after comma is significant. Correct syntax is:
  ifeq ($(SYSTEM),mingw)

I'd not use cut, sed seems to be better tool for that, but you are right if we want just base version than probably the regex you submitted is OK.
meyer (developer)
2013-02-25 04:05


I tested and committed first 5 patches from 01-non-specific/.

Please excuse doing that piece by piece, but I am also reviewing them during the tour. The next commits will contain the rest of the patches from 01-non-specific.
meyer (developer)
2013-02-28 00:30
edited on: 2013-02-28 00:42


A small update:

Sorry, as usually, I got very limited means of time, to commit and test the patches, but in essence I really like what you did with sub-dividing them. I will do my best to commit the rest from 01-non-specific/ during the week so we can move on with the rest of the patches. I will look if the compilation actually worked on anything else than mingw, like ARM.

meyer (developer)
2013-03-08 23:02
edited on: 2013-03-08 23:35

Patch 6/

inf() {
  printf "%b\n" "$*" 1>&3

wrn() {
  printf "\nWARNING:\n" 1>&3
  printf "%b\n\n" "$*" 1>&3

err() {
  printf "\nERROR!\n" 1>&3
  printf "%b\n\n" "$*" 1>&3
  exit 2

exec 3>&1

First you are writing to a custom descriptor 3, and then you will redirect everything from 3 to stdout (descriptor 1). What's the purpose?

Patch 7/
Modified patch looks good:

-if echo $configure_options | grep -q ' --\?[a-zA-Z0-9-]\+='; then
+if echo "$configure_options" | grep -qe '--\?[a-zA-Z0-9-]\+='; then

two things:

- added missing quotes
- and -e flag to escape leading --, and dropped the space

Patch 8/
Not applied.

Sporious lines here:

+  amd64,macosx)   as='as -arch x86_64'
+                  aspp='gcc -arch x86_64 -c';;

Camarade_Tux (reporter)
2013-06-26 23:16
edited on: 2013-06-26 23:31

Soooo. Back on this.

> First you are writing to a custom descriptor 3, and then you will redirect everything from 3 to stdout (descriptor 1). What's the purpose?

The reason is that the patch also does:
  - 4,4) echo "OK, this is a regular 32 bit architecture."
  + 4,4) inf "OK, this is a regular 32 bit architecture."
          echo "#undef ARCH_SIXTYFOUR" >> m.h

If stdout was used, the output from inf would pollute the one from echo. I prefer to make the function safer by also avoiding redirecting to stderr directly.

I've checked the commit that was merged as rev 13312 and then reverted by Damien with rev 13456. It factored some code so that the right ranlib was called for some step.
I believe it is not needed since the RANLIB variable will be updated by another patch to include the $(TOOLPREF) prefix and "ar" doesn't need to have "$(TOOLPREF)".

I'll update patch 7 to reflect your comments

For patch 8, I don't have the spurious lines you mention. I'll continue after I have some sleep.
edit: ah, seems to be patch 9 and I see it

Camarade_Tux (reporter)
2013-06-27 20:43

The new archive "cross-patches-rev4-2013-06-27-non-specific.tar.gz" contains the first set of patches (01-non-specific in the previous archives) after being updated for the current trunk.

I've clarified the use of fd 3 for inf()/wrn()/err().
I've changed the output of these functions to something more compact (everything on a single line in most cases).
I've updated what was patch 7 and added quotes and removed the heading space.
The last patch doesn't have the spurious "amd64,macosx" lines.

I'll be waiting until these patches can be merged before continuing with the other ones.
meyer (developer)
2013-06-27 21:28
edited on: 2013-06-27 21:46

1- Looks fine. Thanks, applied.
2- I don't like the style of the comment, but I will work it out. Otherwise looks good. Thank you.
3- Good improvement, thank you. (but I used -e for the regexp flag)
4- +if test "$?" -ne 0; then
   Should we not say: "test "$ret" -ne 0" instead. So this is the patch I didn't apply.
5- Thanks, nice cleanup. We might however think what we can do about other possibilities that this patch does not cover. BTW:

-case "$arch,$model,$system" in
- amd64,*,macosx) if ./searchpath clang; then
+case "$arch,,$system" in
+ amd64,macosx) if ./searchpath clang; then

this diff looks wrong to me, you are matching empty string with double comma.
(I fixed that manually)

Camarade_Tux (reporter)
2013-06-27 22:50


> 4- +if test "$?" -ne 0; then
> Should we not say: "test "$ret" -ne 0" instead. So this is the patch I didn't apply.

$ret holds the stdout output of runtest. It's $? which holds the return code so that's $? that should be used here.
meyer (developer)
2013-06-27 23:05

Ah yes, my bad, sorry, I forgot I had exactly the same comment while ago.
meyer (developer)
2013-06-27 23:10

OK. Committed. Thanks.
Camarade_Tux (reporter)
2013-06-28 21:34

I've uploaded "cross-patches-rev4-2013-06-28-safe2.tar.gz" (please disregard the previous archive named similarly).

It should be fairly safe but it changes are getting more intrusive.
meyer (developer)
2013-06-30 16:09

Currently the Windows CI builds fail: [^]

awaiting fix.
Camarade_Tux (reporter)
2013-06-30 19:21

The faulty commit seems to be revision 13856. Not sure why, investigating.
Camarade_Tux (reporter)
2013-07-01 09:16

The faulty commit (not the one I mentionned iirc) has been reversed; the CI builds should come back. I also fixed the issue in the patch (some files hadn't been updated) but it relies on having a configuration triplet (or any ID for the machine) and I don't know yet what to use for msvc.

There was another issue that broke *BSD (at least with the bsd make). The patch "0001-build-avoid-an-unneeded-gnu-style-conditional-in-Mak.patch" fixes that.
Camarade_Tux (reporter)
2013-07-01 20:35

New upload: cross-patches-rev4-2013-07-01-safe3.tar.gz

It contains the fixed commit and the other two patches: Makefile (not Makefile.nt) for win32unix and making ocamldoc, debugger and ocamlbuild optional since they don't make much sense for cross-toolchains.

I've tested this on mingw64 32b and linux64. Also ran the testsuite which is OK for both.
meyer (developer)
2013-07-01 21:28

Just committed your patches. Let's wait for the testing. Two things to note:

- you commented out ocamlbuild.native build
- you have not included in the INSTALL the instructions for new configure flags

I've fixed this missing bits.
Camarade_Tux (reporter)
2013-07-01 22:56
edited on: 2013-07-01 23:12

So, first issues.

I broke one one of the patch when I created it: I lost half of it.
For this one, you need to revert 'Revert "configure: store $target and $host in the build configuration." as it breaks several builds.' which means applying the patch again.

The second issue requires a revert of 'build: make building ocamldoc, ocamlbuild and ocamldebug optional.'. It causes errors at install (which I had been silly and hadn't checked =/ ).
Unfortunately this one will be quite difficult to fix. I remember encountering similar issues as what we're seeing now (missing and not managing to fix them. I haven't understood how this part of the build system works.

I'll wait until this week-end to work on the patches again because the second issue will require time.

edit: I've uploaded "fix-build-failures-2013-07-01.tar.gz" which contains the patches matching what I've described above.

meyer (developer)
2013-07-01 23:50

OpenBSD still fails on CI, fixing that would be the highest priority at the moment.

Also please make sure when you attach patches that the build works on the clean and fresh checkout.

meyer (developer)
2013-07-02 22:27

All the tests now run correctly, please submit further cross compiler patches for the inclusion.
chambart (developer)
2013-07-10 01:53

On systems with the binfmt module, the kernel will try to run pxe files leading to an error with a return code of 2, which breaks the configurations. So running the config tests must be avoided.

I did a quick patch to do that, there may be better ways
william (reporter)
2013-07-24 15:00

Your work seems great.
Does ocaml version in current trunk (for version 4.02 I guess) should work "out of the box" or should there be a set of patch to also apply before cross-compiling ocaml ?
meyer (developer)
2013-07-24 18:26

The assumption here is that we we will have all the needed changes for the cross compilation already in place in the stock compiler, without requiring any additional patches.
meyer (developer)
2013-07-24 18:33

chambart, your patch is being reviewed, and most likely will land on the trunk.

Thanks for writing up this patch!
Camarade_Tux (reporter)
2013-07-27 23:36

I've uploaded "cross-patches-2013-07-27-configure-to-disable-ocamldebug-ocamlbuild-ocamldoc.tar.gz".

It's a preview: I've done some last minute changes which need to be tested and I need to do more testing on windows too.

However, except for last-minute bugs, this should be good so I'm putting it up for review. I've split the patch that broke builds into two pieces: first some cleanup for variable names and then making it possible to disable/enable ocamlbuild and ocamldoc.

I'm going to continue my testing during the night and on tomorrow.
meyer (developer)
2013-07-28 00:04

thanks for the patches.

I few comments:

* this change:

-opt-core: runtimeopt ocamlopt libraryopt
-opt: opt-core otherlibrariesopt ocamlbuildlib.native
+	$(MAKE) runtimeopt
+	$(MAKE) ocamlopt
+	$(MAKE) libraryopt
+	$(MAKE) runtimeopt
+	$(MAKE) ocamlopt
+	$(MAKE) libraryopt
+	$(MAKE) otherlibrariesopt
+	$(MAKE) ocamltoolsopt
+	if test -n "$(WITH_OCAMLBUILD)"; then $(MAKE) ocamlbuildlib.native; else :; fi

I believe it breaks paralisation, what we want to do is to wrap every recursive make invokation to a specific rule and then list this rules like it was before.

* In the INSTALL file you have pointed out the space and time efficiency of the build with each component disabled. While it's useful in general, I don't think there is a place in the INSTALL file for this information, the efficiency vary between machines and space taken is different among architectures and systems too. So I excluded these changes.
meyer (developer)
2013-07-28 00:06

Actually the rules changes are fine, I misread, sorry.
meyer (developer)
2013-07-28 00:20

I've tested it, and it seems to work all right.

Apart from:

./build/ 23: .: Can't open config/
make: *** [install] Error 2

During the "make install" step, could you look at it?
william (reporter)
2013-07-28 14:01

Camarade_Tux :
I see a patch to disable "ocamldoc, ocamlbuild, ocamldebug".
other packages need to be disabled : camlp4*, ocamllex, ocamlyacc, if not already done.
Camarade_Tux (reporter)
2013-07-28 14:06

I've started testing these patches on windows and I have an error which is a bit weird and that I didn't have before rebasing my changes on trunk (after Damien's changes). Looking at it.

William, camlp4 is already something that can be disabled. As for ocamllex and ocamlyacc, I haven't found that it was needed. I'll keep it in mind but for now I prefer to have as few changes as possible: this change is already quite difficult to get right.
meyer (developer)
2013-07-28 14:08

small note, ocamllex and ocamlyacc can't be disabled as are used by ocaml itself.
Camarade_Tux (reporter)
2013-07-28 17:57
edited on: 2013-07-28 17:58

I've uploaded an additional commit which will fix builds on Windows:


Basically, $(MAKE) doesn't include the -f flag that is used on Windows.

Camarade_Tux (reporter)
2013-07-28 19:30

Windows builds broke, looping on the same make rule forever. I made a stupid typo in my last patch.

0001-build-typo-that-recursively-called-the-wrong-make-ru.patch should fix it. Sorry for the wrong commit.
gasche (developer)
2013-07-28 21:09

I just committed that last patch. Thanks for the work.
Camarade_Tux (reporter)
2013-07-28 22:55

CI found some more issues and I found that ocamldoc wasn't installed when reading the logs.

0001-build-fix-build-on-Windows.patch fixes everything I noticed and it re-enables builds with mingw+cygwin for both "make world.opt" (which I had used until now) and "make world bootstrap opt opt.opt install" (which the CI uses).

Sorry for the incovenience of broking the builds.
gasche (developer)
2013-07-28 23:09

I committed this last patch. Could we have a comment in the Makefile for the "testsuite" exception, or maybe add a suitably-empty Makefile.nt in the testsuite for homogeneity? An unexplained difference is a maintenance bug waiting to happen.
william (reporter)
2013-07-28 23:55

I used to build ocaml cross-compiler using a set of patches, that were reviewed in this bug report, and with a set of associated commands.

Would it be possible please to tell me what still needs to be done to build the cross-compiler ? Any advice will be precious to me, even if it is very high level : [^]

steps :
make core : still needed ?
make coreboot : still needed ?
make all : ?
source patching : I guess not required anymore
copy of m-nt, s-nt and Makefile : usefull? if yes, what is the purpose of the ./configure before ?
make world ?

I started to try the last ocaml version, but I am still at the ./configure phase :-)
meyer (developer)
2013-07-29 00:17

hello william,

So first you build a normal compiler, like usually:

./configure --prefix $PWD --no-camlp4 --no-ocamlbuild --no-ocamldoc --no-tk
make world.opt -j2
make install
export PATH=$PWD/bin:$PATH
make clean

then you build a cross compiler, for instance for ARM:

./configure --prefix $PWD/arm-compiler --target armv7-linux-gnueabi  --no-camlp4 --no-ocamlbuild --no-ocamldoc 
make world.opt -j2 # this will still fail
make install

hope that helps,
william (reporter)
2013-07-29 08:57

many thanks for this precisions. do you think there is no need for manually configuring config/m.h, config/s.h, and config/Makefile for windows target, using preconfigured config/m-nt.h and config/s-nt.h ?

I heard some time ago that those files were here because "./configure" was not able to manage all possible configurations, especially for windows.
william (reporter)
2013-07-30 00:32

compilation fails at win32unix : any clues ? [^]
meyer (developer)
2013-07-30 00:57

No, you don't meant to edit anything, the cross compilation patches will take care of everything. You'd have to ask Adrien about this failure.
Camarade_Tux (reporter)
2013-07-30 21:02
edited on: 2013-07-30 21:07

Hi William.

Currently this is really a work in-progress. The remaining patches have to be updated against trunk and I doubt any of them will get merged as-is.

The original patches could not be integrated; the need to copy files by hand after running "make core" instead of the regular "make world" was one typical issue. Currently trunk allows disabling ocamlbuild, camlp4, ocamldoc and the debugger and this should be enough to be able to stick with the standard build procedure.

However, keep in mind that everything hasn't been merged yet and that trunk cannot get you a cross-compiler yet (especially for windows, and especially if your host and target have different bitness).

Since all existing patches are completely unusable with today's trunk, I believe it is better to stick to any combination of release, patch and build procedure that you are used to.

To give a rough idea of what's left:
- use the ocaml compiler that is on the system to build the cross-compiler; find the right one in ./configure
- this requires removing ocamlcomp*.sh which was introduced by Jacques Garrigue to speed up compilation (is it still useful nowadays? by how much? could it be done otherwise)
- store integeres for asmcomp as Int64.t (rather than int, which are too short for cross-builds from 32b to 64b and too short the other way round)
- several smaller things

As for using ./configure, I haven't checked that the m.h, s.h and config/Makefile files that are generated by it cover at least as many features as the premade -nt/.mingw* variants. It's something definitely worth doing however!

edit: I've just checked your build log: you seem to use a toolchain from and it is not supported anymore by the config/Makefile.mingw{,64} files. Considering this, I didn't even try to support it.

Camarade_Tux (reporter)
2013-08-26 22:49

Here's another patch: 0001-build-replace-build-mk-config-myocamlbuild_config-.s.patch

I've had to remove and since they prevented me from doing other changes and I couldn't change them. Plus the approach wasn't the best.

The next big change will be the removal of and the use of something more flexible which will make it possible to use the system compilers.
william (reporter)
2013-08-29 00:47

Please find in cross-build_20130829.tar.gz the actual state of Makefile and patches that I use with success to do cross-compilation. It takes into accound some of the improvements made so far, and so work with a quite recent trunk version of ocaml.

Could you please have a quick look at it, and give me hints on what I could do to simplify things further ?
meyer (developer)
2013-08-31 01:57

Your patch is very welcome.

Few remarks from Damien:

- we should not use \0 in sed pattern matches, that does not work on Mac OS X.
- we should not be using [] in tr patterns.
- $() notation for splicing shell commands might not work on non Bash
- make -j3 is probably a left over

otherwise looks good!

and comments from me:

- --no-print-directory might not work on BSD make

I attached modified patch, with exception for fix of --no-print-directory flag, has that been tested on BSD system?
Camarade_Tux (reporter)
2013-08-31 10:30

Thanks for the feedback.

> - we should not use \0 in sed pattern matches, that does not work on Mac OS X.
Very stupid mistake from me; I'm sure I had the same issue a bit earlier.

> - we should not be using [] in tr patterns.
Actually, that's fine: [a-z] [A-Z] will replace [ with [, a-z with A-Z and ] with ]. The use of brackets is from System V while BSD didn't require them.
I wrote the commands with brackets and they looked weird; I find that using [:lower:] [:upper:] looks better (more readable)

> $() notation for splicing shell commands might not work on non Bash
I've spent quite a lot of time trying to understand that one. SUSv2 requires $() and advises its use instead of backquotes.
I'm under the impression that it's csh and tcsh which don't handle $() while the bourne shells do. (t)csh would never be used as /bin/sh so it should be safe (if someone uses them as /bin/sh, he deserves the pain).
Are there shells that we know don't handle $() ? It's something I've been wondering for years.

> - make -j3 is probably a left over
definitely, because of ...

> - --no-print-directory might not work on BSD make
it's a very late change after I've noticed that make's output of "entering/leaving directory ..." would get mixed with the and files. A fairly stupid change on my side which would have broken with BSD make.
I'm going to try to understand what's going on properly.
meyer (developer)
2013-08-31 15:00

> t's a very late change after I've noticed that make's output of >"entering/leaving directory ..." would get mixed with the and
> files. A fairly stupid change on my side which would
> have broken with BSD make.

So the most portable way to fix it would be to pattern match against the output you expect using grep, to filter out the messages from the Makefiles.
Camarade_Tux (reporter)
2013-08-31 17:34

I've uploaded 0001-build-replace-build-mk-config-myocamlbuild_config-.s-2.patch

It uses & instead of \0 for sed. It uses [:lower:] and [:upper:] instead of [a-z] and [A-Z] for tr. It keeps $(). --no-print-directory has been removed and instead the MAKEFLAGS and MAKELEVEL variables are unset in the shell script.

This has been tested on Linux and FreeBSD. Windows should be like Linux since it's built with GNU userland too.
protz (manager)
2013-08-31 17:46

I maintain the OCaml installer for Windows. If you want me to try one (or several) of your patches, let me know which ones, and I'll give it a try on Monday :).
Camarade_Tux (reporter)
2013-09-01 10:19

I'm fairly certain this particular change won't break on Windows (I have a VM available but I've avoided starting it).

That said, more testing is definitely welcome and even more for the windows installer. I don't have the next patches ready in advance since that would mean updating them for trunk very often and repeatedly but testing the current trunk (and with the mkconfig/myocamlbuild_config patch if possible) would already be great. Thanks.
Camarade_Tux (reporter)
2013-09-01 10:32

William, it's fairly difficult to tell you which patch would still be needed and even more because you have patches that only apply to you/mxe. Instead I can tell you which patches I still have to get integrated.

Before I forget about it, there's the bug 5887.
And, in random order:
- asmcomp-internals-store-integers-as-Int64.patch
- chose which ocamldoc to use (so that the system one is used)
- remove the runocamldoc script which is replaced by something else
- reove which is replaced by somethingn else more flexible
- chose the ocamlc/ocamlopt/ocamlrun commands to use: use either the ones from the system (to build the cross-compiler) or the cross-compiling ones (to build the libraries for the cross-compiler)
Camarade_Tux (reporter)
2013-09-07 22:20
edited on: 2013-09-07 22:26

New patches. :)

Archive file is cross-patches-2013-09-07.tar.gz .

The archive contains a new version of the last patch: on Windows, the ml code from the config makefiles has to be at the end of the file because it refers to other variables.
- build: replace build/mk{config,myocamlbuild_config}.sh.

Also, something had to break on Windows for stupid reasons: an unused variable "DO" in the config makefiles, turned into "let do = ..." in OCaml, which gives a syntax error. Well, frustrating and fixed:
- config/Makefile.{mingw,msvc}{,64}: remove unused DO variable.

I've also been running full tests on Windows, FreeBSD and Linux: configure, build, make, install, testsuite.

I found a few issues, notably that ocamldoc.opt was not built anymore on Windows:
- build: ocamldoc.opt was not built anymore on Windows.

The new big patch is "build: replace ocamlcomp*.sh.".
It's a big one which touches lots of files. "" had limitations and hadn't been used everywhere. My patch provides something more flexible and updates chunks of code that weren't using "".

This patch requires a new boot/myocamlbuild.boot file. It's described in the commit message but I've also added a specific patch so that the myocamlbuild.boot update is not forgotten.
- Regenerate a new boot/myocamlbuild.boot by copying _build/myocamlbuild.

Then, 3 patches make it possible to chose the ocamldoc command to run during the build since for cross-compilation, ocamldoc won't be built and we'll have to use the one from the system.
- ocamldoc/Makefile: remove unneeded OCAMLDOC and OCAMLDOC_OPT variables.
- ocamldoc: in configure, chose the "ocamldoc" command to use.
- ocamldoc/runocamldoc: remove it; it's unused now.

Finally, a cosmetic patch:
- byterun/Makefile: remove unneccessary -DCAML_NAME_SPACE in compile flags
The functions in the corresponding files use the "caml_"-prefixed versions, making it useless to -DCAML_NAME_SPACE which purpose, afaiu, is to make it possible to call "alloc()" instead of "caml_alloc()".

edit: this leaves only very few changes left: 0005887, using int64 instead of nativeint in asmcomp, introduce variables to distinguish between host and target compilers, and make sure we use the right commands everywhere. (and a couple more misc changes)

Camarade_Tux (reporter)
2013-09-14 14:13

I've pushed cross-patches-2013-09-07-fixed.tar.gz .

- rebased against current trunk (post-labltk removal)
- fix for build/ not installing where the user asked to (save $BINDIR and friends before sourcing and restore them afterwards if they're not empty)

Discard the previous tarball and the two previous patches: this new tarball includes everything.
Camarade_Tux (reporter)
2013-09-20 22:17

The jenkins build bots have uncovered three issues with the most recent patches.

- "make clean" fails until ./configure has been run again
- openbsd's make doesn't have -C (but freebsd has btw)
- an issue with improper quoting of variables that is apparent on Windows with the "C:\Program Files (x86)\" path: at least parens get interpreted

I've uploaded fixes for the first two issues. This leaves the last one which I'll tackle on tomorrow.

The patches have been tested on gnu/linux with make and with pmake (which is the parent to the bsd makes) and also on openbsd. The two patches are available as:


Sorry for the breakage.
Camarade_Tux (reporter)
2013-09-21 11:47

I've uploaded a new patch to fix the issue on Windows:

I have only tried it on my machine but it doesn't introduce anything new: it simply adds a variable to a blacklist and matches better the blacklist of build/mk{config,myocamlbuild_config}.sh.
meyer (developer)
2013-09-21 12:25

Thank you applied.
Camarade_Tux (reporter)
2013-09-23 00:11

I've uploaded two patches:


They fix the current issues noticed on the jenkins build bots.

Basically, the first one adds a call to "cygpath -m" because ocamlmklib is a Win32 executable and requires Win32 paths.

The second one fixes a change I had done in config/Makefile.mingw but skipped in config/Makefile.{mingw64,msvc,msvc64}.
garrigue (manager)
2013-10-01 01:46

Small bug in trunk: ocamlbuild uses ocamlc.opt even if it isn't the newest compiler.
I.e. the logic to switch between the bytecode and native versions of ocaml doesn't apply to ocamlbuild.
It had me scratching my head for a few hours while fixing a bug in type inference that occurred only in ocamlbuild.
Camarade_Tux (reporter)
2013-10-01 09:40


I'm really sorry for this issue. There is indeed clearly an issue in I'll have it a go at it this evening.
Camarade_Tux (reporter)
2013-10-01 22:36

I spent some time on the issue and something doesn't feel right in

I can't understand why the "native_deps" list in the code below refers to .cmxa, .cmx and .o files. Unless I'm mistaken, these files won't be used during the compilation or link process.
Instead the lists should only contain "ocamlc(.opt)" and .cma and .cmo files.

The corresponding code:
  let ocamlc_solver =
    let native_deps = ["../ocamlc.opt"; "../stdlib/stdlib.cmxa";
                      "../stdlib/std_exit.cmx"; "../stdlib/std_exit"-.-C.o] in
    let byte_deps = ["../ocamlc"; "../stdlib/stdlib.cma";
                    "../stdlib/std_exit.cmo"] in
    if Pathname.exists "../" then S[A"../"] else
    if List.for_all Pathname.exists native_deps then
      S[A"./ocamlc.opt"; A"-nostdlib"]
    else if List.for_all Pathname.exists byte_deps then
      S[ocamlrun; A"./ocamlc"; A"-nostdlib"]
meyer (developer)
2013-10-01 23:12


at the moment we should not care too much about as we don't use it normally, and these scripts are planned to be removed in the future.

I'm committing your patches at the moment.

Please note that there are bootstrapping problems at the moment on trunk, e.g. the ocamllex used to build fresh ocaml is being build with the new ocaml and not the bootstrapped version.
Camarade_Tux (reporter)
2013-10-01 23:18

Well, something has to be changed to fix the aforementionned issue. I believe the patch will be quite simple but I need to understand why using the opt-compiled bytecode compiler requires to have .cmx(a) files around.
It should simply be a matter of shelling out to call "test ocamlc.opt -nt ocamlc" (shelling out because Unix and therefore Unix.stat aren't available).

As for other patches, I'm not sure which ones you have in mind.
meyer (developer)
2013-10-02 01:59

currently these ones are pending:


I'm going to spend some time on it, over the week, probably I will get Damien to review them.
Camarade_Tux (reporter)
2013-10-02 08:37

Hmm, in this list, patches up to 6 (included) at least are already in.

That said, we should probably wait for one or two weeks for the current things to settle before continuing.
Camarade_Tux (reporter)
2013-10-03 08:53

I've uploaded one file:

Also, there is no need to consider previous patches in mantis; I will take care of reuploading updated versions of the ones that haven't been applied.

The patch fixes the issue reported above by Jacques and the second one fixes bootstrapping issues Alain Frisch has encountered.

I've checked that the .cmx/cmxa/o files that ocamlbuild was looking for were not required by chown root:root + chmod 640; the build process didn't fail unlike when I applied that to the .cmo file.
The only check is now native exists + native is more recent than byte-compiled.

PS: I really wish I could remove some of the files I've uploaded (found out they were broken right after uploading)
gasche (developer)
2013-10-03 12:40

Sorry for not having any time to look at the ocamlbuild issue.

Re. faulty uploads: just send me a precise list of stuff to remove, here or by email. I haven't followed the discussion recently but I can still click "Delete".

- Issue History
Date Modified Username Field Change
2012-08-24 00:26 william New Issue
2012-08-24 00:26 william File Added: 0001-various-patches-to-build-an-ocaml-cross-compiler.patch
2012-09-04 16:18 gasche Note Added: 0008020
2012-09-04 16:18 gasche Note Edited: 0008020 View Revisions
2012-09-04 16:20 gasche Status new => acknowledged
2012-09-04 22:46 william Note Added: 0008023
2012-12-26 22:23 Camarade_Tux File Added: 0001-config-Makefile.mingw64-remove-redundant-mms-bitfiel.patch
2012-12-26 22:23 Camarade_Tux File Added:
2012-12-26 22:23 Camarade_Tux File Added: 0003-configure-use-stderr-for-all-messages-targetted-at-t.patch
2012-12-26 22:23 Camarade_Tux File Added: 0004-build-fix-filename-case.patch
2012-12-26 22:23 Camarade_Tux File Added: 0005-build-prefer-forward-slashes-to-backward-slashes.patch
2012-12-26 22:24 Camarade_Tux File Added: 0006-configure-fix-detection-of-non-working-C-compiler.patch
2012-12-26 22:24 Camarade_Tux File Added:
2012-12-26 22:24 Camarade_Tux File Added:
2012-12-26 22:24 Camarade_Tux File Added: 0009-configure-add-target-and-use-target-instead-of-host-.patch
2012-12-26 22:24 Camarade_Tux File Added: 0010-configure-when-cross-compiling-set-TOOLPREF-and-use-.patch
2012-12-26 22:24 Camarade_Tux File Added: 0011-configure-support-cross-compilation-with-mingw.patch
2012-12-26 22:24 Camarade_Tux File Added: 0012-windows-makefiles-remove-unused-DO-variable.patch
2012-12-26 22:25 Camarade_Tux File Added: 0013-windows-makefiles-use-dll-for-the-SO-extension.patch
2012-12-26 22:32 Camarade_Tux Note Added: 0008649
2012-12-26 22:32 Camarade_Tux Note Edited: 0008649 View Revisions
2012-12-28 22:48 meyer Note Added: 0008660
2012-12-28 22:50 meyer Note Edited: 0008660 View Revisions
2012-12-29 13:26 Camarade_Tux Note Added: 0008665
2012-12-29 13:27 Camarade_Tux File Added: cross-patches-rev2-2012-12-29.tar.gz
2012-12-29 13:27 Camarade_Tux Note Edited: 0008665 View Revisions
2012-12-29 13:42 Camarade_Tux Note Edited: 0008665 View Revisions
2012-12-30 04:04 meyer Note Added: 0008671
2012-12-30 04:13 meyer Assigned To => meyer
2012-12-30 04:13 meyer Status acknowledged => assigned
2012-12-30 16:53 Camarade_Tux Note Added: 0008672
2013-01-23 22:25 Camarade_Tux File Added: patches_dump.tar.gz
2013-01-23 22:26 Camarade_Tux Note Added: 0008785
2013-01-29 02:05 meyer Note Added: 0008812
2013-02-17 22:48 Camarade_Tux File Added: cross-patches-rev2-2013-02-17.tar.gz
2013-02-17 22:57 Camarade_Tux Note Added: 0008853
2013-02-18 03:10 meyer Note Added: 0008855
2013-02-18 04:32 meyer Note Added: 0008856
2013-02-18 05:02 meyer Note Edited: 0008855 View Revisions
2013-02-18 23:18 Camarade_Tux Note Added: 0008862
2013-02-18 23:52 meyer Note Added: 0008863
2013-02-19 11:42 william Note Added: 0008866
2013-02-19 19:55 Camarade_Tux Note Added: 0008875
2013-02-19 20:09 meyer Note Added: 0008877
2013-02-20 07:47 Camarade_Tux Note Added: 0008881
2013-02-20 22:18 william Note Added: 0008887
2013-02-23 16:36 meyer Note Edited: 0008855 View Revisions
2013-02-23 16:38 meyer Note Edited: 0008855 View Revisions
2013-02-23 16:41 meyer Note Edited: 0008855 View Revisions
2013-02-23 17:03 meyer Note Edited: 0008855 View Revisions
2013-02-23 17:15 meyer Note Edited: 0008855 View Revisions
2013-02-23 17:15 meyer Note Edited: 0008855 View Revisions
2013-02-23 17:17 meyer Note Edited: 0008855 View Revisions
2013-02-23 17:27 meyer Note Added: 0008891
2013-02-23 17:29 meyer Note Edited: 0008891 View Revisions
2013-02-23 17:43 meyer Note Edited: 0008855 View Revisions
2013-02-24 00:52 william Note Added: 0008896
2013-02-24 01:41 meyer Note Added: 0008897
2013-02-24 11:44 william Note Added: 0008900
2013-02-24 12:49 Camarade_Tux Note Added: 0008901
2013-02-24 12:49 Camarade_Tux File Added: cross-patches-rev4-2013-02-24.tar.gz
2013-02-24 12:50 Camarade_Tux Note Edited: 0008901 View Revisions
2013-02-24 13:00 Camarade_Tux Note Added: 0008902
2013-02-25 03:17 meyer Note Added: 0008908
2013-02-25 04:05 meyer Note Added: 0008909
2013-02-28 00:30 meyer Note Added: 0008933
2013-02-28 00:40 meyer Note Edited: 0008933 View Revisions
2013-02-28 00:42 meyer Note Edited: 0008933 View Revisions
2013-03-08 23:02 meyer Note Added: 0008957
2013-03-08 23:03 meyer Note Edited: 0008957 View Revisions
2013-03-08 23:03 meyer Note Edited: 0008957 View Revisions
2013-03-08 23:11 meyer Note Edited: 0008957 View Revisions
2013-03-08 23:12 meyer Note Edited: 0008957 View Revisions
2013-03-08 23:33 meyer Note Edited: 0008957 View Revisions
2013-03-08 23:35 meyer Note Edited: 0008957 View Revisions
2013-06-26 23:16 Camarade_Tux Note Added: 0009618
2013-06-26 23:31 Camarade_Tux Note Edited: 0009618 View Revisions
2013-06-27 20:39 Camarade_Tux File Added: cross-patches-rev4-2013-06-27-non-specific.tar.gz
2013-06-27 20:43 Camarade_Tux Note Added: 0009622
2013-06-27 21:28 meyer Note Added: 0009623
2013-06-27 21:31 meyer Note Edited: 0009623 View Revisions
2013-06-27 21:46 meyer Note Edited: 0009623 View Revisions
2013-06-27 22:50 Camarade_Tux Note Added: 0009624
2013-06-27 23:05 meyer Note Added: 0009625
2013-06-27 23:10 meyer Note Added: 0009626
2013-06-28 21:07 Camarade_Tux File Added: cross-patches-rev4-2013-06-28-safe.tar.gz
2013-06-28 21:32 Camarade_Tux File Added: cross-patches-rev4-2013-06-28-safe2.tar.gz
2013-06-28 21:34 Camarade_Tux Note Added: 0009645
2013-06-29 09:34 gasche File Deleted: cross-patches-rev4-2013-06-28-safe.tar.gz
2013-06-30 16:09 meyer Note Added: 0009653
2013-06-30 19:21 Camarade_Tux Note Added: 0009654
2013-07-01 09:14 Camarade_Tux File Added: 0001-build-avoid-an-unneeded-gnu-style-conditional-in-Mak.patch
2013-07-01 09:16 Camarade_Tux Note Added: 0009658
2013-07-01 20:28 Camarade_Tux File Added: cross-patches-rev4-2013-07-01-safe3.tar.gz
2013-07-01 20:35 Camarade_Tux Note Added: 0009664
2013-07-01 21:28 meyer Note Added: 0009666
2013-07-01 22:56 Camarade_Tux Note Added: 0009667
2013-07-01 23:11 Camarade_Tux File Added: fix-build-failures-2013-07-01.tar.gz
2013-07-01 23:12 Camarade_Tux Note Edited: 0009667 View Revisions
2013-07-01 23:50 meyer Note Added: 0009668
2013-07-02 22:27 meyer Note Added: 0009680
2013-07-10 01:53 chambart Note Added: 0009740
2013-07-10 01:54 chambart File Added: 0001-Do-not-run-tests-when-cross-compiling.patch
2013-07-24 15:00 william Note Added: 0009850
2013-07-24 18:26 meyer Note Added: 0009858
2013-07-24 18:33 meyer Note Added: 0009859
2013-07-27 23:31 Camarade_Tux File Added: cross-patches-2013-07-27-configure-to-disable-ocamldebug-ocamlbuild-ocamldoc.tar.gz
2013-07-27 23:36 Camarade_Tux Note Added: 0009903
2013-07-28 00:04 meyer Note Added: 0009904
2013-07-28 00:06 meyer Note Added: 0009905
2013-07-28 00:20 meyer Note Added: 0009906
2013-07-28 14:01 william Note Added: 0009929
2013-07-28 14:06 Camarade_Tux Note Added: 0009930
2013-07-28 14:08 meyer Note Added: 0009931
2013-07-28 17:56 Camarade_Tux File Added: 0001-build-pass-f-Makefile.nt-to-sub-make-processes-of-Ma.patch
2013-07-28 17:57 Camarade_Tux Note Added: 0009939
2013-07-28 17:58 Camarade_Tux Note Edited: 0009939 View Revisions
2013-07-28 19:29 Camarade_Tux File Added: 0001-build-typo-that-recursively-called-the-wrong-make-ru.patch
2013-07-28 19:30 Camarade_Tux Note Added: 0009940
2013-07-28 21:09 gasche Note Added: 0009941
2013-07-28 22:53 Camarade_Tux File Added: 0001-build-fix-build-on-Windows.patch
2013-07-28 22:55 Camarade_Tux Note Added: 0009948
2013-07-28 23:09 gasche Note Added: 0009949
2013-07-28 23:55 william Note Added: 0009955
2013-07-29 00:17 meyer Note Added: 0009956
2013-07-29 08:57 william Note Added: 0009968
2013-07-30 00:32 william Note Added: 0010002
2013-07-30 00:57 meyer Note Added: 0010004
2013-07-30 21:02 Camarade_Tux Note Added: 0010021
2013-07-30 21:07 Camarade_Tux Note Edited: 0010021 View Revisions
2013-08-26 22:46 Camarade_Tux File Added: 0001-build-replace-build-mk-config-myocamlbuild_config-.s.patch
2013-08-26 22:49 Camarade_Tux Note Added: 0010241
2013-08-29 00:47 william Note Added: 0010256
2013-08-29 00:48 william File Added: cross-build_20130829.tar.gz
2013-08-31 01:57 meyer Note Added: 0010280
2013-08-31 02:26 meyer File Added: 0001-build-replace-build-mk-config-myocamlbuild_config-.s.modified.patch
2013-08-31 10:30 Camarade_Tux Note Added: 0010282
2013-08-31 15:00 meyer Note Added: 0010284
2013-08-31 17:30 Camarade_Tux File Added: 0001-build-replace-build-mk-config-myocamlbuild_config-.s-2.patch
2013-08-31 17:34 Camarade_Tux Note Added: 0010285
2013-08-31 17:46 protz Note Added: 0010286
2013-09-01 10:19 Camarade_Tux Note Added: 0010290
2013-09-01 10:32 Camarade_Tux Note Added: 0010291
2013-09-07 22:03 Camarade_Tux File Added: cross-patches-2013-09-07.tar.gz
2013-09-07 22:20 Camarade_Tux Note Added: 0010324
2013-09-07 22:26 Camarade_Tux Note Edited: 0010324 View Revisions
2013-09-14 14:09 Camarade_Tux File Added: cross-patches-2013-09-07-fixed.tar.gz
2013-09-14 14:13 Camarade_Tux Note Added: 0010354
2013-09-20 22:15 Camarade_Tux File Added: 0001-build-fix-make-clean-failure-when-.-configure-hasn-t.patch
2013-09-20 22:15 Camarade_Tux File Added: 0002-build-don-t-use-make-s-C-it-s-not-available-everywhe.patch
2013-09-20 22:17 Camarade_Tux Note Added: 0010385
2013-09-21 11:44 Camarade_Tux File Added: 0001-build-skip-IFLEXDIR-in-mk_shell_and_ocamlbuild_confi.patch
2013-09-21 11:47 Camarade_Tux Note Added: 0010387
2013-09-21 12:25 meyer Note Added: 0010388
2013-09-23 00:08 Camarade_Tux File Added: 0001-build-ocamlmklib-on-Windows-expect-a-Windows-style-p.patch
2013-09-23 00:08 Camarade_Tux File Added: 0002-build-typo-on-config-Makefile.mingw64-msvc-and-msvc6.patch
2013-09-23 00:11 Camarade_Tux Note Added: 0010396
2013-10-01 01:46 garrigue Note Added: 0010409
2013-10-01 09:40 Camarade_Tux Note Added: 0010410
2013-10-01 22:36 Camarade_Tux Note Added: 0010414
2013-10-01 23:12 meyer Note Added: 0010415
2013-10-01 23:18 Camarade_Tux Note Added: 0010416
2013-10-02 01:59 meyer Note Added: 0010417
2013-10-02 08:37 Camarade_Tux Note Added: 0010418
2013-10-02 22:40 Camarade_Tux File Added:
2013-10-02 22:40 Camarade_Tux File Added: 0002-build-always-build-lex-and-tools-with-the-compiler-f.patch
2013-10-03 08:47 Camarade_Tux File Added: 0001-build-always-build-lex-tools-and-ocamlc-with-the-com-fixed.patch
2013-10-03 08:53 Camarade_Tux Note Added: 0010419
2013-10-03 12:40 gasche Note Added: 0010420
2013-12-16 11:46 doligez Tag Attached: patch
2014-01-21 13:58 doligez Assigned To meyer =>

Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker