Comment author: @dra27

Regarding the curious file reference, I agree we could improve that, but it would simply be to delete it and replace it with something like "Linking error".

For the other, there's far too much detail removed from the report to be able to diagnose the problem, but it doesn't (yet) look to me like a dependency analysis issue. Should ctl.cmxa contain the Pretty_print_ctl module? If not, has the order in which you produce the linking instructions earlier on changed? Given that you say that the link command given is the same in both build systems, I would first diagnose where in your working build system that module comes from and then look to why it's presumably not being built correctly in your updated one.

Comment author: @elfring

I am curious on how information can be improved further around the mentioned “Linking error”.

Which details do you miss from the described software situation?

I can imagine to some degree that my script adjustments should result in some effects for the handling of build dependencies. My knowledge is still evolving also for the programming language “OCaml”. So I find it unclear at the moment which differences could be really relevant in the module files which were generated according to the used build system.

Would you like to compare build results in your test environments once more?
https://github.com/elfring/Coccinelle-20160205/blob/replace_suffix_rules_by_pattern_rules1/ctl/Makefile#L58
https://github.com/coccinelle/coccinelle/blob/1c4474890c065711b16c7d02089a56da5a07e13a/ctl/Makefile#L57

The source files are available for the questionable module reference.
Is the mentioned error message really appropriate?

Comment author: @dra27

OCaml requires that the symbols be specified before they are used, so we could add a check in the error case which looks to see if the order of libraries is wrong, but I'm not sure what would change in the message? The source files are not available - there is no way to go from the Ctl_engine.cmx file packed in ctl/ctl.cmxa to the source file for it.

For me, at least, your repro case is too large - I'm not about to go and compile what I've seen described elsewhere as a "harsh test" (ocaml/opam#2944 (comment)) to investigate something which at present doesn't look to me like a bug.

You've shortened the /usr/bin/ocamlopt.opt call in your report above (all the ellipses) and I have suggested some other things to investigate in that command. If there's a bug, then we will certainly work to fix it, but I'm afraid this feels to me as though it should possibly still be on caml-list, etc.?

Comment author: @elfring

I assume that changes for safer identification of linking errors could reveal related software improvement opportunities.

The size of the affected software development repository can make the desired clarification of open issues harder than a discussion for a small test example.
There are general development challenges to consider when some software components are also bundled.

I shortened the affected link command intentionally. I would prefer to interpret it also more in terms of the used software build system.
This build system should ensure that the discussed command is only started after all prerequisites are up-to-date. This process seems to work to some degree in both published versions for the Coccinelle software.
So it can become a bigger puzzle to find the differences at other places where the OCaml linker can disagree with the provided references at the moment.

Comment author: @Octachron

Dear elfring, this issue tracker should be used to report issues, bugs, or feature wishes directly related to the ocaml project itself; not issues with your own personal projects or your understanding of the aforementioned tools.

All evidences at hand strongly suggests that the root cause of this issues stems from your side. Consequently, you will be far better served by other discussion media such that the caml mailing list, the #ocaml irc channel on freenode, or https://discuss.ocaml.org. May I also re-emphases that you may need to focus on specific questions rather than vague monologue?

Consequently, I am closing this issue for now. Please do not reopen it without conclusive evidence that you have isolated an ocaml issue, for instance a short reproducible case.

Comment author: @elfring

Specific questions:

• How would you like to adjust your evolving source code for the questionable information “File "none", line 1:”?
  Can this glitch be fixed anyhow?

• How safe is the message “No implementations provided for the following modules”?

Comment author: @elfring

Which kind of evidence will be conclusive enough for you?

Comment author: @dra27

I'd already suggested that removing the questionable reference to file none simply involves not displaying that line (it's just a result of a sentinel value leaking into a message). You seem to be putting a lot of weight into - it could always go in a separate issue (which may get picked up as a "junior_job" PR).

The other message is perfectly fine to us - you need to suggest what's wrong with it and how it might be changed, not us. I've already pointed out that source file information is a non-starter. You'd get exactly the same error from the C linker if you don't specify a library containing symbols you use.

I'm afraid shortening the link command has prevented the ability to give you any suggestion of what to do next. It sounds as though you have not actually fixed your own build system in your fork (I think that's correct?) - you need to do that first, which will enlighten you as to what is wrong or has changed. Only having done that will you know for certain whether you've found a bug in OCaml or its tools or, if you haven't found a bug, how the details of your story might allow us to work out how the messages given may be improved (but I must confess I'm doubtful of either of these things being the case).

Conclusive evidence here would be a full link command where you can show that one of the earlier objects in the link command provides the Pretty_print_ctl module, yet ocamlopt is claiming that it has not been provided.

Comment author: @elfring

dra:
Thanks that you can agree to reconsider the mentioned position information. I find this promising.
Would you prefer to handle corresponding software improvements under a new issue number?

I know already that there are some open issues remaining at various places in the Coccinelle software. The corresponding main developers are appropriately informed.
https://github.com/coccinelle/coccinelle/issues

Is it really easier for you to see the following data again?

elfring@Sonne:~/Projekte/Coccinelle/20160205> git branch && LANG=C make --no-builtin-rules V=1
…
  master
* replace_suffix_rules_by_pattern_rules1
make[1]: Entering directory '/home/elfring/Projekte/Coccinelle/20160205'
make[1]: '.depend' is up to date.
…
/usr/bin/ocamlopt.opt -unsafe -I /home/elfring/Projekte/Coccinelle/20160205/bundles/parmap/ -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pyml/ -I /usr/lib64/ocaml -I /usr/local/share/menhir -I commons -I commons/ocamlextra -I ctl -I engine -I extra -I globals -I ocaml -I parsing_c -I parsing_cocci -I popl09 -I python    -cclib -lpyml_stubs -ccopt -lpcre -cclib -L/home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -cclib -lpcre_stubs -cclib -lparmap_stubs -o spatch.opt str.cmxa unix.cmxa bigarray.cmxa nums.cmxa /usr/lib64/ocaml/dynlink.cmxa /home/elfring/Projekte/Coccinelle/20160205/bundles/pyml//pyml.cmxa /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre//pcre.cmxa /home/elfring/Projekte/Coccinelle/20160205/bundles/parmap//parmap.cmxa commons/commons.cmxa globals/globals.cmxa ctl/ctl.cmxa parsing_cocci/cocci_parser.cmxa parsing_c/parsing_c.cmxa ocaml/cocciocaml.cmxa python/coccipython.cmxa engine/cocciengine.cmxa popl09/popl.cmxa extra/extra.cmxa flag_cocci.cmx cocci.cmx testing.cmx read_options.cmx main.cmx
File "_none_", line 1:
Error: No implementations provided for the following modules:
         Pretty_print_ctl referenced from ctl/ctl.cmxa(Ctl_engine)
make[3]: *** [Makefile:227: spatch.opt] Error 2
make[3]: Leaving directory '/home/elfring/Projekte/Coccinelle/20160205'
make[2]: *** [Makefile:166: opt-compil] Error 2
make[2]: Leaving directory '/home/elfring/Projekte/Coccinelle/20160205'
make[1]: *** [Makefile:146: all-release] Error 2
make[1]: Leaving directory '/home/elfring/Projekte/Coccinelle/20160205'
make: *** [Makefile:123: all] Error 2
elfring@Sonne:~/Projekte/Coccinelle/20160205> find . -name 'pretty_print_ctl*'
./ctl/pretty_print_ctl.cmi
./ctl/pretty_print_ctl.o
./ctl/pretty_print_ctl.mli
./ctl/pretty_print_ctl.ml
./ctl/pretty_print_ctl.cmx
elfring@Sonne:~/Projekte/Coccinelle/20160205> cd ctl && ls -l ctl.* pretty_print_ctl*
-rw-r--r-- 1 elfring users 330198 26. Jun 13:41 ctl.a
-rw-r--r-- 1 elfring users   3034 26. Jun 13:41 ctl.cmxa
-rw-r--r-- 1 elfring users    495 26. Jun 13:41 pretty_print_ctl.cmi
-rw-r--r-- 1 elfring users    926 26. Jun 13:41 pretty_print_ctl.cmx
-rw-r--r-- 1 elfring users   4900 27. Okt 2016  pretty_print_ctl.ml
-rw-r--r-- 1 elfring users    363 12. Jun 14:53 pretty_print_ctl.mli
-rw-r--r-- 1 elfring users  26200 26. Jun 13:41 pretty_print_ctl.o

Which debug tools should I try out further to provide you another bit of evidence around questionable link references?

Comment author: @Octachron

This is more evidence that your build process is broken, nothing more. You can check with ocamlobjinfo that none of the included cmxa contain an implementation for the Pretty_print_ctl module, then you will need to fix your build process for (probably) "ctl/ctl.cmxa".

Comment author: @elfring

elfring@Sonne:~/Projekte/Coccinelle/20160205/ctl> /usr/bin/ocamlobjinfo ctl.cmxa|ack [Pp]retty_print_ctl
8f3b17e7b5889ac11a58d2c78105da77 Pretty_print_ctl
435cdfb6cd49228d7ab3f397c56cfef1 Pretty_print_ctl
Name: Pretty_print_ctl
Pretty_print_ctl
8f3b17e7b5889ac11a58d2c78105da77 Pretty_print_ctl

Comment author: @dra27

There's too much clipped from that ocamlobjinfo output to be able to diagnose anything (you've lost the headers and context for those lines, so you can't tell if they're module imports or exports; signatures or implementation)

However, as @Octachron said a few comments ago, this is still looking a bug in your build system, and this would be better on discuss.ocaml.org or caml-list. Please return to the bug-tracker when it is clearly a bug.

Comment author: @elfring

dra:

• Can the attached output of the command “/usr/bin/ocamlobjinfo ctl.cmxa > ctl_cmxa-ocamlobjinfo.txt” help for a better understanding of a questionable software build result?

• Can this debug tool extract also directly the information your are looking for in this use case?

Comment author: @dra27

Yes, it reveals that your build system has assembled ctl.cmxa incorrectly - are you sure that the command which built that is identical to the previous version? ctl_engine.cmx is clearly added to ctl.cmxa before pretty_print_ctl.cmx.

Comment author: @elfring

I can compare also the files “ctl_cmxa-working-ocamlobjinfo.txt” and “ctl_cmxa-questionable-ocamlobjinfo.txt” by the tool “kdiff3” for example.
I guess that my OCaml knowledge was insufficient so far to interpret the relevance of a different placement of data around names like “Flag_ctl” and “Pretty_print_ctl”.

I find it interesting which differences can be seen there for the published software build approaches. How significant should they be for resolving the needed references by the OCaml linker?

The following make specification

$(OCAMLC_CMD) -a -o $(LIB) $(OBJS)

was originally resolved to a command as follows.

/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -a -o ctl.cma flag_ctl.cmo ast_ctl.cmo pretty_print_ctl.cmo ctl_engine.cmo wrapper_ctl.cmo

I dared to adjust a variable like “OBJS” with the consequence that a command with a different listing of object files was generated.
elfring/Coccinelle-20160205@9597598#diff-ecee58f112af500f11322e78423ea2aeL12

/usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -a -o ctl.cma ast_ctl.cmo flag_ctl.cmo ctl_engine.cmo pretty_print_ctl.cmo wrapper_ctl.cmo

Now it seems that I need to distinguish the listing of source files better from their link order for this tool. I can add further make variables which specifiy the preferred (or at least working) combination.

Can the OCaml linking software automatically find itself out in which order the passed object files should be combined (or with the help of special parameters) optimally?

Comment author: @dra27

I have already linked this report with number 6778 which, if changed, would have emitted an error when you changed the compilation for ctl.cmxa.

In general, no, it's not possible for the OCaml linker to work out the order in which object files must be given. Modules can (and do) include toplevel initialisation statements which mean that changing the order in which they are linked can change the semantics of the resulting program.

There are various techniques (with ocamlfind, or more powerful build systems such as jbuilder or ocamlbuild, or code layout techniques which can allow ocamldep to help determine module link order), but Mantis is not the place for a discussion about them, as none of them are going to yield changes to the compiler itself.

The only non-duplicated issue from this report at the moment is the poor message referring to file none - please feel free to open a separate issue for that, as I don't think the discussion here provides a useful basis for someone picking that issue in future.

Comment author: @elfring

How do you think about the possibility to change the error information from “No implementations provided” to “Linking: References could not be resolved so far”?

I am still curious how affected software components will be improved also for this issue.