Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0004847OCamlOCaml generalpublic2009-08-18 17:432011-05-29 12:20
Reporterdaweil 
Assigned Tofrisch 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformWindows MSVC OSWindows XPOS VersionServica Pack3
Product Version3.11.1 
Target VersionFixed in Version3.11.2+dev 
Summary0004847: callback from C to CAML
Descriptionsmall example given in ocaml documentation at ยง18.8 works on ocaml3.10 and not on ocaml3.11.

What .obj or .lib is missing?

Could the Windows version of the example be described in the official Ocaml documentation?

PS : I use Microsoft Visual 2005
Steps To Reproduceunzip the attached file. type "make debug" or "make opt", observe the following message:

ocamlc -custom -output-obj -o modcaml.obj mod.ml
modcaml.c

ocamlc -c modwrap.c
modwrap.c

ocamlc -c main.c
main.c

cp C:/ProgramFiles/ocaml3.11.0/lib/libcamlrun.lib mod.lib
lib.exe mod.lib modcaml.obj modwrap.obj /out:modwrap.lib #E:/users/dwl/dev/R209/AUTdwlINFR209/AUTLciKernel/caml_src/flexdll-bin-0.19/*.obj
Microsoft (R) Library Manager Version 8.00.50727.762

Copyright (C) Microsoft Corporation. All rights reserved.



link.exe modwrap.lib main.obj /out:main.exe
Microsoft (R) Incremental Linker Version 8.00.50727.762

Copyright (C) Microsoft Corporation. All rights reserved.



modwrap.lib(win32.obj) : error LNK2019: unresolved external symbol _flexdll_dump_exports referenced in function _caml_dlopen

modwrap.lib(win32.obj) : error LNK2019: unresolved external symbol _flexdll_dlopen referenced in function _caml_dlopen

modwrap.lib(win32.obj) : error LNK2019: unresolved external symbol _flexdll_dlclose referenced in function _caml_dlclose

modwrap.lib(win32.obj) : error LNK2019: unresolved external symbol _flexdll_dlsym referenced in function _caml_dlsym

modwrap.lib(win32.obj) : error LNK2019: unresolved external symbol _flexdll_dlerror referenced in function _caml_dlerror

modwrap.lib(debugger.obj) : error LNK2019: unresolved external symbol _connect@12 referenced in function _open_connection

modwrap.lib(debugger.obj) : error LNK2019: unresolved external symbol _socket@12 referenced in function _open_connection

modwrap.lib(debugger.obj) : error LNK2019: unresolved external symbol _setsockopt@20 referenced in function _open_connection

modwrap.lib(debugger.obj) : error LNK2019: unresolved external symbol _getsockopt@20 referenced in function _open_connection

modwrap.lib(debugger.obj) : error LNK2019: unresolved external symbol _WSAStartup@8 referenced in function _winsock_startup

modwrap.lib(debugger.obj) : error LNK2019: unresolved external symbol _WSACleanup@0 referenced in function _winsock_cleanup

modwrap.lib(debugger.obj) : error LNK2019: unresolved external symbol _htons@4 referenced in function _caml_debugger_init

modwrap.lib(debugger.obj) : error LNK2019: unresolved external symbol _gethostbyname@4 referenced in function _caml_debugger_init

modwrap.lib(debugger.obj) : error LNK2019: unresolved external symbol _inet_addr@4 referenced in function _caml_debugger_init

main.exe : fatal error LNK1120: 14 unresolved externals
TagsNo tags attached.
Attached Fileszip file icon caml_from_c.zip [^] (1,528 bytes) 2009-08-18 17:43
zip file icon caml_from_c_main_with_flexdll.zip [^] (2,598 bytes) 2009-08-19 16:28

- Relationships

-  Notes
(0005050)
daweil (reporter)
2009-08-18 17:47
edited on: 2009-08-18 17:49

NB : I put the issue in the wrong category, it is not a feature request but a blocking issue for migrating our software from 3.10 to 3.11

(0005051)
frisch (developer)
2009-08-18 22:31

You'd better use flexlink to produce the main program. You also need to link
advapi32.lib and ws2_32.lib.

Here is a working Makefile:

include $(shell ocamlc -where)/Makefile.config

byte:
    ocamlc -output-obj -o modcaml.obj mod.ml
    ocamlc -c modwrap.c
    ocamlc -c main.c
    $(MKEXE) -o main.exe modwrap.obj main.obj modcaml.obj $(PREFIX)/lib/libcamlrun.lib $(BYTECCLIBS)

opt:
    ocamlopt -output-obj -o modcaml.obj mod.ml
    ocamlopt -c modwrap.c
    ocamlopt -c main.c
    $(MKEXE) -o main.exe modwrap.obj main.obj modcaml.obj $(PREFIX)/lib/libasmrun.lib $(NATIVECCLIBS)
(0005052)
daweil (reporter)
2009-08-19 09:25

Thanks for your fast answer. Now I want to do the next step that is required for our software : callback from C to CAML but but the CAML in a DLL.
(in ocaml3.10, I had to patch the caml sources to do it, but it seems that I should not need thispatch anymore in pcaml3.11)

So I added the following target in the makefile you suggest :
FLEXOPT= -show-exports -show-imports
bytedll:
    ocamlc -output-obj -o modcaml.obj mod.ml
    ocamlc -c modwrap.c
    ocamlc -c main.c
    $(MKDLL) $(FLEXOPT) -implib -o libmodwrap.dll modwrap.obj modcaml.obj $(PREFIX)/lib/libcamlrun.lib $(BYTECCLIBS)
    $(MKEXE) $(FLEXOPT) -o main.exe main.obj libmodwrap.lib ws2_32.lib advapi32.lib

It fails with the following message :
** Cannot resolve symbols for main.obj: _caml_startup _fib _format_result

What's wrong ? The symbols "_caml_startup _fib _format_result" appears to be exporteed when I use the -show_exports option of flexlink
(0005053)
frisch (developer)
2009-08-19 14:14

The symbols are exported in the sense of flexdll. Since the DLL will be loaded in the main program directly (not with flexdll), through its import library, you need real DLL exports. You can achieve that by passing /def:XXX flags to the linker, or with a .def file, or by adding declspec(dllexport) directives to modwrap.c. For the .def file solution, you can create a file modwrap.def with those lines:

EXPORTS
  caml_startup
  fib
  format_result

And then the Makefile rule would be:

bytedll:
    ocamlc -c modwrap.c
    ocamlc -cclib "-implib -link /def:modwrap.def" -output-obj -o libmodwrap.dll modwrap.obj mod.ml
    ocamlc -c main.c
    $(MKEXE) -o main.exe main.obj libmodwrap.lib

Note that I use here the new feature of OCaml that allows you to get a DLL directly (-output-obj X.dll).
(0005057)
daweil (reporter)
2009-08-19 16:27
edited on: 2009-08-19 16:29

Thanks for your answer. I'm making progress. I'm able to make a dll containing caml code compiled in byte & native mode and link a main file to the .lib file exporting the symbols I want.

Note that I put the wrapper outside the dll, it corresponds to the use case I have.

Now, I would like to understand the other method which use flexdll to dynamically load my dll. I manage to compile but execution fails :

./mainflex.exe
No master relocation table
make: *** [byteflexdll] Error 1


Can you explain what happen?
Unzip caml_from_c_main_with_flexdll.zip to reproduce the issue.

(0005058)
frisch (developer)
2009-08-19 21:12

Flexlink can really produce two very different kinds of DLLs:

- regular DLLs, which correspond to "main program" linked as a DLL (that, they are in charge of performing the relocation for other DLLs loaded by flexlink) and can be loaded in normal ways (either explicit with LoadLibrary or through their import library);

- flexlink DLLs, which need to be loaded by flexdll_dlopen.


The "-output-obj X.dll" feature of OCaml produces a regular DLL. If you want to produce a flexlink DLL, you need to produce the .obj yourself and use flexlink explictly.

Also, since mainflex.c does not refer to OCaml headers, it seems better to call the C compiler directly.

Here is a working Makefile, for both bytecode and native:

byteflexdll:
    ocamlc -output-obj -o modcaml.obj mod.ml
    ocamlc -c modwrap.c
    $(MKDLL) -o libmodwrap.dll modwrap.obj modcaml.obj $(PREFIX)/lib/libcamlrun.lib $(BYTECCLIBS)
    cl /MD /nologo /c -I "$(shell flexlink -where)" mainflex.c
    $(MKEXE) $(FLEXOPT) -o mainflex.exe mainflex.obj
    ./mainflex.exe

optflexdll:
    ocamlopt -output-obj -o modcaml.obj mod.ml
    ocamlc -c modwrap.c
    $(MKDLL) -o libmodwrap.dll modwrap.obj modcaml.obj $(PREFIX)/lib/libasmrun.lib $(BYTECCLIBS)
    cl /MD /nologo /c -I "$(shell flexlink -where)" mainflex.c
    $(MKEXE) $(FLEXOPT) -o mainflex.exe mainflex.obj
    ./mainflex.exe
(0005061)
daweil (reporter)
2009-08-24 16:31

Thanks. All these examples works fine. I think you can close the issue. Will this makefile be include in the official caml documentation?
(0005110)
daweil (reporter)
2009-09-18 10:20

It doesn't work on Windows 64bits. I found 2 issues.
1) ocamlc -output-obj fails with the message
"ocamlc -output-obj -o modcaml.obj mod.ml
modcaml.c
modcaml.c(2465) : error C2371: 'caml_get_public_method' : redefinition; different basic types
        C:/ProgramFiles/ocaml-3.11.1win64VC8/lib\caml/mlvalues.h(190) : see declaration of 'caml_get_public_method'
modcaml.c(2721) : warning C4133: 'initializing' : incompatible types - from 'value (__cdecl *)(value,value)' to 'primitive'
File "mod.ml", line 1, characters 0-1:
Error: Error while building custom runtime system
"

2) I cannot use flexlink together with /DEBUG link option if I want to debug by main program:
"flexlink -x64 -merge-manifest -exe -o main_opt.exe modwrap.obj main.obj libmodwrap.lib -link "/DEBUG"
flexdll_msvc.obj : fatal error LNK1103: debugging information corrupt; recompile module
** Fatal error: Error during linking"
(0005114)
frisch (developer)
2009-09-18 22:50

I won't be able to look at (1) in the next two weeks. For (2), it is indeed the case that flexlink does not support COFF debugging information and there is little chance that it will be implemented (by myself).
(0005116)
daweil (reporter)
2009-09-23 15:10

All these issues are only on 64 machines.

For 2), the problem may be that I don't use the same version of compiler, so I should try again to recompile flexlink myself (I tried a few weeks ago but failed)
(0005117)
xleroy (administrator)
2009-09-23 16:32

I looked into (1) and I think I've diagnosed and fixed the problem. Still needs to be tested under Win64. Fix will go in 3.11.2.

- Issue History
Date Modified Username Field Change
2009-08-18 17:43 daweil New Issue
2009-08-18 17:43 daweil File Added: caml_from_c.zip
2009-08-18 17:47 daweil Note Added: 0005050
2009-08-18 17:49 daweil Note Edited: 0005050
2009-08-18 17:49 daweil Note Edited: 0005050
2009-08-18 22:31 frisch Note Added: 0005051
2009-08-18 22:31 frisch Status new => assigned
2009-08-18 22:31 frisch Assigned To => frisch
2009-08-18 22:31 frisch Status assigned => resolved
2009-08-18 22:31 frisch Resolution open => fixed
2009-08-19 09:25 daweil Note Added: 0005052
2009-08-19 09:25 daweil Status resolved => feedback
2009-08-19 09:25 daweil Resolution fixed => reopened
2009-08-19 11:14 doligez Severity feature => major
2009-08-19 14:14 frisch Note Added: 0005053
2009-08-19 16:27 daweil Note Added: 0005057
2009-08-19 16:28 daweil File Added: caml_from_c_main_with_flexdll.zip
2009-08-19 16:29 daweil Note Edited: 0005057
2009-08-19 21:12 frisch Note Added: 0005058
2009-08-24 16:31 daweil Note Added: 0005061
2009-08-27 10:31 frisch Status feedback => resolved
2009-08-27 10:31 frisch Resolution reopened => fixed
2009-09-18 10:20 daweil Note Added: 0005110
2009-09-18 10:20 daweil Status resolved => feedback
2009-09-18 10:20 daweil Resolution fixed => reopened
2009-09-18 22:50 frisch Note Added: 0005114
2009-09-23 15:10 daweil Note Added: 0005116
2009-09-23 16:32 xleroy Note Added: 0005117
2009-09-23 16:33 xleroy Status feedback => resolved
2009-09-23 16:33 xleroy Resolution reopened => fixed
2009-09-23 16:33 xleroy Fixed in Version => 3.11.2+dev
2011-05-29 12:20 xleroy Status resolved => closed


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker