Mantis Bug Tracker
Main | My View | View Issues | Change Log | Roadmap
  

View Issue Details [ Jump to Notes ] [ Issue History ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005652OCamlOCamlbuild (the tool)public2012-06-18 22:122013-01-19 12:58
Reporterhongboz 
Assigned Tomeyer 
PrioritynormalSeverityfeatureReproducibilityhave not tried
StatusassignedResolutionreopened 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0005652: Add support for camlp4 native plugin
DescriptionThis is my ad-hoc solution, ocaml_specific.mli does not export anything, so I
have to copy paste.
The speed improvement is obvious. I gained 2~3 times speed improvement

(*stolen from Ocaml_specific.ml*)
let camlp4_flags camlp4s =
  List.iter begin fun camlp4 ->
    flag ["ocaml"; "pp"; camlp4] (A camlp4)
  end camlp4s;;
camlp4_flags ["camlp4o.opt"; "camlp4r.opt"; "camlp4of.opt"; "camlp4rf.opt"; "camlp4orf.opt"; "camlp4oof.opt"];;
let camlp4_flags' camlp4s =
  List.iter begin fun (camlp4, flags) ->
    flag ["ocaml"; "pp"; camlp4] flags
  end camlp4s;;
camlp4_flags' ["camlp4orr.opt", S[A"camlp4of.opt"; A"-parser"; A"reloaded"];
               "camlp4rrr.opt", S[A"camlp4rf.opt"; A"-parser"; A"reloaded"]];;
let camlp4 ?(default = A "camlp4r.opt") ?(printer=A "r")
    tag i o env build = (
  let ml = env i and pp_ml = env o in
  let tags = (((tags_of_pathname ml) ++ "ocaml" ++ "pp") ) ++ tag in
  let _deps = Rule.build_deps_of_tags build (tags ++ "ocamldep") in
  let pp = Command.reduce (Flags.of_tags tags) in
  let pp = match pp with | N -> default | _ -> pp in
  Cmd (S [ pp; P ml; A "-printer";printer; A "-o"; Px pp_ml ])
)
TagsNo tags attached.
Attached Filespatch file icon x.patch [^] (1,201 bytes) 2012-12-22 06:32 [Show Content]

- Relationships

-  Notes
(0007567)
meyer (developer)
2012-06-18 22:20
edited on: 2012-06-18 22:23

 Patch looks sensible with a reservation that ocaml.opt needs to be on path, so it should be rather:

flag ["ocaml"; "pp"; "native"; camlp4] (A camlp4)

EDIT:
Now I found that solver handles if it should pick up the ".opt" version of the toolchain, so I suppose the place that should be looked is options.ml
(0007570)
hongboz (developer)
2012-06-18 23:14

 yes, but options has nothing related with camlp4. It would be nice to export it
(0007696)
hongboz (developer)
2012-07-10 19:36

 make a change in options.ml is reasonalble
Options.camlp4

Since users can make their own camlp4 driver and won't bother myocamlbuild.ml
(0007997)
meyer (developer)
2012-09-02 03:34

 Checked in a fix in r12902.

The user still could still use -use-ocamlfind option and the "syntax(camlp4o.opt)" tag.
(0007998)
hongboz (developer)
2012-09-02 03:58

 will this fix cause such a problem
camlp4o.opt ff.cma
?
(0008002)
meyer (developer)
2012-09-02 05:14

 Sorry, I don't understand.

The fix works by filling holes with the camlp4 flavors with a preference on .opt version of these. If the .opt version is not available in the system it will fallback to the bytecode version.

There is a workaround that had been working without the fix that I checked in by using ocamlfind -syntax option (the proposed: syntax(camlp4o.opt))

However perhaps we need to wrap syntax in a nicer way, avoiding explicitly specifying .opt version, and let the ocamlbuild to chose the right one.
(0008040)
meyer (developer)
2012-09-10 04:56

 hongboz could you try the latest trunk and confirm the issue is gone?

Thanks
(0008045)
hongboz (developer)
2012-09-10 10:47

 meyer, I will have a try after ICFP
(0008635)
hongboz (developer)
2012-12-20 03:44

 This commit gives no way to use camlp4o instead of camlp4o.opt when you only have pa_*.cmo instead....
(0008640)
meyer (developer)
2012-12-22 00:28

 hongboz: I'm more than happy to review if you come up with a patch
(0008642)
hongboz (developer)
2012-12-22 06:30

 I have added a simple patch, please have a look
(0008655)
hongboz (developer)
2012-12-28 02:31

 meyer, will you take a look at the patch, if it's okay ?
thanks!
(0008656)
avsm (reporter)
2012-12-28 14:07

 The patch currently in trunk breaks our build system, since it tries to run "camlp4o.opt" with bytecode plugins instead of native code versions.

Most of the ocamlfind META files don't provide native code alternatives, and so it won't be safe to implicitly switch between the byte/native code versions of camlp4o without patching them first.

Can the patch in trunk be reverted to the old behaviour for the moment, so that package builds don't break?
(0008657)
meyer (developer)
2012-12-28 19:20

 Good catch, thanks for reporting. Reverted the fix in commit 13167 on trunk.

avsm, could you please monitor issue, as we currently suffer from not enough tests for ocamlbuild (this is being fixed at the moment;) and report back once the problem is fixed properly? I'm going have a go with it over the weekend.
(0008664)
avsm (reporter)
2012-12-29 07:31

 Sure; I've added trunk snapshots to our Jenkins tests now, so I can test patches fairly easily.
(0008669)
meyer (developer)
2012-12-29 22:00

 hongboz: your patch does not flag ocaml.opt being "native", so I'm testing slightly modified version now - and adding a test.
(0008776)
meyer (developer)
2013-01-19 12:58
edited on: 2013-01-19 13:18

 I've included your patch in commit 13260.

BTW: Native plugins are not being built with trunk version of Camlp4, can you look at it?

$ ocamlfind ocamlopt -pp 'camlp4o.opt -parser macro -DTEST' dummy.ml  
Camlp4: Uncaught exception: DynLoader.Error ("Camlp4MacroParser.cmxs", "file not found 
in path")



- Issue History
Date Modified Username Field Change
2012-06-18 22:12 hongboz New Issue
2012-06-18 22:20 meyer Note Added: 0007567
2012-06-18 22:20 meyer Assigned To => meyer
2012-06-18 22:20 meyer Status new => assigned
2012-06-18 22:23 meyer Note Edited: 0007567 View Revisions
2012-06-18 22:23 meyer Status assigned => acknowledged
2012-06-18 23:14 hongboz Note Added: 0007570
2012-07-10 19:36 hongboz Note Added: 0007696
2012-09-02 03:34 meyer Note Added: 0007997
2012-09-02 03:34 meyer Status acknowledged => resolved
2012-09-02 03:34 meyer Resolution open => fixed
2012-09-02 03:58 hongboz Note Added: 0007998
2012-09-02 05:14 meyer Note Added: 0008002
2012-09-10 04:56 meyer Note Added: 0008040
2012-09-10 04:56 meyer Status resolved => feedback
2012-09-10 04:56 meyer Resolution fixed => reopened
2012-09-10 10:47 hongboz Note Added: 0008045
2012-09-10 10:47 hongboz Status feedback => assigned
2012-12-20 03:44 hongboz Note Added: 0008635
2012-12-21 21:50 hongboz Assigned To meyer => hongboz
2012-12-22 00:28 meyer Note Added: 0008640
2012-12-22 06:30 hongboz Note Added: 0008642
2012-12-22 06:30 hongboz File Added: x.patch
2012-12-22 06:31 hongboz File Deleted: x.patch
2012-12-22 06:32 hongboz File Added: x.patch
2012-12-28 02:31 hongboz Note Added: 0008655
2012-12-28 14:07 avsm Note Added: 0008656
2012-12-28 19:20 meyer Note Added: 0008657
2012-12-29 07:31 avsm Note Added: 0008664
2012-12-29 21:37 meyer Assigned To hongboz => meyer
2012-12-29 22:00 meyer Note Added: 0008669
2013-01-19 12:58 meyer Note Added: 0008776
2013-01-19 13:18 meyer Note Edited: 0008776 View Revisions

Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker