Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005957OCamlOCaml backend (code generation)public2013-03-21 18:562013-04-17 11:09
Reporterchetsky@gmail.com 
Assigned Togasche 
PrioritynormalSeverityminorReproducibilityalways
StatusresolvedResolutionfixed 
Platformx86OSUbuntu LinuxOS Version12.04
Product Version4.00.1 
Target VersionFixed in Version4.01.0+dev 
Summary0005957: linking -big- files causes failure in ocamlc
DescriptionWhen linking big .cmo files (like created by -pack in the Jane St core lib), ocamlc will fail with an Invalid_argument("String.create") -- it's trying to create a string of length 43Mbytes.
Steps To Reproduce(can repro with an opam installation pretty easily):

(1) ocaml4.00.1 + opam (1.0.0)
(2) opam install -y ocamlfind core

(3) compile the below file "yadda_main.ml" with:

  % camlfind ocamlc -verbose -package core,threads.posix -linkpkg -thread -
custom -g -o yadda yadda_main.ml

================================================================
module Mutex = Core.Std.Mutex

let main() =
let m = Mutex.create() in
  Mutex.lock m ;
  (try
     if Mutex.try_lock m then print_string "Free" else print_string "Held by
other"
   with _ -> print_string "Held by me") ;
  print_newline() ;
  flush stdout ;
  Mutex.unlock m
;;

main() ;;
Additional InformationI built a version of ocamlc with "-g" and got a backtrace. I believe that the problem is with Misc.input_bytes, as it is used to read debug info. I am sure that this is not the right fix, but this -does- work (make world, make world.opt, make bootstrap, install, then rerun testcase).

================================================================
diff -Bwiu --recursive ../../Caml4.00/src.OLD/ocaml-4.00.1/bytecomp/bytelink.ml
ocaml-4.00.1/bytecomp/bytelink.ml
--- ../../Caml4.00/src.OLD/ocaml-4.00.1/bytecomp/bytelink.ml 2012-04-16
08:27:42.000000000 -0700
+++ ocaml-4.00.1/bytecomp/bytelink.ml 2013-03-20 20:59:48.459111534 -0700
@@ -188,7 +188,7 @@
 
 (* Record compilation events *)
 
-let debug_info = ref ([] : (int * string) list)
+let debug_info = ref ([] : (int * string list) list)
 
 (* Link in a compilation unit *)
 
@@ -199,7 +199,7 @@
   Symtable.patch_object code_block compunit.cu_reloc;
   if !Clflags.debug && compunit.cu_debug > 0 then begin
     seek_in inchan compunit.cu_debug;
- let buffer = input_bytes inchan compunit.cu_debugsize in
+ let buffer = input_big_bytes inchan compunit.cu_debugsize in
     debug_info := (currpos_fun(), buffer) :: !debug_info
   end;
   output_fun code_block;
@@ -255,7 +255,7 @@
 let output_debug_info oc =
   output_binary_int oc (List.length !debug_info);
   List.iter
- (fun (ofs, evl) -> output_binary_int oc ofs; output_string oc evl)
+ (fun (ofs, evll) -> output_binary_int oc ofs; (List.iter (output_string
oc) evll))
     !debug_info;
   debug_info := []
 
diff -Bwiu --recursive ../../Caml4.00/src.OLD/ocaml-4.00.1/utils/misc.ml
ocaml-4.00.1/utils/misc.ml
--- ../../Caml4.00/src.OLD/ocaml-4.00.1/utils/misc.ml 2012-07-30
11:59:07.000000000 -0700
+++ ocaml-4.00.1/utils/misc.ml 2013-03-20 21:05:02.207099022 -0700
@@ -160,6 +160,25 @@
   result
 ;;
 
+let input_big_bytes ic n =
+ let _blocksiz = 65536 in
+ let input0 n =
+ assert (n <> 0) ;
+ let toread = min n _blocksiz in
+ let result = String.create toread in
+ let nread = input ic result 0 toread in
+ assert (nread <> 0) ;
+ if nread <> toread then
+ String.sub result 0 nread
+ else result in
+
+ let rec inrec acc n =
+ if n = 0 then List.rev acc
+ else
+ let buf = input0 n in
+ inrec (buf::acc) (n-(String.length buf)) in
+ inrec [] n
+
 (* Integer operations *)
 
 let rec log2 n =
diff -Bwiu --recursive ../../Caml4.00/src.OLD/ocaml-4.00.1/utils/misc.mli
ocaml-4.00.1/utils/misc.mli
--- ../../Caml4.00/src.OLD/ocaml-4.00.1/utils/misc.mli 2012-05-30
06:29:48.000000000 -0700
+++ ocaml-4.00.1/utils/misc.mli 2013-03-20 20:47:46.707140317 -0700
@@ -72,6 +72,10 @@
         (* [input_bytes ic n] reads [n] bytes from [ic] and returns them
            in a new string. It raises [End_of_file] if EOF is encountered
            before all the bytes are read. *)
+val input_big_bytes : in_channel -> int -> string list;;
+ (* [input_bytes ic n] reads [n] bytes from [ic] and returns them
+ in a new string. It raises [End_of_file] if EOF is encountered
+ before all the bytes are read. *)
 
 val log2: int -> int
         (* [log2 n] returns [s] such that [n = 1 lsl s]
TagsNo tags attached.
Attached Filesdiff file icon longstring-3.diff [^] (9,721 bytes) 2013-03-25 22:16 [Show Content]

- Relationships
duplicate of 0005920closed ocamlc linking failed due to Invalid_argument of String.create when with -g 

-  Notes
(0008997)
gasche (developer)
2013-03-21 22:11

This is the same bug report as http://caml.inria.fr/mantis/view.php?id=5920, [^] but I'm keeping this one as your patch is a good way to start discussion.

As far as patches are concerned, it seems that you identified the right point to change for the debuginfo problem, but I must say I like BenoƮt's approach in http://caml.inria.fr/mantis/file_download.php?file_id=874&type=bug [^] better. It might be interesting to merge the two things, that is extend the use of his LongString type to a input_long_bytes function and use it in the place you changed.

That said, the remark on 32bits users suffering from a lack of maintainer love still stands. Your best long-term workaround is to use a 64 bit architecture.
(0008998)
chetsky@gmail.com (reporter)
2013-03-22 02:11

Gasche, I looked at Benoit's changes, and

(a) to someone who doesn't understand the compiler much, yes his changes look analogous to what I tried to do

(b) his Longstring is much nicer than my little -hack- of "string list". Heh. I thought of writing something prettier (a Bigstring module) but decided that before going down that path, I'd log the report -- in case somebody else had already gone down this road. As, it turn out, somebody did.

So (c) I'll try out his Longstring and report back.
(0009012)
chetsky@gmail.com (reporter)
2013-03-25 22:12
edited on: 2013-03-25 22:16

gasche, I verified that Benoit's patch, with some minor additions to cover the case that my patch was for, builds, passes bootstrap, and also passes the testcase that I submitted above.

It's a pretty minor change, and I'm going to upload "longstring-2.diff", containing both benoit's patch, and my changes on top of his.

[Ugh: I included a file by mistake. "longstring-3.diff" should contain the right diff]

(0009025)
doligez (administrator)
2013-04-02 13:44

I think we should apply this patch. Should it go into trunk only, or also in the 4.00 branch?
(0009154)
gasche (developer)
2013-04-17 11:09

I just committed the change (in trunk only) with minor cosmetic changes.

Thanks, chetsky, for your help.

- Issue History
Date Modified Username Field Change
2013-03-21 18:56 chetsky@gmail.com New Issue
2013-03-21 22:04 gasche Relationship added duplicate of 0005925
2013-03-21 22:04 gasche Relationship deleted 0005925
2013-03-21 22:04 gasche Relationship added duplicate of 0005920
2013-03-21 22:11 gasche Note Added: 0008997
2013-03-21 22:11 gasche Status new => acknowledged
2013-03-22 02:11 chetsky@gmail.com Note Added: 0008998
2013-03-25 22:12 chetsky@gmail.com Note Added: 0009012
2013-03-25 22:12 chetsky@gmail.com File Added: longstring-2.diff
2013-03-25 22:16 chetsky@gmail.com Note Edited: 0009012 View Revisions
2013-03-25 22:16 chetsky@gmail.com File Added: longstring-3.diff
2013-03-29 17:15 gasche File Deleted: longstring-2.diff
2013-04-02 13:44 doligez Note Added: 0009025
2013-04-17 11:09 gasche Note Added: 0009154
2013-04-17 11:09 gasche Status acknowledged => resolved
2013-04-17 11:09 gasche Fixed in Version => 4.01.0+dev
2013-04-17 11:09 gasche Resolution open => fixed
2013-04-17 11:09 gasche Assigned To => gasche


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker