Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memory leak in bigarray read from file #3612

Closed
vicuna opened this issue Apr 28, 2005 · 7 comments
Closed

memory leak in bigarray read from file #3612

vicuna opened this issue Apr 28, 2005 · 7 comments

Comments

@vicuna
Copy link

vicuna commented Apr 28, 2005

Original bug ID: 3612
Reporter: administrator
Assigned to: @gasche
Status: closed (set by @xavierleroy on 2017-02-16T14:15:01Z)
Resolution: fixed
Priority: high
Severity: major
Target version: 4.03.0+dev / +beta1
Fixed in version: 4.03.0+dev / +beta1
Category: otherlibs
Monitored by: till @oandrieu @mmottl

Bug description

Full_Name: Gary Huber
Version: 3.08.3
OS: Red Hat 8.0
Submission from: machaut.ucsd.edu (132.239.25.28)

Dear Ocaml folks,

I think I may have discovered a memory leak when reading
Bigarray objects from a file.
I use the first program to generate a file, which is read by the
second program. When I run the second program using the
memory leak checker valgrind, it shows some leaks.

By the way, I love Ocaml - I abandoned C++ over 2 years
ago for Ocaml. Thanks for developing such a great language.


(* First program: bug_gen.ml *)

type vector = (float, Bigarray.float64_elt, Bigarray.c_layout)
Bigarray.Array1.t
let ndim = 8;;

let x = Bigarray.Array1.create Bigarray.float64 Bigarray.c_layout ndim;;
for k = 0 to ndim-1 do
Bigarray.Array1.set x k 11.1
done;;

let n = 10000;;

let fp = open_out_bin "stuff.dat";;

output_value fp n;;
for i = 0 to n-1 do
output_value fp x
done;;


(* Second program: bug.ml *)

type vector = (float, Bigarray.float64_elt, Bigarray.c_layout)
Bigarray.Array1.t
let fp = open_in_bin "stuff.dat";;

let n = ((input_value fp) : int);;

for i = 0 to n-1 do
let v = ((input_value fp):vector) in ()
done
;;

close_in fp;;


Output from valgrind:

valgrind --leak-check=yes ./bug
==18183== Memcheck, a.k.a. Valgrind, a memory error detector for x86-linux.
==18183== Copyright (C) 2002-2003, and GNU GPL'd, by Julian Seward.
==18183== Using valgrind-2.0.0, a program supervision framework for x86-linux.
==18183== Copyright (C) 2000-2003, and GNU GPL'd, by Julian Seward.
==18183== Estimated CPU clock rate is 1499 MHz
==18183== For more details, rerun with: -v
==18183==
==18183==
==18183== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==18183== malloc/free: in use at exit: 1065285 bytes in 10016 blocks.==18183==
malloc/free: 30018 allocs, 20002 frees, 1975298 bytes allocated.
==18183== For counts of detected errors, rerun with: -v
==18183== searching for pointers to 10016 not-freed blocks.
==18183== checked 4578268 bytes.
==18183==
==18183== 60 bytes in 1 blocks are definitely lost in loss record 1 of 7
==18183== at 0x400296B2: malloc (vg_replace_malloc.c:153)
==18183== by 0x804ECDE: caml_init_major_heap (in /home/ghuber/hcmm/flip/bug)
==18183== by 0x805653D: caml_init_gc (in /home/ghuber/hcmm/flip/bug)
==18183== by 0x8058F6C: caml_main (in /home/ghuber/hcmm/flip/bug)
==18183==
==18183==
==18183== 1088 bytes in 17 blocks are possibly lost in loss record 2
of 7
==18183== at 0x400296B2: malloc (vg_replace_malloc.c:153)
==18183== by 0x804D124: bigarray_deserialize (bigarray_stubs.c:785)
==18183== by 0x8054AA8: intern_rec (in /home/ghuber/hcmm/flip/bug)==18183==
by 0x8054D82: caml_input_val (in /home/ghuber/hcmm/flip/bug)
==18183==
==18183==
==18183== 339392 bytes in 5303 blocks are definitely lost in loss record 7 of 7
==18183== at 0x400296B2: malloc (vg_replace_malloc.c:153)
==18183== by 0x804D124: bigarray_deserialize (bigarray_stubs.c:785)
==18183== by 0x8054AA8: intern_rec (in /home/ghuber/hcmm/flip/bug)==18183==
by 0x8054D82: caml_input_val (in /home/ghuber/hcmm/flip/bug)
==18183==
==18183== LEAK SUMMARY:
==18183== definitely lost: 339452 bytes in 5304 blocks.
==18183== possibly lost: 1088 bytes in 17 blocks.
==18183== still reachable: 724745 bytes in 4695 blocks.
==18183== suppressed: 0 bytes in 0 blocks.
==18183== Reachable blocks (those to which a pointer was found) are not shown.
==18183== To see them, rerun with: --show-reachable=yes
==18183==

@vicuna
Copy link
Author

vicuna commented Jul 20, 2005

Comment author: administrator

In this example, input_value allocates data block in minor heap, and those
blocks are never copied to the major heap. Hence the bigarray finalization
routine is never called.

@vicuna
Copy link
Author

vicuna commented Apr 6, 2006

Comment author: @oandrieu

Bonjour,

ce bug est trés gênant et pas vraiment "minor" AMHA.

Et si j'ai bien compris la nature du pb, on doit avoir ce même comportement pour toute valeur custom possédant à la fois une fonction de finalisation et des fonctions de sérialisation, non ?

Dans mon cas, j'ai un programme distribué (avec ocamlmpi) qui envoie de grosses valeurs via Bigarray. Là tout s'arrête très vite après quelques itérations car le programme receveur accumule tous les bigarray sans jamais les libérer.

Y'a t-il une bidouille, un "workaround" ?

[Edit: typos]

@vicuna
Copy link
Author

vicuna commented Apr 11, 2006

Comment author: @oandrieu

OK I found a simple enough workaround. I now serialize the bigarray in a tuple with a string of size Max_young_wosize (1K on 32bit arch, 2K on 64bit). It seems to work so far.

@vicuna
Copy link
Author

vicuna commented Jul 11, 2014

Comment author: hnrgrgr

See #82

@vicuna
Copy link
Author

vicuna commented Jan 20, 2015

Comment author: @damiendoligez

See also #92 , which supersedes #82.

@vicuna
Copy link
Author

vicuna commented Jun 9, 2015

Comment author: @damiendoligez

Upping the priority because this bug was rediscovered and reported to me.
Also, #92 has the solution, so we should fix this before 4.03.

@vicuna
Copy link
Author

vicuna commented Aug 16, 2015

Comment author: @gasche

This is now fixed in trunk -- by merging Pierre Chambart's patchset.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants