0005258: Unix fd leak under windows

View Issue Details [ Jump to Notes ]

[ Issue History ] [ Print ]

Project

Category

View Status

Date Submitted

Last Update

0005258

OCaml

OCaml general

public

2011-04-27 23:48

2011-06-10 19:15

Reporter

toots

Assigned To

doligez

Priority

normal

Severity

major

Reproducibility

always

Status

closed

Resolution

fixed

Platform

OS Version

Product Version

3.12.0

Target Version

Fixed in Version

3.12.1+dev

Summary

0005258: Unix fd leak under windows

Description

The data structure for fd type in win32unix is the following:

struct filedescr {
  union {
    HANDLE handle;
    SOCKET socket;
  } fd; /* Real windows handle */
  enum { KIND_HANDLE, KIND_SOCKET } kind;
  int crt_fd; /* C runtime descriptor */
  unsigned int flags_fd; /* See FLAGS_FD_* */
};

Win32 has a different API for opening files with a compatibility layer for code using the ususal fopen etc.. API.

By default, in win32unix, files are opened using windows' API, that is the function CreateFile. Accordingly, crt_fd is set to NO_CRT_FD, which indicates that there is not C runtime descriptor existing for now.

If later a C runtime descriptor is required, which is the case for instance when convering a Unix.fd to an in_channel, then the following function is used:

int win_CRT_fd_of_filedescr(value handle)
{
  if (CRT_fd_val(handle) != NO_CRT_FD) {
    return CRT_fd_val(handle);
  } else {
    int fd = _open_osfhandle((long) Handle_val(handle), O_BINARY);
    if (fd == -1) uerror("channel_of_descr", Nothing);
    return fd;
  }
}

This function calls _open_osfhandle, which returns a C runtime descriptor from a regular windows file handle.

The problems occurs when trying to close the file, which is handled by the following function:

CAMLprim value unix_close(value fd)
{
  if (Descr_kind_val(fd) == KIND_SOCKET) {
    if (closesocket(Socket_val(fd)) != 0) {
      win32_maperr(WSAGetLastError());
      uerror("close", Nothing);
    }
  } else {
    if (! CloseHandle(Handle_val(fd))) {
      win32_maperr(GetLastError());
      uerror("close", Nothing);
    }
  }
  return Val_unit;
}

The issue here is that if a C runtime descriptor has been previously obtained, then using CloseHandle to close the file leaks the associated C runtime descriptor.

As a consequence, if too many file descriptors are leaks this way, then all subsequent functions used to get a C runtime descriptor will fail with a "too many opened files" error.

The issue is made even more obscur by the fact that, in this situation, the OS does not report those leaks in the list of opened files, because the underlying native file handle has indeed been closed properly...

Attached is a file (test_open.ml) that can be used to reproduce the issue. If one uncomments the call to Unix.in_channel_of_descr then the program fails after 2046 iterations (here) with the following error:

Running loop 0002045
Opened file
Fatal error: exception Unix.Unix_error(15, "channel_of_descr", "")

The solution to this issue is to close the file using _close when a C runtime descriptor is present. According to the documentation, this also close the underlying native file handle:

"To close a file opened with _open_osfhandle, call _close. The underlying handle is also closed by a call to _close, so it is not necessary to call the Win32 function CloseHandle on the original handle."
  http://msdn.microsoft.com/en-us/library/bdts1c9x.aspx [^]

Therefore, I am also attaching a patch that implements those changes. With these patches, the test_open.ml program works.

Tags

No tags attached.

Attached Files

test_open.ml [^] (510 bytes) 2011-04-27 23:48

fix_fd_leak.patch [^] (1,635 bytes) 2011-04-27 23:49 [Show Content] [Hide Content]

Index: build-tree/ocaml-3.12.0/otherlibs/win32unix/channels.c
===================================================================
--- build-tree.orig/ocaml-3.12.0/otherlibs/win32unix/channels.c	2010-01-22 06:48:24.000000000 -0600
+++ build-tree/ocaml-3.12.0/otherlibs/win32unix/channels.c	2011-04-27 14:38:52.000000000 -0500
@@ -30,6 +30,7 @@
   } else {
     int fd = _open_osfhandle((long) Handle_val(handle), O_BINARY);
     if (fd == -1) uerror("channel_of_descr", Nothing);
+    CRT_fd_val(handle) = fd;
     return fd;
   }
 }
Index: build-tree/ocaml-3.12.0/otherlibs/win32unix/close.c
===================================================================
--- build-tree.orig/ocaml-3.12.0/otherlibs/win32unix/close.c	2002-04-30 09:00:48.000000000 -0600
+++ build-tree/ocaml-3.12.0/otherlibs/win32unix/close.c	2011-04-27 14:39:48.000000000 -0500
@@ -15,6 +15,9 @@
 
 #include <mlvalues.h>
 #include "unixsupport.h"
+#include <io.h>
+
+extern int _close(int);
 
 CAMLprim value unix_close(value fd)
 {
@@ -24,9 +27,17 @@
       uerror("close", Nothing);
     }
   } else {
-    if (! CloseHandle(Handle_val(fd))) {
-      win32_maperr(GetLastError());
-      uerror("close", Nothing);
+    /* If we have an fd then closing it also closes
+     * the underlying handle. Also, closing only
+     * the handle and not the fd leads to fd leaks. */
+    if (CRT_fd_val(fd) != NO_CRT_FD) {
+      if (_close(CRT_fd_val(fd)) != 0)
+         uerror("close", Nothing);
+    } else {
+      if (! CloseHandle(Handle_val(fd))) {
+        win32_maperr(GetLastError());
+        uerror("close", Nothing);
+      }
     }
   }
   return Val_unit;

Relationships

Notes
(0005872) ripoche (reporter) 2011-04-28 10:48	This issue also affects 3.11.2.

Issue History
Date Modified	Username	Field	Change
2011-04-27 23:48	toots	New Issue
2011-04-27 23:48	toots	File Added: test_open.ml
2011-04-27 23:49	toots	File Added: fix_fd_leak.patch
2011-04-28 10:48	ripoche	Note Added: 0005872
2011-04-29 11:41	doligez	Assigned To	=> doligez
2011-04-29 11:41	doligez	Status	new => acknowledged
2011-06-10 19:15	doligez	Status	acknowledged => closed
2011-06-10 19:15	doligez	Resolution	open => fixed
2011-06-10 19:15	doligez	Fixed in Version	=> 3.12.1+dev