Interfacing C with Caml Light

This chapter describes how user-defined primitives, written in C, can be added to the Caml Light runtime system and called from Caml Light code.

Overview and compilation information

Declaring primitives

User primitives are declared in a module interface (a .mli file), in the same way as a regular ML value, except that the declaration is followed by the = sign, the function arity (number of arguments), and the name of the corresponding C function. For instance, here is how the input primitive is declared in the interface for the standard library module io:

        value input : in_channel -> string -> int -> int -> int
                    = 4 "input"
Primitives with several arguments are always curried. The C function does not necessarily have the same name as the ML function.

Values thus declared primitive in a module interface must not be implemented in the module implementation (the .ml file). They can be used inside the module implementation.

Implementing primitives

User primitives with arity n <= 5 are implemented by C functions that take n arguments of type value, and return a result of type value. The type value is the type of the representations for Caml Light values. It encodes objects of several base types (integers, floating-point numbers, strings, ...), as well as Caml Light data structures. The type value and the associated conversion functions and macros are described in details below. For instance, here is the declaration for the C function implementing the input primitive:

        value input(channel, buffer, offset, length)
                value channel, buffer, offset, length;

When the primitive function is applied in a Caml Light program, the C function is called with the values of the expressions to which the primitive is applied as arguments. The value returned by the function is passed back to the Caml Light program as the result of the function application.

User primitives with arity greater than 5 are implemented by C functions that receive two arguments: a pointer to an array of Caml Light values (the values for the arguments), and an integer which is the number of arguments provided:

        value prim_with_lots_of_args(argv, argn)
                value * argv;
                int argn;
          ... argv[0] ...;              /* The first argument */
          ... argv[6] ...;              /* The seventh argument */

Implementing a user primitive is actually two separate tasks: on the one hand, decoding the arguments to extract C values from the given Caml Light values, and encoding the return value as a Caml Light value; on the other hand, actually computing the result from the arguments. Except for very simple primitives, it is often preferable to have two distinct C functions to implement these two tasks. The first function actually implements the primitive, taking native C values as arguments and returning a native C value. The second function, often called the ``stub code'', is a simple wrapper around the first function that converts its arguments from Caml Light values to C values, call the first function, and convert the returned C value to Caml Light value. For instance, here is the stub code for the input primitive:

        value input(channel, buffer, offset, length)
                value channel, buffer, offset, length;
          return Val_long(getblock((struct channel *) channel,
                                   &Byte(buffer, Long_val(offset)),
(Here, Val_long, Long_val and so on are conversion macros for the type value, that will be described later.) The hard work is performed by the function getblock, which is declared as:
        long getblock(channel, p, n)
             struct channel * channel;
             char * p;
             long n;

To write C code that operates on Caml Light values, the following include files are provided:
Include fileProvides
mlvalues.hdefinition of the value type, and conversion macros
alloc.hallocation functions (to create structured Caml Light objects)
memory.hmiscellaneous memory-related functions (for in-place modification of structures, etc).
These files reside in the Caml Light standard library directory (usually /usr/local/lib/caml-light).

Linking C code with Caml Light code

The Caml Light runtime system comprises three main parts: the bytecode interpreter, the memory manager, and a set of C functions that implement the primitive operations. Some bytecode instructions are provided to call these C functions, designated by their offset in a table of functions (the table of primitives).

In the default mode, the Caml Light linker produces bytecode for the standard runtime system, with a standard set of primitives. References to primitives that are not in this standard set result in the ``unavailable C primitive'' error.

In the ``custom runtime'' mode, the Caml Light linker scans the bytecode object files (.zo files) and determines the set of required primitives. Then, it builds a suitable runtime system, by calling the native code linker with:

This builds a runtime system with the required primitives. The Caml Light linker generates bytecode for this custom runtime system. The bytecode is appended to the end of the custom runtime system, so that it will be automatically executed when the output file (custom runtime + bytecode) is launched.

To link in ``custom runtime'' mode, execute the camlc command with:

The value type

All Caml Light objects are represented by the C type value, defined in the include file mlvalues.h, along with macros to manipulate values of that type. An object of type value is either:

Integer values

Integer values encode 31-bit signed integers. They are unboxed (unallocated).


Blocks in the heap are garbage-collected, and therefore have strict structure constraints. Each block includes a header containing the size of the block (in words), and the tag of the block. The tag governs how the contents of the blocks are structured. A tag lower than No_scan_tag indicates a structured block, containing well-formed values, which is recursively traversed by the garbage collector. A tag greater than or equal to No_scan_tag indicates a raw block, whose contents are not scanned by the garbage collector. For the benefits of ad-hoc polymorphic primitives such as equality and structured input-output, structured and raw blocks are further classified according to their tags as follows:
TagContents of the block
0 to No_scan_tag-1A structured block (an array of Caml Light objects). Each field is a value.
Closure_tagA closure representing a functional value. The first word is a pointer to a piece of bytecode, the second word is a value containing the environment.
String_tagA character string.
Double_tagA double-precision floating-point number.
Abstract_tagA block representing an abstract datatype.
Final_tagA block representing an abstract datatype with a ``finalization'' function, to be called when the block is deallocated.

Pointers to outside the heap

Any pointer to outside the heap can be safely cast to and from the type value. This includes pointers returned by malloc, and pointers to C variables obtained with the & operator.

Representation of Caml Light data types

This section describes how Caml Light data types are encoded in the value type.

Atomic types

Caml typeEncoding
intUnboxed integer values.
charUnboxed integer values (ASCII code).
floatBlocks with tag Double_tag.
stringBlocks with tag String_tag.

Product types

Tuples and arrays are represented by pointers to blocks, with tag 0.

Records are also represented by zero-tagged blocks. The ordering of labels in the record type declaration determines the layout of the record fields: the value associated to the label declared first is stored in field 0 of the block, the value associated to the label declared next goes in field 1, and so on.

Concrete types

Constructed terms are represented by blocks whose tag encode the constructor. The constructors for a given concrete type are numbered from 0 to the number of constructors minus one, following the order in which they appear in the concrete type declaration. Constant constructors are represented by zero-sized blocks (atoms), tagged with the constructor number. Non-constant constructors declared with a n-tuple as argument are represented by a block of size n, tagged with the constructor number; the n fields contain the components of its tuple argument. Other non-constant constructors are represented by a block of size 1, tagged with the constructor number; the field 0 contains the value of the constructor argument. Example:

Constructed termRepresentation
()Size = 0, tag = 0
falseSize = 0, tag = 0
trueSize = 0, tag = 1
[]Size = 0, tag = 0
h::tSize = 2, tag = 1, first field = h, second field = t

Operations on values

Kind tests

Operations on integers

Accessing blocks

The expressions Field(v,n), Code_val(v), Env_val(v), Byte(v,n), Byte_u(v,n) and Double_val(v) are valid l-values. Hence, they can be assigned to, resulting in an in-place modification of value v. Assigning directly to Field(v,n) must be done with care to avoid confusing the garbage collector (see below).

Allocating blocks

From the standpoint of the allocation functions, blocks are divided according to their size as zero-sized blocks, small blocks (with size less than or equal to Max_young_wosize), and large blocks (with size greater than to Max_young_wosize). The constant Max_young_wosize is declared in the include file mlvalues.h. It is guaranteed to be at least 64 (words), so that any block with constant size less than or equal to 64 can be assumed to be small. For blocks whose size is computed at run-time, the size must be compared against Max_young_wosize to determine the correct allocation procedure.

Raising exceptions

C functions cannot raise arbitrary exceptions. However, two functions are provided to raise two standard exceptions:

Living in harmony with the garbage collector

Unused blocks in the heap are automatically reclaimed by the garbage collector. This requires some cooperation from C code that manipulates heap-allocated blocks.

After a structured block (a block with tag less than No_scan_tag) is allocated, all fields of this block must be filled with well-formed values before the next allocation operation. If the block has been allocated with alloc or alloc_tuple, filling is performed by direct assignment to the fields of the block:
        Field(v, n) = vn;
If the block has been allocated with alloc_shr, filling is performed through the initialize function:
        initialize(&Field(v, n), vn);

The next allocation can trigger a garbage collection. The garbage collector assumes that all structured blocks contain well-formed values. Newly created blocks contain random data, which generally do not represent well-formed values.

If you really need to allocate before the fields can receive their final value, first initialize with a constant value (e.g. Val_long(0)), then allocate, then modify the fields with the correct value (see rule 3).

Local variables containing values must be registered with the garbage collector (using the Push_roots and Pop_roots macros), if they are to survive a call to an allocation function.

Registration is performed with the Push_roots and Pop_roots macros. Push_roots(r,n) declares an array r of n values and registers them with the garbage collector. The values contained in r[0] to r[n-1] are treated like roots by the garbage collector. A root value has the following properties: if it points to a heap-allocated block, this block (and its contents) will not be reclaimed; moreover, if this block is relocated by the garbage collector, the root value is updated to point to the new location for the block. Push_roots(r,n) must occur in a C block exactly between the last local variable declaration and the first statement in the block. To un-register the roots, Pop_roots() must be called before the C block containing Push_roots(r,n) is exited. (Roots are automatically un-registered if a Caml exception is raised.)

Direct assignment to a field of a block, as in
        Field(v, n) = w;
is safe only if v is a block newly allocated by alloc or alloc_tuple; that is, if no allocation took place between the allocation of v and the assignment to the field. In all other cases, never assign directly. If the block has just been allocated by alloc_shr, use initialize to assign a value to a field for the first time:
        initialize(&Field(v, n), w);
Otherwise, you are updating a field that previously contained a well-formed value; then, call the modify function:
        modify(&Field(v, n), w);

To illustrate the rules above, here is a C function that builds and returns a list containing the two integers given as parameters:

value alloc_list_int(i1, i2)
        int i1, i2;
  value result;
  Push_roots(r, 1);
  r[0] = alloc(2, 1);                     /* Allocate a cons cell */
  Field(r[0], 0) = Val_int(i2);           /* car = the integer i2 */
  Field(r[0], 1) = Atom(0);               /* cdr = the empty list [] */
  result = alloc(2, 1);                   /* Allocate the other cons cell */
  Field(result, 0) = Val_int(i1);         /* car = the integer i1 */
  Field(result, 1) = r[0];                /* cdr = the first cons cell */
  return result;
The ``cons'' cell allocated first needs to survive the allocation of the other cons cell; hence, the value returned by the first call to alloc must be stored in a registered root. The value returned by the second call to alloc can reside in the un-registered local variable result, since we won't do any further allocation in this function.

In the example above, the list is built bottom-up. Here is an alternate way, that proceeds top-down. It is less efficient, but illustrates the use of modify.

value alloc_list_int(i1, i2)
        int i1, i2;
  value tail;
  Push_roots(r, 1);
  r[0] = alloc(2, 1);                     /* Allocate a cons cell */
  Field(r[0], 0) = Val_int(i1);           /* car = the integer i1 */
  Field(r[0], 1) = Val_int(0);            /* A dummy value
  tail = alloc(2, 1);                     /* Allocate the other cons cell */
  Field(tail, 0) = Val_int(i2);           /* car = the integer i2 */
  Field(tail, 1) = Atom(0);               /* cdr = the empty list [] */
  modify(&Field(r[0], 1), tail);          /* cdr of the result = tail */
  return r[0];
It would be incorrect to perform Field(r[0], 1) = tail directly, because the allocation of tail has taken place since r[0] was allocated.

A complete example

This section outlines how the functions from the Unix curses library can be made available to Caml Light programs. First of all, here is the interface curses.mli that declares the curses primitives and data types:

type window;;                   (* The type "window" remains abstract *)
value initscr: unit -> window = 1 "curses_initscr"
  and endwin: unit -> unit = 1 "curses_endwin"
  and refresh: unit -> unit = 1 "curses_refresh"
  and wrefresh : window -> unit = 1 "curses_wrefresh"
  and newwin: int -> int -> int -> int -> window = 4 "curses_newwin"
  and mvwin: window -> int -> int -> unit = 3 "curses_mvwin"
  and addch: char -> unit = 1 "curses_addch"
  and mvwaddch: window -> int -> int -> char -> unit = 4 "curses_mvwaddch"
  and addstr: string -> unit = 1 "curses_addstr"
  and mvwaddstr: window -> int -> int -> string -> unit = 4 "curses_mvwaddstr"
;; (* lots more omitted *)
To compile this interface:
        camlc -c curses.mli

To implement these functions, we just have to provide the stub code; the core functions are already implemented in the curses library. The stub code file, curses.o, looks like:

#include <curses.h>
#include <mlvalues.h>

value curses_initscr(unit)
        value unit;
  return (value) initscr();     /* OK to coerce directly from WINDOW * to value
                                   since that's a block created by malloc() */

value curses_wrefresh(win)
        value win;
  wrefresh((WINDOW *) win);
  return Val_unit;

value curses_newwin(nlines, ncols, x0, y0)
        value nlines, ncols, x0, y0;
  return (value) newwin(Int_val(nlines), Int_val(ncols),
                        Int_val(x0), Int_val(y0));

value curses_addch(c)
        value c;
  addch(Int_val(c));            /* Characters are encoded like integers */
  return Val_unit;

value curses_addstr(s)
        value s;
  return Val_unit;

/* This goes on for pages. */
(Actually, it would be better to create a library for the stub code, with each stub code function in a separate file, so that linking would pick only those functions from the curses library that are actually used.)

The file curses.c can be compiled with:

        cc -c -I/usr/local/lib/caml-light curses.c
or, even simpler,
        camlc -c curses.c
(When passed a .c file, the camlc command simply calls cc on that file, with the right -I option.)

Now, here is a sample Caml Light program that uses the curses module:

#open "curses";;
let main_window = initscr () in
let small_window = newwin 10 5 20 10 in
  mvwaddstr main_window 10 2 "Hello";
  mvwaddstr small_window 4 3 "world";
  for i = 1 to 100000 do () done;
To compile this program, run:
        camlc -c
Finally, to link everything together:
        camlc -custom -o test test.zo curses.o -lcurses