Re: When functional languages can be accepted by industry?

From: Gerd Stolpmann (Gerd.Stolpmann@darmstadt.netsurf.de)
Date: Thu Apr 20 2000 - 14:45:34 MET DST

  • Next message: Markus Mottl: "Re: When functional languages can be accepted by industry?"

    On Mon, 17 Apr 2000, John Prevost wrote:
    >Perl has it's problems, and the quality of modules varies
    >immensely--but when I find a module, I can install it and try it out
    >in about 20 seconds. That lets me get on to the more important
    >problems of writing code.

    I had the simplicity of installing a Perl module in mind when I designed
    findlib. But of course the approach is different, mostly because Perl's loader
    works completely dynamically, but for OCaml the order of linking/loading must be
    determined at compile/link time.

    >While this sort of thing might, by a number of arguments, not be the
    >sort of thing that should be considered part of the "core language",
    >I'd like to argue that such an official blessing would be enough to
    >get people to start using the tools consistantly, rather than
    >everybody doing things their own way. And it's only when everybody's
    >working in approximately the same way that this kind of simplicity of
    >working with third-party modules becomes possible.

    I agree, but it is hard to achieve. The first problem are the different
    "development" environments ( - an industrial term, better we say build
    environment); there is some lowest common denominator for Unix systems (make;
    file commands such as cp, mv, rm; text tools such as sed) but the Windows
    and Macintosh worlds are completely different, and most people running these
    systems do not have build environments. I would say: let's begin with realistic
    goals, and that would be presuming the Unix-like environment.

    Another problem is that the command-line interface of the OCaml compilers has
    a too low abstraction niveau. It is more or less what a C compiler provides;
    the difference is that reusing components was never a goal of the C language,
    so I think that the CLI should be improved to better support components.
    Findlib is my suggestion for this. You can simply link your program with add-on
    components by naming the components, i.e. you specify *what* you want and not
    *how* the compiler should proceed to get the result. For example,

    ocamlc -o program -package x,y -linkpkg program.ml

    compiles program.ml and links it with the components x and y. You need not to
    specify

    - where the components are installed
    - which files must be linked
    - which components must be additionally linked because x or y need them
      (directly or indirectly)
    - if there is a dependency between x and y, you need not to specify the linking
      order
    - additional system libraries

    Findlib finds it for you.

    On the first glance, this looks just like being some luxury that is not really
    necessary. I am currently working on an (industrial) project where an OCaml
    module is part of a bigger program. Typically, the sources are compiled with
    something like

    ocamlc -package markup,netstring,str,unix,dynlink,pcre ...

    which is much more descriptive than

    ocamlc -I /opt/ocaml-2.04/site-lib/markup
           -I /opt/ocaml-2.04/site-lib/netstring
           -I /opt/ocaml-2.04/site-lib/pcre
           markup.cma str.cma netstring.cma unix.cma dynlink.cma pcre.cma
           -cclib -lpcre -cclib -lunix -cclib -lstr
           ...

    and also more portable; you can simply compile without having to know where all
    the necessary packages are installed on a particular system. I think it is not
    luxury, it is a necessary abstraction that simplifies notation and hides the
    peculiarities of the system installation. I think, these points are important
    for industrial usage, too (note that Findlib resulted (partly) from
    personal experience in industrial development).

    Of course, the current Findlib implementation has some weaknesses. First, it is
    not integrated into the compilers. That is not a big problem, but you must in
    fact write

    ocamlfind ocamlc -o program -package x,y -linkpkg program.ml

    - the extra "ocamlfind" is the command that transforms the new options such as
    -package into traditional options. - There is another problem with missing
    integration, as Findlib tries to find out the configuration of an OCaml system
    when it is compiled itself. For example, Findlib must know if POSIX or
    bytecode-only threads are used, and it inspects the files in the stdlib
    directory for this purpose. This may not work with future OCaml versions.

    Second, Findlib depends a bit (but not much) on Unix. This is simple to fix
    (for the Findlib core).

    Third, Findlib cannot handle name clashes. If the same (toplevel) module M
    occurs in two components x and y, and component z refers to M, findlib normally
    knows which M is meant (because it knows the module on which z depends), but
    it has no chance to tell the linker its knowledge. If the linker knew the
    component dependencies the linker could almost always resolve name clashes in
    the right way.

    Furthermore, Findlib does not address at all:

    - Versioning components. You cannot refer to specific versions of components,
      because I have no idea how this could be semantically cleanly solved.
      Especially, this seems to be in contradiction with automatic
      dependency analysis.

    - The build process. You need a "make" tool, and Findlib does not support you
      in writing Makefiles.

    Especially, at least conventions about the latter would be necessary to get a
    standard installation procedure for OCaml components.

    >Mmm. I actually have one minor gripe about the I/O stuff--not being
    >able to turn a set of functions into an in_channel or out_channel has
    >bit me a number of times. It's not so bad, until you want to do
    >something like implement an encrypting network stream and then use
    >stuff from Printf to write to it.

    Interesting idea, and not very expensive, because the basic I/O operations are
    normally performed for bigger chunks. Perhaps in/out_channel should be defined
    as classes such that you can easily define a new implementation. The old
    functional style could be retained, i.e.

    let output chan buff ofs len = chan # output buff ofs len

    and so on for all basic functions. This means that you CAN take advantage from
    the new class-based channel definition, but you can also continue preferring the
    functional style.

    >I think that if you took something like, say, Findlib, and asked if
    >you could integrate it into O'Caml, you'd discover at least a few
    >people who would make time to go over things looking for issues and
    >warts to clean up. It's hard to be enthused if it's not going to be
    >"official".
    >
    >Even though I've been waiting for a good solution for consistent
    >handling of third party modules almost as long as I've been using
    >O'Caml, the fact that only a few people use findlib cuts down on its
    >usefulness to me. If Findlib were accepted into O'Caml, I would go
    >out of my way to send findlibifying patches to authors of various
    >packages, instead of just getting depressed.

    I have no objections to the integration of Findlib into the OCaml core.

    Perhaps it is possible to make it at least "semi-official". I can put the
    sources into the CVS tree, and the distribution tarball could be downloadable
    from the Caml FTP server. It would be still a separate distribution with
    separate responsibility, but it looks like the "preferred" way of handling
    components; comparable to camltk or camlp4.

    But this is only a suggestion. Perhaps people want a different tool? I do not
    know. I designed Findlib primarily for my own problem constellation; and
    although I think it is quite general, other people might have different
    priorities (for example, versioning might be more important than handling
    component dependencies).

    Gerd

    -- 
    ----------------------------------------------------------------------------
    Gerd Stolpmann      Telefon: +49 6151 997705 (privat)
    Viktoriastr. 100             
    64293 Darmstadt     EMail:   Gerd.Stolpmann@darmstadt.netsurf.de (privat)
    Germany                     
    ----------------------------------------------------------------------------
    



    This archive was generated by hypermail 2b29 : Fri Apr 21 2000 - 19:23:28 MET DST