English version
Accueil     À propos     Téléchargement     Ressources     Contactez-nous    

Ce site est rarement mis à jour. Pour les informations les plus récentes, rendez-vous sur le nouveau site OCaml à l'adresse ocaml.org.

Browse thread
[Caml-list] entropy etc. in OCaml
[ Home ] [ Index: by date | by threads ]
[ Search: ]

[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: 2004-08-11 (10:22)
From: Markus Mottl <markus@o...>
Subject: Re: [Caml-list] entropy etc. in OCaml
On Wed, 11 Aug 2004, Viktor Tron wrote:
> Does anyone know of an OCaml library implementing or binding
> Information Theoretical concepts like data entropy?
> (e.g. gsl does not provide these).

I don't have a separate library for that, but you might want to take a
look at AIFAD:


It implements several functions for computing the entropy of discrete
data including structured values.  Its purpose is decision tree learning
on structured data (represented by algebraic datatypes).

One function for computing entropy from histograms is the following
(taken from src/entropy_utils.ml in the distribution):

let calc_entropy histo n =
  if n = 0 then 0.0
    let rec loop sum ix =
      if ix < 0 then sum
        let freq = histo.(ix) in
        if freq = 0 then loop sum (ix - 1)
          let ffreq = float freq in
          loop (sum +. ffreq *. log ffreq) (ix - 1) in
    let sum = loop 0.0 (Array.length histo - 1) in
    let f_n = float n in
    log2 f_n -. sum /. f_n /. log_2

If you pass it an array of integers (histogram) that counts the frequency
of class values in variable "histo" and the number of observations in "n"
(must be the sum of frequencies in the histogram), then this function
will return you the entropy in bits.


Markus Mottl          http://www.oefai.at/~markus          markus@oefai.at

To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners