Version française
Home     About     Download     Resources     Contact us    

This site is updated infrequently. For up-to-date information, please visit the new OCaml website at

Browse thread
ocamlnet mini tutorial: netclient and cookie based authentication
[ Home ] [ Index: by date | by threads ]
[ Search: ]

[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: -- (:)
From: Hendrik Tews <tews@c...>
Subject: ocamlnet mini tutorial: netclient and cookie based authentication

for ocamlnet novices I document here what you need to access web
sites that use cookies for authentication (eg, wikis or mailman).
With this you can let ocaml perform your mailing-list
administrator tasks or write a wiki bot.

Cookie authentication works as follows: When you first visit the
site you have to authenticate with username and password that are
usually sent (in cleartext) in a post request to the server. The
server responses with a cookie that will be used for
authentication in all following requests. 

Because you have to retrieve and set cookies in the headers you
cannot use the Http_client.Convenience module. You really have to
dive into ocamlnet.

Here comes the solution:

Netstring extension:

- You need get_set_cookie, which will retrieve cookies from a
  response header but which is not in ocamlnets netstring library
  currently. Download from [1] or with two additional helper
  functions from [2].


- Create a pipeline with "new Http_client.pipeline", it will
  process all http requests. (Don't ask me about timeouts and
  retries, the defaults work fine for me.)

Get going (download some page without authentication):

- Create a get call "new Http_client.get url"

- Process the call: Add it to the pipeline (method #add) and run
  the pipeline (#run). By this the call will be changed and
  contain the response. 

- Check the response status (#response_status), which should
  usually be `OK and get the page (#response_body#value).

- In case of error convert the status into something readable
  with Nethttp.string_of_http_status or consult the response
  status message (#response_status_text).


- Create post call with 
  "new url parameters", the parameters are a
  string * string list, eq 
    [("wpName", "mywikiname");
     ("wpPassword", "mywikipasswd");
     ("wpLoginattempt", "Log in")]
  for a MediaWiki.

  [In order to get the parameters save the login page, change the
  method of the right from into "GET", load the page into a
  browser, press the login button and parse the new URL you get.]

- Process the call.

- Check the status (#response_status): For MediaWiki it should be
  `Found, for mailman and others `Ok (don't ask me why).

- Retrieve the response header (#response_header) and extract the
  cookies with get_set_cookies from [2].

Make an authenticated request:

- Create your call "new get url" for getting a page or 
  "new post url" for performing an action.

- Access the request header (#request_header `Base) and add your
  cookies there (set_cookies from [2]). This is all done via
  side-effects, no need to write the modified header back into
  the call.

- Process the call and get the response.


- with findlib and -package netclient -linkpkg:

  ocamlfind {ocamlc,ocamlopt} -package netclient -linkpkg your files ...


- Praise Gerd Stolpmann for ocamlnet and findlib.

That's it. For an example go to [3], which contains a MediaWiki
bot that updates structural information in the Mozilla wiki [4].
The needed utility functions are in wiki_http: wiki_login to
login, download to download a web page, get_page to get a wiki page
and write_wiki_page to write one.