Mantis Bug Tracker
Main | My View | View Issues | Change Log | Roadmap
  

View Issue Details [ Jump to Notes ] [ Issue History ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005494OCamlOCaml standard librarypublic2012-01-27 08:532012-11-15 03:10
ReporterMartin Jambon 
Assigned To 
PrioritynormalSeverityfeatureReproducibilityN/A
StatusresolvedResolutionsuspended 
PlatformAllOSOS Version
Product Version3.12.1 
Target VersionFixed in Version 
Summary0005494: Read/write binary representation of int32, int64 and float
DescriptionIt would be convenient to have fast and reliable functions for reading
and writing the binary representation of int32, int64 and float
(without going through int64). Here is an interface suggestion:

All functions read or write a string at the specified position.

Modules Int32 and Int64:

val read_binary : string -> int -> t
val write_binary : string -> int -> t -> unit
val unsafe_read_binary : string -> int -> t
val unsafe_write_binary : string -> int -> t -> unit

Module Pervasives if not Float:

val read_binary_float : string -> int -> float
val write_binary_float : string -> int -> float -> unit
val unsafe_read_binary_float : string -> int -> float
val unsafe_write_binary_float : string -> int -> float -> unit


For the record, I am currently using the following code for floats
in https://github.com/mjambon/biniou/blob/master/bi_io.ml [^] .
It is somewhat fragile to say the least:

let float_endianness =
  match String.unsafe_get (Obj.magic 1.0) 0 with
      '\x3f' -> `Big
    | '\x00' -> `Little
    | _ -> assert false

let read_untagged_float64 ib =
  let i = Bi_inbuf.read ib 8 in
  let s = ib.i_s in
  let x = Obj.new_block Obj.double_tag 8 in
  (match float_endianness with
       `Little ->
     for j = 0 to 7 do
       String.unsafe_set (Obj.obj x) (7-j) (String.unsafe_get s (i+j))
     done
     | `Big ->
     for j = 0 to 7 do
       String.unsafe_set (Obj.obj x) j (String.unsafe_get s (i+j))
     done
  );
  (Obj.obj x : float)

let write_untagged_float64 ob x =
  let i = Bi_outbuf.alloc ob 8 in
  let s = ob.o_s in
  (match float_endianness with
       `Little ->
     for j = 0 to 7 do
       String.unsafe_set s (i+j) (String.unsafe_get (Obj.magic x) (7-j))
     done
     | `Big ->
     for j = 0 to 7 do
       String.unsafe_set s (i+j) (String.unsafe_get (Obj.magic x) j)
     done
  )

let () =
  let s = "\x3f\xf0\x06\x05\x04\x03\x02\x01" in
  let x = 1.00146962706651288 in
  let y = read_untagged_float64 (Bi_inbuf.from_string s) in
  if x <> y then
    assert false;
  let ob = Bi_outbuf.create 8 in
  write_untagged_float64 ob x;
  if Bi_outbuf.contents ob <> s then
    assert false
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
(0007072)
xleroy (administrator)
2012-03-14 10:37

 It's always hard to draw a line between what should go in the OCaml stdlib and what is best done in external libraries. In this particular case, before considering inclusion in the stdlib, I'd like to see the interface and implementation worked out in an external library, perhaps Batteries? or even a specific "fast I/O" library?
(0007076)
Martin Jambon (reporter)
2012-03-14 18:30

 Agreed. These functions are probably not more useful than most requests for inclusion in the stdlib, but they are harder to maintain because they rely on undocumented features. Let's see what the Batteries team thinks of this.
(0007103)
Martin Jambon (reporter)
2012-03-19 20:22

 The question just popped up on Stack Overflow: http://stackoverflow.com/questions/9776245/ocaml-int-to-binary-string-conversion [^]

This is the Batteries-devel thread: https://lists.forge.ocamlcore.org/pipermail/batteries-devel/2012-March/001600.html [^]
(0007105)
meyer (developer)
2012-03-20 02:29

 If I can just add my two cents - I perceive stdlib always as being just enough to bootstrap OCaml system + a little bit of convenience. That's what I personally think is a good balance. However the line is blurry what's being this additional convenience... Just a general thought...

However, on other hand, there are some modules that are just must have and provide basic blocks for example for system programming which are not used to just compile OCaml (e.g. Unix module or Bignum..), but in these case I think the stdlib serves the same purpose as C standard library for instance as a still lean "standard library".

Just two cents,
Wojciech
(0008513)
warwick (reporter)
2012-11-15 03:10

 I just wanted to add the opinion that I think it's good if the OCaml standard library does certain things in quite a powerful way, and other things not at all.

So for example the List module should (and does) have powerful features for manipulating lists. Otherwise programmers have to work with basic and extended versions of all the core modules: List and 'ListExtra', Array and 'ArrayExtra', String and 'StringExtra', etc. I think it's confusing and a bit inefficient to have this split between basic and extended modules handling the same data structures.

Then there can be other things that the standard library doesn't do at all, such as data compression, sending e-mail, etc ... and it's clear that you need an external library for these things.

Just my two cents as well!

Warwick

- Issue History
Date Modified Username Field Change
2012-01-27 08:53 Martin Jambon New Issue
2012-03-14 10:37 xleroy Note Added: 0007072
2012-03-14 10:37 xleroy Status new => resolved
2012-03-14 10:37 xleroy Resolution open => suspended
2012-03-14 10:37 xleroy Category OCaml general => OCaml standard library
2012-03-14 18:30 Martin Jambon Note Added: 0007076
2012-03-19 20:22 Martin Jambon Note Added: 0007103
2012-03-20 02:29 meyer Note Added: 0007105
2012-11-15 03:10 warwick Note Added: 0008513

Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker