MantisBT - OCaml
View Issue Details
0005771OCamlback end (clambda to assembly)public2012-10-02 16:022015-12-11 19:20
chambart 
lefessan 
normalfeaturealways
closedfixed 
 
4.01.0+dev 
0005771: [patch] add primitives for directly reading 2, 4 or 8 bytes in strings and char bigarrays
This patch provides primitives to improve speed of code reading a lot of values from network/files. When those are implemented using C stubs, quite a lot of time is spent in the calls.

the provided primitives are:

type bigstring = (char, int8_unsigned_elt, c_layout) Array1.t

external caml_bigstring_get_16 : bigstring -> int -> int = "%caml_bigstring_get16"
external caml_bigstring_get_32 : bigstring -> int -> int32 = "%caml_bigstring_get32"
external caml_bigstring_get_64 : bigstring -> int -> int64 = "%caml_bigstring_get64"

external caml_bigstring_set_16 : bigstring -> int -> int -> unit = "%caml_bigstring_set16"
external caml_bigstring_set_32 : bigstring -> int -> int32 -> unit = "%caml_bigstring_set32"
external caml_bigstring_set_64 : bigstring -> int -> int64 -> unit = "%caml_bigstring_set64"

and the equivalent ones on strings.

external caml_string_get_16 : string -> int -> int = "%caml_string_get16"
external caml_string_get_32 : string -> int -> int32 = "%caml_string_get32"
external caml_string_get_64 : string -> int -> int64 = "%caml_string_get64"

external caml_string_set_16 : string -> int -> int -> unit = "%caml_string_set16"
external caml_string_set_32 : string -> int -> int32 -> unit = "%caml_string_set32"
external caml_string_set_64 : string -> int -> int64 -> unit = "%caml_string_set64"

Unsafe versions of the primitives also exist.
Those primitives allow loading values that are not alligned.
On architectures that does not allow unaligned access this is implemented by loading one byte at a time.

It is allowed on x86 and x86-64, but for the other architectures I made the safe guess that it is forbiden. Is it effectively the case on power processors ?

On a future version of the patch I can implement a more efficient unaligned load that requires only 2 loads and some shifts.
No tags attached.
patch 0001-Add-primitives-to-load-and-write-2-4-and-8-bytes-to-.patch (40,784) 2012-11-09 16:02
https://caml.inria.fr/mantis/file_download.php?file_id=792&type=bug
Issue History
2012-10-02 16:02chambartNew Issue
2012-10-02 16:02chambartFile Added: 0001-Add-allow_unsigned_access-to-arch-description.patch
2012-10-02 16:02chambartFile Added: 0002-Add-primitives-to-load-and-write-directly-2-4-and-8-.patch
2012-10-06 21:24gerdNote Added: 0008211
2012-10-07 00:26chambartNote Added: 0008212
2012-11-09 16:02chambartFile Added: 0001-Add-primitives-to-load-and-write-2-4-and-8-bytes-to-.patch
2012-11-09 16:02chambartNote Added: 0008476
2012-11-09 16:54lefessanFile Deleted: 0001-Add-allow_unsigned_access-to-arch-description.patch
2012-11-09 16:54lefessanFile Deleted: 0002-Add-primitives-to-load-and-write-directly-2-4-and-8-.patch
2012-11-09 17:15lefessanNote Added: 0008478
2012-11-09 17:15lefessanStatusnew => resolved
2012-11-09 17:15lefessanFixed in Version => 4.01.0+dev
2012-11-09 17:15lefessanResolutionopen => fixed
2012-11-09 17:15lefessanAssigned To => lefessan
2012-12-17 12:36frischNote Added: 0008616
2012-12-17 12:36frischStatusresolved => confirmed
2012-12-17 13:13frischNote Edited: 0008616bug_revision_view_page.php?bugnote_id=8616#r598
2012-12-19 16:44frischNote Added: 0008632
2013-06-27 13:55lefessanNote Added: 0009620
2013-06-27 13:55lefessanStatusconfirmed => resolved
2015-12-11 19:20xleroyStatusresolved => closed
2017-02-23 16:35doligezCategoryOCaml backend (code generation) => Back end (clambda to assembly)
2017-02-23 16:44doligezCategoryBack end (clambda to assembly) => back end (clambda to assembly)

Notes
(0008211)
gerd   
2012-10-06 21:24   
Supporting this.

I think we should also add primitives for nativeint, and for plain int (to avoid boxing when going through int32/int64).
(0008212)
chambart   
2012-10-07 00:26   
It is not needed to add a special primitive for int, if you write a function like this:

let f s i = Int32.to_int (caml_bigstring_get_32 s i)

there will be no allocaion: the compiler avoid boxing the value
(0008476)
chambart   
2012-11-09 16:02   
Add a version updated for current trunk.
(0008478)
lefessan   
2012-11-09 17:15   
Integrated in trunk at revision r13087.
(0008616)
frisch   
2012-12-17 12:36   
(edited on: 2012-12-17 13:13)
This commit breaks the MSVC port (in str.c and bigarray_stubs.c: it is not allowed to declare variables after the first statement in a function body).

(0008632)
frisch   
2012-12-19 16:44   
Same question as 0005774: what's the point of adding compiler support for those primitives if they are not exposed to user-land code? I understand this can be done in external libraries, but why not do it in stdlib?
(0009620)
lefessan   
2013-06-27 13:55   
The patch has been applied a long time ago. The library ocplib-endian uses these primitives when they are available, and non optimized primitives otherwise (while still using as little allocations as possible) in both cases.