Browse thread
How to read different ints from a Bigarray?
[
Home
]
[ Index:
by date
|
by threads
]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: | 2009-10-28 (18:01) |
From: | Goswin von Brederlow <goswin-v-b@w...> |
Subject: | Re: [Caml-list] Re: How to read different ints from a Bigarray? |
Sylvain Le Gall <sylvain@le-gall.net> writes: > On 28-10-2009, Goswin von Brederlow <goswin-v-b@web.de> wrote: >> Sylvain Le Gall <sylvain@le-gall.net> writes: >> >>> Hello, >>> >>> On 28-10-2009, Goswin von Brederlow <goswin-v-b@web.de> wrote: >>>> Hi, >>>> >>> >>> Well, we talk about this a little bit, but here is my opinion: >>> - calling a C function to add a single int will generate a big overhead >>> - OCaml string are quite fast to modify values >>> >>> So to my mind the best option is to have a buffer string (say 16/32 >>> char) where you put data inside and flush it in a single C call to >>> Bigarray. >>> >>> E.g.: >>> let append_char t c = >>> if t.idx >= 64 then >>> ( >>> flush t.bigarray t.buffer; >>> t.idx <- 0 >>> ); >>> t.buffer.(t.idx) <- c; >>> t.idx <- t.idx + 1 >>> >>> let append_little_uint16 t i = >>> append_char t ((i lsr 8) land 0xFF); >>> append_char t ((i lsr 0) land 0xFF) >>> >>> >>> I have used this kind of technique and it seems as fast as C, and a lot >>> less C coding. >>> >>> Regards, >>> Sylvain Le Gall >> >> This wont work so nicely: >> >> - Writes are not always in sequence. I want to do a stream access >> too where this could be verry effective. But the plain buffer is >> more for random / known offset access. At a minimum you would have >> holes for alignment. >> >> - It makes read/write buffers complicated as you need to flush or peek >> the string in case of uncommited changes. I can't do write-only >> buffers as I want to be able to write a buffer and then add a >> checksum to it in my application. The lib should not block that. >> > > I was thinking to pure stream. It still stand with random access but you > don't get a lot less C function call. You just have to write less C > code. set_uint8 buf 5 1 -> read in 64 byte from stream, skip to 5, set byte set uint8 buf 100 1 -> write 64 byte, read other 64 byte, set byte That can become real expensive. >> I also still wonder how bad a C function call really is. Consider the >> case of writing an int64. >> >> Directly: You get one C call that does range check, endian convert and >> write in one go. >> >> Bffered: With your code you have 7 Int64 shifts, 8 Int64 lands, 8 >> conversions to int, at least one index check (more likely 8 to avoid >> handling unaligned access) and 1/8 C call to blit the 64 byte buffer >> string into the Bigarray. > > Not at all, you begin to break your int64 into 3 int (24bit * 2 + 16bit) > and then 7 int shift, 8 int land. > > You can even manage to only break into 1 or 2 int. > > And off course, you bypass index check. fun with unaligned writes. >> PS: Is a.{i} <- x a C call? > > Yes. That obviously sucks. I was hoping since the compiler has a special syntax for it it would be built-in. Bigarray being a seperate module should have clued me in. That obviously speaks against splitting int64 into 8 bytes and calling a.{i} <- x for each. I think I will implement your method and C stubs for every set/get and compare. Maybe ideal would be a format string based interface that calls C with a format string and a record of values. Because what I really need is to read/write records in an architecture independend way. Something like type t = { x:int; y:char; z:int64 } let t_format = "%2u%c%8d" put_formated buf t_format t But how to get that type safe? Maybe a camlp4 module that generates the format string and type from a single declaration so they always match. > Regards, > Sylvain Le Gall MfG Goswin