Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allowing to modify Buffers in place #6538

Closed
vicuna opened this issue Sep 3, 2014 · 4 comments
Closed

Allowing to modify Buffers in place #6538

vicuna opened this issue Sep 3, 2014 · 4 comments

Comments

@vicuna
Copy link

vicuna commented Sep 3, 2014

Original bug ID: 6538
Reporter: @alainfrisch
Status: feedback (set by @damiendoligez on 2014-12-17T21:51:04Z)
Resolution: open
Priority: normal
Severity: feature
Category: standard library
Related to: #6975
Monitored by: @ygrek @hcarty

Bug description

We currently have "mutable strings with fixed size" (Bytes) and "expandable but otherwise immutable strings" (Buffer). It would be useful to expose a "mutable and expandable string" data structure to users, and the simplest way to do it is probably to extend the Buffer API with some more "blit" functions (from strings, bytes, and perhaps another buffer). Or is there any conceptual reason to enforce that Buffer is append-only?

@vicuna
Copy link
Author

vicuna commented Dec 17, 2014

Comment author: @damiendoligez

I'd like to see some discussion on this subject.

@vicuna
Copy link
Author

vicuna commented Sep 13, 2015

Comment author: gerd

In Ocamlnet I'm using a derivative of Buffer called Netbuffer, allowing in-place modification, even with direct access to the underlying string (NB. this module is not yet ported to bytes):

http://projects.camlcity.org/projects/dl/ocamlnet-4.0.4/doc/html-main/Netbuffer.html

In practice, it turns out that Netbuffer.delete is by far the most important addition (for the purposes in Ocamlnet). Usually data at the beginning of the buffer is deleted, which is handy for stream-based data processors that append new data at the end of the buffer and remove processed data at the beginning. (Note, however, that there are better data structures for this use case, in particular page-aligned buffers, e.g. http://projects.camlcity.org/projects/dl/ocamlnet-4.0.4/doc/html-main/Netpagebuffer.html).

Netbuffer also tries to address one of the peculiarities of Buffer, namely that Buffer defines I/O functions. The point is that you want direct access to the underlying string for I/O, as this avoids another copy of the data. In Netbuffer I added a function add_inplace exposing the data buffer, e.g. you can do

Netbuffer.add_inplace buf Pervasives.input

This way buffering and I/O are again orthogonal in the API. This price to pay is that the buffer details are exposed in the API (and you are bound to it forever).

My experience is that inserting and overwriting strings is not really important. There is usually a good alternative that rebuilds the whole buffer in the way needed, and this alternative is often not slower. I'd consider this more as comfort feature.

@vicuna
Copy link
Author

vicuna commented Sep 19, 2015

Comment author: @c-cube

I like the idea of making Buffer more powerful. It should be a more powerful variant of Bytes.t, with indexing, concatenation, etc. and, indeed, some access to the underlying array for efficient IO.

An alternative is to give a set of low-level access primitives (say, export the type in a module BufferInternals or something, and let libraries write their own overlay). Or, provide Buffer.call: Buffer.t -> (bytes -> int -> int -> 'a) -> 'a, that will call the function (presumably a IO function of some sort) on the internal buffer, without breaking the abstraction barrier too much.

@github-actions
Copy link

This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant