Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0005501OCamlOCaml runtime systempublic2012-02-05 15:062012-04-12 10:57
Reportergerd 
Assigned Todoligez 
PrioritynormalSeveritytrivialReproducibilityN/A
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version3.12.1 
Target VersionFixed in Version4.00.0+dev 
Summary0005501: Wish: Increase IO_BUFFER_SIZE
DescriptionI think IO_BUFFER_SIZE=4096 (for channel I/O) is nowadays a bit low. For current machines and OS, this should be increased, maybe to 64K. (It's a constant in byterun/io.c).

A few indications why this is reasonable:
 - Certain OS increased their pipe buffer size (e.g. Linux to 64K, but it
   grows beyond that if needed; FreeBSD and OSX to 16K, with the option to
   grow to 64K)
 - RAM bandwidth is continously increasing
 - Systems have more RAM than ever
Additional InformationRecently I ran into a case where the producer of a pipe slowed the consumer down, because only 4K were written at a time. The pattern was especially bad because the producer wrote the results of computations and used the CPU to 100%, and the 4K blocks were emitted with a certain frequency. The consumer was poll-based, and the poll loop was running way to often, also at the same frequency. The result was that the consumer also used another core to almost 100%. The lesson to learn for me was that the consumer cannot do anything against this when the producer drives it the wrong way. Using more of the pipe buffer finally solved the problem (by running the pipe like "producer | dd bs=1M iflags=fullblock | consumer")
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
(0007306)
frisch (developer)
2012-04-10 14:40

I'm also in favor of increasing the default value for IO_BUFFER_SIZE (to something between 16kB and 64kB). Who's against?
(0007309)
doligez (administrator)
2012-04-10 16:09

Fixed in version 4.00 (commit 12331) and trunk (commit 12332).

- Issue History
Date Modified Username Field Change
2012-02-05 15:06 gerd New Issue
2012-03-26 14:16 lefessan Assigned To => lefessan
2012-03-26 14:16 lefessan Status new => acknowledged
2012-03-26 14:18 lefessan Assigned To lefessan =>
2012-04-10 14:40 frisch Note Added: 0007306
2012-04-10 15:48 doligez Assigned To => doligez
2012-04-10 15:48 doligez Status acknowledged => assigned
2012-04-10 16:09 doligez Note Added: 0007309
2012-04-10 16:09 doligez Status assigned => closed
2012-04-10 16:09 doligez Resolution open => fixed
2012-04-10 16:09 doligez Fixed in Version => 4.00.0+dev


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker