on-disk buffering of data streams
January 6, 2002
Many need to transfer large amounts of data over comparatively slow
connections or connections of unreliable speed. Because of this, if a
continous stream of data is required at the end of the connection once
the transfer has begun, buffering is needed. Writing to CDR or Tape
drives are prime examples of this requirement.
I wrote this program because all programs I found for this purpose
used memory buffers only. This limits the maximum amount of buffered
data on most machines to the available memory plus swap space while for
many purposes the transfer speed of available disk media is
dbuffer is a simple program that mimicks the command line interface of
the buffer program distributed with e.g. SuSE Linux, but buffers data
in a file on disk.
dbuffer requires an ANSI-C compiler (plus needed helper
programs and files) and an implementation of the pthreads library on
the target systems, including development headers.
I have successfully compiled dbuffer on SuSE Linux 7.1 and
7.2 and SGI IRIX 6.5.11m. Any notice of other successful builds is
A source archive can be downloaded here.