Project dbuffer
on-disk buffering of data streams

Thomas Jahns

January 6, 2002

Introduction

Many need to transfer large amounts of data over comparatively slow connections or connections of unreliable speed. Because of this, if a continous stream of data is required at the end of the connection once the transfer has begun, buffering is needed. Writing to CDR or Tape drives are prime examples of this requirement.

I wrote this program because all programs I found for this purpose used memory buffers only. This limits the maximum amount of buffered data on most machines to the available memory plus swap space while for many purposes the transfer speed of available disk media is sufficient.

dbuffer is a simple program that mimicks the command line interface of the buffer program distributed with e.g. SuSE Linux, but buffers data in a file on disk.

Requirements

dbuffer requires an ANSI-C compiler (plus needed helper programs and files) and an implementation of the pthreads library on the target systems, including development headers.

Portability

I have successfully compiled dbuffer on SuSE Linux 7.1 and 7.2 and SGI IRIX 6.5.11m. Any notice of other successful builds is welcome.

Download

A source archive can be downloaded here.