Amanda-Users

question about blocksize and tape buffers

2003-02-24 18:18:44
Subject: question about blocksize and tape buffers
From: Mike Simpson <msimpson AT abyssus.doit.wisc DOT edu>
To: amanda-users AT amanda DOT org
Date: Mon, 24 Feb 2003 16:07:56 -0600
Hi --

In doing some simple testing with my new AIT-3 based tape libraries, 
I note that they really seem to prefer larger block sizes, despite 
the admonition in Sony's manual to use 512 byte blocks for best 
performance.  Here's what I'm seeing using "dd" to write to the tape 
device (/dev/nst0) under RedHat Linux 7.3:

  bs= 512        5.4 mb/s
     1024        6.0 mb/s
     2048        6.1 mb/s
     4096        6.7 mb/s

  bs= 32k        7.5 mb/s
      64k        7.3 mb/s
     128k       10.6 mb/s
     256k       10.2 mb/s

  bs= 2mb        8.6 mb/s

The vendor-cited maximum native data transfer rate is 12 mb/s.

The "st" driver man page notes that it uses up to 16 x 128k internal 
buffers, which is why I tried 2mb for a top-end blocksize.  I've got 
the drive's dip switches set to manually disallow compression, and 
not let the driver change it.  The manual for the AIT-3 drive seems 
to indicate their are two buffers on the drive, an "interface" buffer 
(2 MB) and a "group" buffer (8 MB), although I'm not sure what each 
one is used for.

>From watching "iostat" during the test runs, the lower block sizes 
look like they're starving the drive, i.e. you can see the kb/s drop 
to zero every few seconds, then come back up to peak (11-12 mb/s) for 
5-6 seconds, then slide back to zero, etc.  At the higher blocksizes 
(128k, 256k) the drive pegs 11-12 mb/s and stays there.  For some 
reason 2mb doesn't work as well, I'm not sure why.

>From the Amanda docs and code, it looks like the Amanda taper
allocates "tapebufs" number of 32k buffers as shared memory.  Data
from the holding disk files gets pushed into the buffers, and then the
buffers flush to the tape device.  It also looks like there's a
"blocksize" config directive in the tapetypes section, but the man
page says that, "The minimum blocksize value is 32 KBytes.  The
maximum blocksize value is 32 KBytes." (?)

My question is this:  given the better performance under the higher
blocksizes, would increasing the buffer size from 32k up to something
like 128k give me better performance?  Can I obtain the same objective
(keeping the drive streaming) by just increasing "tapebufs" to some
fairly large number?

Thanks for any thoughts,

-mgs



<Prev in Thread] Current Thread [Next in Thread>