Amanda-Users

Re: How to force amanda to serialize dumper/taper

2006-12-07 00:17:17
Subject: Re: How to force amanda to serialize dumper/taper
From: Evan Harris <eharris AT puremagic DOT com>
To: Jaz Singh <jazee AT tds DOT net>
Date: Wed, 6 Dec 2006 22:57:02 -0600 (CST)

Unfortunately, eliminating (or drastically reducing) the holding disk workaround won't solve the problem in my case, since several of my larger dumps are from nonlocal disks, and the machines they are on cannot sustain a write across the network that is fast enough to keep the tape drive streaming. It would just move the problem from not being fast enough to stream from a local disk to not being fast enough to stream from remote, resulting in the same shoe-shine problem.

In a few cases, where a dump contains large directories filled with many small files, even local drives cannot keep the tape drive streaming because of the directory/stat lookup times. That is, without using a holding disk to stage them.

Evan

On Wed, 6 Dec 2006, Jaz Singh wrote:

My experience is the same.  For large dumps, it was significantly slower
to allow dumper and taper to work at the same time.  I fixed this by
limiting the holding disk to 5GB.  Our holding disk is a dedicated 300GB
SCSi drive.
This allows the small incrementals to build up but forces the 200GB+
dumps to go straight to tape.  Our tape loader (Quantum M1500) pushes at
about 25MB/sec when tar-ing straight to tape.  When dumping to holding
disk while taping at the same time, this drops down to 10MB/sec.  For a
backup that normally takes 8 hours on a weekday, this is too slow.  This
is all local disk backup of about 800GB per weeknight and 2.5TB on Friday.
Network backups will probably benefit from larger holding disk, since
the network limits the transfer to holding.


On Wed, 6 Dec 2006, Brian Cuttler wrote:

Evan,

What OS, type of tape, type of drive, HW platform ?

I'm unfamiliar with a parameter to do what your asking
for for good measure, what version of Amanda ?

Actually, how did you determine that a drive within a changer
was shoe-shining ? What else shares the bus with the tape and
with the amanda work area drive(s) ?

On Wed, Dec 06, 2006 at 02:34:31PM -0600, Evan Harris wrote:

I'm having a problem with my tape drive shoe-shining because the
holding
disk can't keep up with the tape drive if it is also being written
to by a
dumper.  Without the extra disk seek overhead of dumpers writing to the
holding disk at the same time, the holding disk should be plenty fast
enough to keep the tape drive streaming.

Is there any way I can force amanda to serialize the dumper/taper so
that
they are never run concurrently?  I've already set inparallel to 1, but
that only affects how many dumpers can run, not the taper.

I've also tried increasing the tapebufs parameter to 8000 (256MiB)
to see
if that would at least let the drive stay streaming for longer
periods, but
if it made any difference, it wasn't significant.  What thresholds
does the
taper use to decide when the tapebufs are filled enough to start tape
motion?  There doesn't seem to be any docs on that, or settings to
customize.

I did get a suggestion that I should just leave the tape out of the
drive,
let the dumpers fill the holding disk and then load the tape and run
amflush, but that doesn't really work when using a changer, plus the
holding disk isn't large enough for the total size of all the backups,
though it can fit them one-by-one.

Seems like there should be a "speed" config option for holding disks
like
there is for network interfaces and tape drives, so that amanda
could test
to see if the holding disk can't handle dumpers using the holding
disk at
the same time a taper is running.  That seems like it'd solve the
problem
nicely, and even seems to fit with the scheme amanda uses for network
interfaces.

Thanks.

Evan
---
  Brian R Cuttler                 brian.cuttler AT wadsworth DOT org
  Computer Systems Support        (v) 518 486-1697
  Wadsworth Center                (f) 518 473-6384
  NYS Department of Health        Help Desk 518 473-0773