|
Re: How to force amanda to serialize dumper/taper
2006-12-07 18:03:53
On Thu, 7 Dec 2006, Brian Cuttler wrote:
I've assumed that the spindle the amanda work area is on was dedicated
to the work area and not split, I know nothing about the architecture
That is correct. That disk is only used by amanda.
of the specific system bus. If you are able to get through put with
amdump than I'd guess its not a bus issue to either the tape nor the
drive. That leaves perhaps CPU or sharing the bus access to the disk
drive ?
It is almost certainly an issue (as I think I stated previously) with the
dumper writes and drive seeks between reading from and writing to the same
spindle degrading the holding disk throughput enough to cause the disk to be
unable to have sufficient bandwidth to keep the tape drive streaming.
But I'm going to have to step back at this point, since it sounds like
a problem specific to a system that I'm not readily able to visualize.
Here's an attempt to explain the issue better:
Let's say my holding disk has a throughput of about 20MB/sec. I think
everyone can agree that with 20MB/sec of throuhput from the drive and
11MB/sec to the tape, there should be no problem keeping the tape drive
streaming, assuming there isn't a CPU or bus bottleneck, and that there is
no additional disk contention for the same spindle. We've got 9MB/sec of
left-over bandwidth, which is plenty.
Now, the tech specs on drive used for my holding disk say that the average
seek time is 9ms. That means for every seek done (on average), I lose 184KB
of bandwidth.
Ok, so my disk has a total of 20MB/sec bandwidth. For the purposes of this
discussion, lets assume that the read and write bandwidths are equal. Lets
also say that the dumper (for the purposes of this discussion, we have
limited amanda to only allow one dumper at a time, not several running
concurrently, in order to avoid additional complications) is consuming about
6MB/sec on average (reasonable assumption for dumps coming from a remote
machine on a 100mbit network backbone). That leaves 14MB/sec left. To keep
the tape drive streaming (from your own specs for uncompressed writes)
requres 11MB/sec. That leaves us with 3MB/sec.
Now say that interleaving the reads and writes to the disk is causing 25
seeks per second. 25 seeks * 184KB/sec is 4.6MB/sec of throughput lost to
seek overhead, and you can see that we are now have a deficit of 1.6MB/sec.
From this example, you should be able to see that it is impossible to keep
the tape drive streaming unless either the number of seeks or the amount of
data being written by the dumper (or both) is reduced.
Does that make the issue clear? The only way to reliably fix the shoe-shine
problem (absent replacement of hardware) is to keep the dumper from using
the disk while taper is using it, or being able to throttle the dumper's use
of the disk drive to leave enough bandwidth to keep the taper from starving.
And it is appears that neither method of limiting is currently supported by
Amanda, which seems very surprising to me.
Evan
Evan
On Thu, 7 Dec 2006, Brian Cuttler wrote:
On Wed, Dec 06, 2006 at 04:33:16PM -0600, Evan Harris wrote:
Debian testing, SDLT 110/220, Intel P4 2.8Ghz 3gig RAM. Amanda 2.5.1p1-2.
The holding disk is a dedicated PATA IDE drive (master) on its own
cable/bus (no slave). The tape drive is on its own SCSI bus (no other
devices). Bonnie tests on the IDE drive give roughly 20MB/sec, and I have
no trouble keeping the tape drive streaming using dd from the IDE drive to
the tape drive. Amanda tapetest speed on the tapedrive came in right at
10MB/sec.
I have an SDLT 220 also, mine being set for high density but without
HW compression, I specifically perform SW compression (client side though
in this case the client==server). I am also peaking about 10MB/sec per
the amdump report.
I have to look at the tape specs... Sun online docs show a sustained
transfer rate of 11 MB/Sec, so you and I are both doing pretty well
in that dept.
Tell me again why you feel your drive is shoe-shining ?
Are the specs for your particular drive substantially different ?
I'm currently testing on a standalone SDLT drive first. But the final
config will be the same type of drive in an ADIC changer.
Evan
On Wed, 6 Dec 2006, Brian Cuttler wrote:
Evan,
What OS, type of tape, type of drive, HW platform ?
I'm unfamiliar with a parameter to do what your asking
for for good measure, what version of Amanda ?
Actually, how did you determine that a drive within a changer
was shoe-shining ? What else shares the bus with the tape and
with the amanda work area drive(s) ?
On Wed, Dec 06, 2006 at 02:34:31PM -0600, Evan Harris wrote:
I'm having a problem with my tape drive shoe-shining because the holding
disk can't keep up with the tape drive if it is also being written to
by a
dumper. Without the extra disk seek overhead of dumpers writing to the
holding disk at the same time, the holding disk should be plenty fast
enough to keep the tape drive streaming.
Is there any way I can force amanda to serialize the dumper/taper so
that
they are never run concurrently? I've already set inparallel to 1, but
that only affects how many dumpers can run, not the taper.
I've also tried increasing the tapebufs parameter to 8000 (256MiB) to
see
if that would at least let the drive stay streaming for longer periods,
but
if it made any difference, it wasn't significant. What thresholds does
the
taper use to decide when the tapebufs are filled enough to start tape
motion? There doesn't seem to be any docs on that, or settings to
customize.
I did get a suggestion that I should just leave the tape out of the
drive,
let the dumpers fill the holding disk and then load the tape and run
amflush, but that doesn't really work when using a changer, plus the
holding disk isn't large enough for the total size of all the backups,
though it can fit them one-by-one.
Seems like there should be a "speed" config option for holding disks
like
there is for network interfaces and tape drives, so that amanda could
test
to see if the holding disk can't handle dumpers using the holding disk
at
the same time a taper is running. That seems like it'd solve the
problem
nicely, and even seems to fit with the scheme amanda uses for network
interfaces.
Thanks.
Evan
---
Brian R Cuttler brian.cuttler AT wadsworth DOT org
Computer Systems Support (v) 518 486-1697
Wadsworth Center (f) 518 473-6384
NYS Department of Health Help Desk 518 473-0773
---
Brian R Cuttler brian.cuttler AT wadsworth DOT org
Computer Systems Support (v) 518 486-1697
Wadsworth Center (f) 518 473-6384
NYS Department of Health Help Desk 518 473-0773
---
Brian R Cuttler brian.cuttler AT wadsworth DOT org
Computer Systems Support (v) 518 486-1697
Wadsworth Center (f) 518 473-6384
NYS Department of Health Help Desk 518 473-0773
|
|
|
