Amanda-Users

Re: amdump takes too long for level 0

2004-09-30 23:03:32
Subject: Re: amdump takes too long for level 0
From: Gene Heskett <gene.heskett AT verizon DOT net>
To: "Nadeem, Sarwar" <Sarwar.Nadeem AT parsons DOT com>
Date: Thu, 30 Sep 2004 22:59:54 -0400
On Thursday 30 September 2004 17:35, Nadeem, Sarwar wrote:
>Hi There,
>
>I am new to amanda. We are running Amanda version 2.4.1p1 on a
> Solaris 5.6 box. For one of my filesystem which around 82GB in size
> (22GB after compression) the amdump takes 15 hours to do a level 0
> backup....
>
>
>
>These dumps were to tape DAILY1-13.
>Tonight's dumps should go onto 3 tapes: DAILY1-13, DAILY1-14,
> DAILY1-15.

And you are going to re-use DAILY1-13 again tonight?  Thats most odd.  
What are your dumpcycle, tapecycle and runspercycle in your 
amanda.conf?

>
>STATISTICS:
>                          Total       Full      Daily
>                        --------   --------   --------
>Dump Time (hrs:min)       15:40      15:39       0:00   (0:00 start,
> 0:00 idle) Output Size (meg)       21833.8    21833.8        0.0
>Original Size (meg)     84033.1    84033.1        0.0
>Avg Compressed Size (%)    26.0       26.0        --
>Tape Used (%)              31.2       31.2        0.0
>Filesystems Dumped            1          1          0
>Avg Dump Rate (k/s)       396.6      396.6        --
>Avg Tp Write Rate (k/s)   396.6      396.6        --
>
>
>NOTES:
>  taper: tape DAILY1-13 kb 22357792 fm 1 [OK]
>
>
>DUMP SUMMARY:
>                                      DUMPER STATS                 
> TAPER STATS HOSTNAME  DISK           L  ORIG-KB   OUT-KB COMP% 
> MMM:SS   KB/s  MMM:SS   KB/s --------------------------
> -------------------------------------- -------------- rockhead 
> -rabackup/APP1 0 86049920 22357760  26.0  939:27  396.6  939:28 
> 396.6
>
>(brought to you by Amanda version 2.4.1p1)
>
>
>
>Is this normal?
>
>We are using a DLT7000 drive to do this dump.
>
>
>-cheers
>Sarwar

And you are using software compression I assume.

Reading between the lines a bit, and bear in mind solaris is Jon 
LaBadies territory (linux here), my guess is that the box might not 
have the horsepower to do all that compression in a timely manner. 
And that version of amanda is now quite long in the tooth but I don't 
think an update will fix the speed.  An upgrade should be seemless 
however if you can recover the original configuration arguments and 
use them again.

Are you using a holding disk big enough to cache a backup of that 
size?  You would need about 30GB, and a very small reserve setting in 
your amanda.conf, like "reserve -500MB" or thereabouts.

If there isn't enough space there, amanda goes into a one file at a 
time mode, writing directly to the drive.  The drive will tell you 
about that by doing a lot of "shoe shining", eg stopping because its 
out of data, then backing up and re-cueing then starting back up when 
the buffers are full again.  Thats needless wear and tear on both the 
drive and the tape for obvious reasons.

Are you doing the compression on the server?, as opposed to offloading 
it to the client machines, where it can be done by all clients in 
parallel therefore speeding things up some.  This can have a 
noticeable impact on the backup speeds all by itself as the server, 
while able to multitask ok, may not have the memory or horsepower to 
run several copies of gzip simultainiously.  gzip is a cpu hog even 
here on an Athlon 2800XP. :)

I have a buffers=80 line in my amanda.conf, 4 times the default 640k 
it would use, and could probably set it for 160 or even 320 if I 
wanted as there is 1GB of ram in this box.  You should probably up 
that some, depending on how much ram is in the server. (or the client 
if its doing the compression)

Many of use are using a rather lengthy disklist, with the idea of not 
having more than 600 megs or so in a single entry, and some quite a 
bit smaller than that.

This has 3 advantages:

1) If there is more than one physical drive involved, use the spindle 
number option in the disklist as amanda can access 2 or more paths at 
a time provided they are on different disks.

2) Having smaller entries means amanda can juggle the schedule with an 
eye toward putting the same amount of data on a tape each night.  
Back when I was using DDS2 tapes, amanda regularly filled one to 98% 
capacity when the dumpcycle and such were properly adjusted and 
allowed to settle for a couple of dumpcycles.

3) By having numerous entries, you can turn the compression off for 
those subdirs that contain rpms, tar.gz, or tar.bz2's.  You don't 
lose that much compression, maybe 2%, and you don't waste lots of 
time trying to squeeze that last drop of blood out of a turnip.bz2 
file.

Look at the emails amanda sends you and adjust the use of compression 
accordingly by changing the dumptype.  Wasteing time on a disklist 
entry that only compresses to 90% is not worth the effort.

Anyway, come on back with a few more details & somebody here can 
probably suggest something that should help.  The above is just a 
list of suggestions, any one of which, or all combined, might cut the 
backup time by several hours.  We'll look at that and then see if you 
still need bigger iron.

How fast is that DLT7000 rated for?

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.26% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attorneys please note, additions to this message
by Gene Heskett are:
Copyright 2004 by Maurice Eugene Heskett, all rights reserved.

<Prev in Thread] Current Thread [Next in Thread>