On Sunday 14 January 2007 14:03, Patrick M. Hausen wrote:
>Hi, all!
>
>We installed a disk based backup system a couple of months
>ago, because we have been hitting the capacity of our tape
>changer on almost every nightly run - and we did not even
>backup everything. So a new system was in called for.
>
>The backup server features 2 GB of main memory, a reasonably
>fast CPU (2.8 GHz Xeon) and 4 TB of storage in two separate
>RAID 5 sets.
>
>In our primary Amanda instance we have 200 GB sized vtapes,
>200 GB of holding disk space (holding disk and vtapes on different
>RAID sets). The compressed nightly backup data are frequently less
>than 100 GB and never more than 130 GB. We have 95 DLEs on 35 hosts.
>
>When we started, we had only one RAID set, so I did not configure
>a holding disk. With the current architecture of Amanda this
>essentially means "inparallel 1". The backups took about 20 hours
>after all 95 DLEs were added.
>
>Since we needed more disk space and a second Amanda instance anyway,
>I doubled the disk space by buying more disks and ended up with two
>identical RAID 5 sets. With the machine we have there have to be
>two sets, because there are two separate controllers connected
>to 6 SATA disks each.
>
>This bought us a holding disk and parallelism. So I hoped. Now
>the backups take 12 to 13 hours per nightly run. Sometimes 8
>if there is not much to backup.
>
>The config changes I made to achieve this were:
>
>inparallel 10
>maxdumps 1
>netusage 12500 kbps
>
>holdingdisk hd1 {
> directory "/raid1/holdingdisk"
> use 204800 mbytes
>}
>
>But I'm not satisfied, yet ;-)
>
>Example:
>
>Original Size (meg) 218907.0
>Avg Dump Rate (k/s) 1158.9
>Tape Time (hrs:min) 1:08
>
>This run took about 13 hours. About one hour of this is tape time.
>So the rest is needed to transfer the data from the clients to the
> server.
>
>Since the server doesn't do anything but backups, I'd like to max
>out CPU and network usage on the server. Therefore I compress everything
> on the server besides 4 hosts that are located in a remote datacenter
> where we have to pay for data transferred.
Patrick; While hardware differences might make that a workable option, I
generally tend to think that in the case of many clients, it is better to
do the compression on the clients because you can have as many of them
running in parallel as there are disk spindles to account for.
These drives (or I believe this could also be used to designate the whole
machine too although amanda may do this already) should be assigned a
number in your disklist so that amanda can then run backups from spindles
1, 4, 5, 9, 10, 15, 17, 13, 21 and 7. Even if these targets are only
500mhz machines, they will be, with 10 of them all munching along at the
same time, cumulatively 4 or 5x faster than a single server box trying to
compress the data after the network transmission. So you save network
bandwidth in the ratio of the compression achieved which also cuts the
total time the network pipes are filled to 100%.
I believe this can scale well, particularly when dealing with a disk raid
as the storage medium rather than the much slower, and sequential, tape
funnel it all has to pour through.
I was for a time, backing up a firewall box too, a 500mhz machine that I
both backed up, and mirrored to this box, and the backup times were not
more than 10-15 minutes longer for almost double the data, and I was
doing the clients compression on the clients box.
Now its just this box, and I'm done in anything from an hour to 3 hours
maximum depending on how much compression it needs to do. And I don't
bother to compress that which isn't very compressable & only gets shrunk
to 80% of its original size for a level 0. OTOH, some stuff will squeeze
to less than 10% of its original size, so that is a worthwhile effort.
You will quickly find out what can be smunched, and what isn't worth the
effort from the reports amanda will email you.
>All other backup clients have a dedicated 100 Mbit/s connection to the
>backup server. My goal is to have the network almost saturated.
>If you use a calculator, then with a 100 Mbit/s network the 200+
>GB of the above run should be transferred in less than 5 hours.
>If I didn't make a stupid mistake here.
>
>The CPU compressing the data runs at about 60% idle when busy compresing
>data and almost completely idle the rest of the time.
>
>So how do I go about identifying the bottlenecks? Just crank up
>"inparallel"? It must be possible to get down to, say, half of the
>13 hours, IMHO.
I believe the inParallel increases will only help to the extent that
defining the spindle numbers will allow. But I've been wrong before so
do your own investigation(s). I at the moment, don't have the hardware
to test the theory.
>
>Thanks a lot for any hints,
>Patrick
--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2007 by Maurice Eugene Heskett, all rights reserved.
|
