On Sun, Jan 14, 2007 at 08:03:32PM +0100, Patrick M. Hausen wrote:
> Hi, all!
>
> We installed a disk based backup system a couple of months
> ago, because we have been hitting the capacity of our tape
> changer on almost every nightly run - and we did not even
> backup everything. So a new system was in called for.
>
> The backup server features 2 GB of main memory, a reasonably
> fast CPU (2.8 GHz Xeon) and 4 TB of storage in two separate
> RAID 5 sets.
>
> In our primary Amanda instance we have 200 GB sized vtapes,
> 200 GB of holding disk space (holding disk and vtapes on different
> RAID sets). The compressed nightly backup data are frequently less
> than 100 GB and never more than 130 GB. We have 95 DLEs on 35 hosts.
>
> When we started, we had only one RAID set, so I did not configure
> a holding disk. With the current architecture of Amanda this
> essentially means "inparallel 1". The backups took about 20 hours
> after all 95 DLEs were added.
>
> Since we needed more disk space and a second Amanda instance anyway,
> I doubled the disk space by buying more disks and ended up with two
> identical RAID 5 sets. With the machine we have there have to be
> two sets, because there are two separate controllers connected
> to 6 SATA disks each.
>
> This bought us a holding disk and parallelism. So I hoped. Now
> the backups take 12 to 13 hours per nightly run. Sometimes 8
> if there is not much to backup.
>
> The config changes I made to achieve this were:
>
> inparallel 10
> maxdumps 1
> netusage 12500 kbps
>
> holdingdisk hd1 {
> directory "/raid1/holdingdisk"
> use 204800 mbytes
> }
>
> But I'm not satisfied, yet ;-)
>
> Example:
>
> Original Size (meg) 218907.0
> Avg Dump Rate (k/s) 1158.9
> Tape Time (hrs:min) 1:08
>
> This run took about 13 hours. About one hour of this is tape time.
> So the rest is needed to transfer the data from the clients to the server.
>
> Since the server doesn't do anything but backups, I'd like to max
> out CPU and network usage on the server. Therefore I compress everything on
> the server besides 4 hosts that are located in a remote datacenter where
> we have to pay for data transferred.
>
> All other backup clients have a dedicated 100 Mbit/s connection to the
> backup server. My goal is to have the network almost saturated.
> If you use a calculator, then with a 100 Mbit/s network the 200+
> GB of the above run should be transferred in less than 5 hours.
> If I didn't make a stupid mistake here.
I doubt you will be able to approximate 100 Mb/s continuous
throughput.
>
> The CPU compressing the data runs at about 60% idle when busy compresing
> data and almost completely idle the rest of the time.
>
> So how do I go about identifying the bottlenecks? Just crank up
> "inparallel"? It must be possible to get down to, say, half of the
> 13 hours, IMHO.
Your reports should show two things of interest;
- time for estimates (and planning)
There are several ways to reduce this
- percentage of dump time when you had idle dumpers and how many
This will tell you whether you are already using your dumpers
and whether adding more might help
Some people find amplot helpful in analyzing bottlenecks.
You might consider uping maxdumps to 2 to let clients dump more
than one DLE simultaneously if server dumpers are available.
If you do this disk contention on clients with 2 DLE's on the
same disk drive could be reduced by use of spindle numbers
in the disklist.
--
Jon H. LaBadie jon AT jgcomp DOT com
JG Computing
4455 Province Line Road (609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)
|
