BackupPC-users

Re: [BackupPC-users] Tuning for disk contention

2009-09-05 21:05:01
Subject: Re: [BackupPC-users] Tuning for disk contention
From: dan <dandenson AT gmail DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Sat, 5 Sep 2009 18:52:35 -0600

       the "perfect" max load should be the number of cpus you have,
       so a quad-core server can sustain a load of 4 without any
       problem... after that number, the higher the load, the higher
       will be the performance lost

I dont agree here at all.  backuppc is not generally CPU bound.  If your target is to max out a quad core then you really need to beef up your storage devices.  Im talking 10 spindles of SAS/SCSI at 15000 RPMS in RAID10 before backuppc is going to peg a 2.4Ghz quad core.
 
       a high load and still having cpu idle is a sign of IO load.
       high system cpu usage is a sign of the kernel spending too
       much time managing the IO

       for IO load can be 99/100%, but only as long the IO wait
       doesnt increase alot... this values depends of the HD type,
       raid layout, number of disks, bus, etc

       in modern HDs (sata, scsi, sas) IO waits between 5-20 are
       normal... 100 is high load...if you see times above 150,
       that HD is probably getting too much load and is slowing
       down a lot the rest of the system (but again, this values
       depend of your OS/config/FS/HD layout)

       dont forget that writes usually put more load on a HD than
       reads, the HDs cache helps a lot, but usually write cache is
       not enabled

       use iostat (like iostat -kx 30 ) to monitor the HD loads
       and see how the io load and wait is going... sporadic
       high loads/times are normal, specially in small check times,
       but sustained high loads/times are a sign of problems

       if you want to try to get a little more from your HDs and
       the data isnt critical in case of a powerloss (usually
       backups arent, next backup would "fix" the bad data), you
       can enable the write cache.

       be aware that at least xfs doesnt like to lose data with
       powerloss+write cache, its format assumes that data hits
       the HD when it tells and can miss behave if the cache
       just decided to only really write half of the data before
       the powerloss. that is why is highly recommended to use xfs
       with a UPS and do a controlled shutdown in case of powerloss


You make a lot more sence here, but I think you overestimate CPU usage.  backuppc is so IO bound that after your get a 2Ghz+ Dual core and 2GB RAM you can pretty much blame your disks for slow performance.  I have a dual core 2Ghz Opteron with 2GB of ram and 8 drives in a linux raid10 and hard disk speed is still my bottleneck.  I run 4 concurrent backups on that machine and it does give high system load numbers but still handles the desktops in the office faster than 3 concurrent while 5 concurrent takes quite a bit longer to complete. filesystem choice and io scheduler do make a difference but faster disks is the only real cure.


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/