BackupPC-users

Re: [BackupPC-users] Tuning for disk contention

2009-09-06 21:01:27
Subject: Re: [BackupPC-users] Tuning for disk contention
From: higuita <higuita AT GMX DOT net>
To: backuppc-users AT lists.sourceforge DOT net
Date: Mon, 7 Sep 2009 01:57:45 +0100
Hi

On Sat, 5 Sep 2009 18:52:35 -0600, dan <dandenson AT gmail DOT com> wrote:
> >        the "perfect" max load should be the number of cpus you have,
> >        so a quad-core server can sustain a load of 4 without any
> >        problem... after that number, the higher the load, the higher
> >        will be the performance lost
> I dont agree here at all.  backuppc is not generally CPU bound.  If
> your target is to max out a quad core then you really need to beef up your

        i'm not saying that backuppc is cpu bound, i'm describing what
        is the "perfect" load, as he asked...
        the "s in the perfect is because its not the only metric needed
        to check how a machine is going, but is one of the most important.

        the load is the number of process requiring CPU during a set of
        time, but isnt just equal to the % of cpu usage, other 
        operations, like IO, require cpu slots even if they dont really
        use cpu (DMA transfers and like). If load=# of cpus, no process
        is waiting for cpu slot, when load > # cpus, you have process 
        that will be waiting for cpu slot in each cpu cycle... 
        the higher the load, the more process will be stopped, waiting
        for their turn to use a cpu slot

        He have a load of 9, we dont know how much cpu he have, but
        even if its a quad-core, seems to me a high load, so he is 
        probably running too much concurrent backups at same time.
        for sure the problem is in the IO, not in the cpu, but the 
        load shows that is doing more work than he can manage.

        Solving the bottleneck (in this case is adding more disks for
        sure) will decrease the load, as each operation will terminate
        sooner and freeing the cpu slots for other process
        
 
> You make a lot more sence here, but I think you overestimate CPU usage.

        see above :)

> a dual core 2Ghz Opteron with 2GB of ram and 8 drives in a linux raid10
> and hard disk speed is still my bottleneck.  I run 4 concurrent backups
> on that machine and it does give high system load numbers but still
> handles the desktops in the office faster than 3 concurrent while 5
> concurrent takes quite a bit longer to complete. 

        how much is the "high system load"?

        have you checked the 15 minutes load average with 3, 4 and
        5 concurrent backups?
        i suspect that with 3, you have a load around 1.5, with 4, 
        a load around 2.5, with 5, a load around 4.5... that is, you
        are getting a higher jump from 4 to 5 than from 3 to 4

        having a load of 6 during one minute isnt important, if its
        just one minute

        but again, load isnt the only metric, iostat gives the rest 
        of the info needed.

> filesystem choice and
> io scheduler do make a difference but faster disks is the only real
> cure.

        faster HD or more disk to spread the load for more HDs...
        for most people, this is true
cya
higuita
-- 
Naturally the common people don't want war... but after all it is the
leaders of a country who determine the policy, and it is always a 
simple matter to drag the people along, whether it is a democracy, or
a fascist dictatorship, or a parliament, or a communist dictatorship.
Voice or no voice, the people can always be brought to the bidding of
the leaders. That is easy. All you have to do is tell them they are 
being attacked, and denounce the pacifists for lack of patriotism and
exposing the country to danger.  It works the same in every country.
           -- Hermann Goering, Nazi and war criminal, 1883-1946

Attachment: signature.asc
Description: PGP signature

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/