Re: [BackupPC-users] backuppc slow rsync speeds
2012-09-18 11:29:18
Les Mikesell <lesmikesell AT gmail DOT com> wrote on 09/17/2012
02:44:20 PM:
> On Mon, Sep 17, 2012 at 11:54 AM, Timothy J Massey
> <tmassey AT obscorp DOT com> wrote:
> >
> > However, I have recently inherited a server that is >3TB big,
and 97%
> > full, too! Backups of that system take 3.5 *days* to complete.
I *can't*
> > live with that. I need better performance.
>
<Quick fixes snipped>
> In any case you have to keep in mind that if
you ever
> have to restore, there will be considerable downtime. For the
price
> of a disk these days, it might be worth keeping a copy up to date
with
> native rsync that you could just swap into place if you needed it
and
> back that up with backuppc if you want to keep a history.
Fortunately, BackupPC is a "backup of the backup"
right now, and is not expected to be used for real. Yet. That's
why I can take the time and try to actually solve the problem, rather than
apply band-aids.
But that will likely end in November, if not sooner.
> > It looks like everything is under-utilized.
For example, I'm getting a
> > measly 40-50MB of read performance from my array of four drives,
>
> If every read has to seek many tracks (a likely scenario), that's
not
> unreasonable performance.
Which is why I asked if others were getting better
performance. They are. So it's not just inherent to the task.
There's something different.
> My physical drive and network
> > lights echo this: they are *not* busy. My interrupts
are certainly
> > manageable and context switches are very low. Even my CPU
numbers look
> > tremendous: nearly no time in wait, and about 50% CPU idle!
>
> I think you should be seeing wait time. Unless perhaps you have
some
> huge files that end up contiguous on the disk, I'd expect the CPU
to
> be able to decompress and checksum as fast as the disk can deliver
-
> and there shouldn't be much other computation involved.
I'm not. I've got two of these new boxes built.
In both cases, they have 2-4% wait time when doing a backup. One
is a RAID-5 and one is a RAID-6.
Might it have something to do with md? Could
the time that would normally be considered wait time for BackupPC be counted
as CPU time for md? That doesn't seem logical to me, but I can say
that there just isn't any wait time on these systems.
> Mine seem to track the target host disk speed more than anything else.
> The best I see is 208GB with a full time of 148 minutes.
But that
> is with backuppc running as a VM on the East coast backing up a target
> in California and no particular tuning for efficiency. Compression
is
> on and no checksum caching.
That's the same settings I'm using. But that's
about double the performance I'm getting. 247GB in 340 minutes, or
about 12MB/s.
I've just turned compression off for a couple of hosts.
We'll see how this affects their performance. I'll let everyone
know.
Timothy J. Massey
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
<Prev in Thread] |
Current Thread |
[Next in Thread>
|
Re: [BackupPC-users] backuppc slow rsync speeds, Les Mikesell
- Re: [BackupPC-users] backuppc slow rsync speeds,
Timothy J Massey <=
- Re: [BackupPC-users] backuppc slow rsync speeds, Les Mikesell
- Re: [BackupPC-users] backuppc slow rsync speeds, Timothy J Massey
- Re: [BackupPC-users] backuppc slow rsync speeds, Les Mikesell
- Re: [BackupPC-users] backuppc slow rsync speeds, Timothy J Massey
- Re: [BackupPC-users] backuppc slow rsync speeds, Les Mikesell
- Re: [BackupPC-users] backuppc slow rsync speeds, Timothy J Massey
- Re: [BackupPC-users] backuppc slow rsync speeds, Les Mikesell
- Re: [BackupPC-users] backuppc slow rsync speeds, John Rouillard
- Re: [BackupPC-users] backuppc slow rsync speeds, Les Mikesell
- Re: [BackupPC-users] backuppc slow rsync speeds, John Rouillard
|
|
|