BackupPC-users

Re: [BackupPC-users] Really slow with rsync, even on incrementals

2011-02-23 11:42:16
Subject: Re: [BackupPC-users] Really slow with rsync, even on incrementals
From: John Goerzen <jgoerzen AT complete DOT org>
To: backuppc-users AT lists.sourceforge DOT net
Date: Wed, 23 Feb 2011 16:39:57 +0000 (UTC)
Michael Stowe <mstowe <at> chicago.us.mensa.org> writes:

> Another way to put this is that you'd like BackupPC to overcome
> performance problems caused by your amazingly slow I/O.  Not that I'm
> arguing against rewriting BackupPC to accomodate slow media, but I'm
> certainly not going to bother, since it's certainly not slow for me.

Well, I don't think that's entirely accurate.  BackupPC is on the order of 10
times slower than alternatives making full backups after the first.  It is on
the order of 20-50 times slower than alternatives making incrementals.  This
referring to the same disk setup.

The order of magnitude here won't go away with faster disks.  If, say, the disks
are twice as fast, then it could still be 10 times slower but maybe that
wouldn't be as much a problem.

I've also noted performance problems using BackupPC+rsync to back up 25GB files,
and that on enterprise server-class hardware, not my personal USB drive at home.

I believe I've found a design issue in how BackupPC uses rsync, but I'm not
clever enough yet to work out what it is or if/how I could work around it.

> Might I suggest that you back up to something quicker, and then copy it
> over to your USB drive?  (If that's too slow, than I fail to see how
> BackupPC can do any better.)

I don't expect that to be worth the expense.  In other words, half of 55 hours
is still too long.

The only thing I think that can improve this is switching to tar, but then I
have the problem I'm mentioning in the other thread.

-- John



------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT data 
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/