Re: [BackupPC-users] Really slow with rsync, even on incrementals

Bowie Bailey <Bowie_Bailey AT BUC DOT com> wrote on 02/23/2011 12:07:41 PM: > It seems like something in your setup is causing problems. BackupPC > should not be that slow. I use rsync on most of my backups and do not > see anything like this. > > My largest machine: > > 314GB > 1.8 million files > Full backup: 10 hours > Inc backup: 1 hour

This parallels fairly closely with the numbers I see (and wrote about separately).

> And these machines are nothing special. The backup server is an Athlon > 64 1Ghz w/ 1G ram. The server being backed up is a P4 2.5Ghz w/ 512M > ram. Of course, I don't have USB drives or filesystem encryption > slowing me down.
I must have missed the beginning of this thread: is the original poster using *both* USB drives (which I knew about) *and* filesystem-level encryption? Wow.

USB is very CPU-intensive. The host CPU must poll the bus constantly. I strongly suggest "upgrading" to eSATA. It's still cheap, and at least you won't have the USB overhead.

Encryption is *very* CPU-intensive. For example, from what I've seen (mostly with scp), 1MB/s is a *good* amount of data transfer while using encryption! That's great for your average home-drive or flash drive type storage, but BackupPC lives and dies by throughput...

Are you also using compression? Good luck. Compression is almost as CPU intensive as encryption! (I forgot to note in my stats previously that I do *not* use compression on BackupPC: I want my backups as simple as possible.)

I think you might want to start a *LOT* simpler. Get rid of a lot of the extra layers, even if temporarily. Start with a simple server: 1 SATA drive, 1GB RAM, 1GHz processor and 1 Gb interface. See how that performs--I think you will be pleasantly surprised. Then start adding layers: move to a USB drive. See how that changes things. Then add encryption. See where the bottleneck is.

And remember: there are *lots* of others seeing results *very* different from yourself. Try to find out why.

Timothy J. Massey


Out of the Box Solutions, Inc. Creative IT Solutions Made Simple! http://www.OutOfTheBoxSolutions.com tmassey AT obscorp DOT com		22108 Harper Ave. St. Clair Shores, MI 48080 Office: (800)750-4OBS (4627) Cell: (586)945-8796

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT data 
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev

_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/