BackupPC-users

Re: [BackupPC-users] Why is my rsync so much slower to do an incremental backup than tar over ssh?

2009-07-15 09:13:52
Subject: Re: [BackupPC-users] Why is my rsync so much slower to do an incremental backup than tar over ssh?
From: Carl Wilhelm Soderstrom <chrome AT real-time DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Wed, 15 Jul 2009 08:10:19 -0500
On 07/15 08:13 , gimili wrote:
> Why is my rsync so much slower to do an incremental backup than tar?

Because rsync makes checksums of all the files, instead of just checking the
timestamp.

> Have I made an error?

No, tar really is faster than rsync in some cases.

> It is hard to believe that tar can be so fast with the incremental 
> backup with that much data?
> 
> Are there any serious problems with using tar?

- tar does not catch files which have changed, but have timestamps which say
they are not changed. If the timestamp gets set to some point prior to the
last reference backup, the file won't be backed up.

- tar copies *all* the files when doing a 'full' backup. rsync only copies
the changed ones. (Like an 'incremental', but with slightly more thorough
checking).

This is a simplified explanation, but covers the basics. I'm sure others
will chime in with more details.

If tar works better for you in your environment, due to network bandwidth,
processor power available, memory availability, etc; that's why it's an
option. :)

As a noteworthy data point, when making an initial copy of files (not using
backuppc, just plain tar or rsync); tar is 2x-4x faster than rsync,
presumably due to all of rsync's calculating overhead. (Newer versions may
be faster, this was a while ago that I found this out by practical testing).
Rsync wins when making subsequent copies that don't require a large
percentage of the data to be transferred.

-- 
Carl Soderstrom
Systems Administrator
Real-Time Enterprises
www.real-time.com

------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time, 
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/