BackupPC-users

Re: [BackupPC-users] Why is my rsync so much slower to do an incremental backup than tar over ssh?

2009-07-15 09:21:41
Subject: Re: [BackupPC-users] Why is my rsync so much slower to do an incremental backup than tar over ssh?
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Wed, 15 Jul 2009 08:18:13 -0500
gimili wrote:
> I am new at backuppc.  I am backing up 250GB from one machine to another 
> with backuppc.  Both machines use debian lenny as the os.  2.4ghz 
> machines with 4GB ram in client and 2GB in backuppc machine.  Both 
> drives are sata.
> 
> Initially I used rsync and it took 7 hours for a full backup and 5 hours 
> for an incremental.  I did every tweak I could find on to improve speed.
> 
> I switched to tar and it still took 7 hours for a full backup but it 
> only took 10minutes for an incremental.  I modified a few files to make 
> sure that it was working.  The modified files were indeed backed up.
> 
> Why is my rsync so much slower to do an incremental backup than tar?

The backuppc side is running a perl implementation of rsync and working against 
a compressed copy of the files.  It actually compares the previous full run to 
the existing files so it knows about deletions, old files under renamed 
directories, etc.  Tar just uses the timestamp on incrementals.

> Have I made an error?

Some difference is normal - but that seems somewhat extreme unless your 250 
gigs 
is millions of tiny files.  It would probably help to add RAM to the backuppc 
server.  I think there is also some quirk about the --checksum-seed=32761 that 
doesn't take effect on the first run but I've forgotten the details - and it 
shouldn't matter on incrementals where most files haven't changed anyway.

> It is hard to believe that tar can be so fast with the incremental 
> backup with that much data?

There's a lot less work involved if you don't actually have many new/changed 
files.

> Are there any serious problems with using tar?

There's more network traffic, especially for the subsequent full runs and the 
incrementals will not pick deletions or files that for various reasons have old 
timestamps but are new or in new locations.  Those may or may not be serious 
problems for you.

-- 
   Les Mikesell
    lesmikesell AT gmail DOT com


------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time, 
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/