BackupPC-users

Re: [BackupPC-users] tar vs. rsync benchmark and limitation test results

2011-02-24 23:29:45
Subject: Re: [BackupPC-users] tar vs. rsync benchmark and limitation test results
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: backuppc-users AT lists.sourceforge DOT net
Date: Thu, 24 Feb 2011 22:27:49 -0600
On 2/24/11 9:34 PM, John Goerzen wrote:
>
> Next, I used mv to rename a file to a different name in the same
> directory and then ran another incremental.
>
> In this case, BackupPC noticed the file with the new name and backed it
> up.  It did not notice that the old file had gone away, which is
> somewhat as expected.  ls -lc confirmed that mv changed the ctime on the
> file.

If you rename a directory, you won't get the old contents underneath in their 
new locations either.  You'll still have the backup in the old path but might 
not know where to find it.

> Somewhat surprising was that rsync checksum caching provided only a
> marginal benefit (a reduction from 24.0 to 22.6 minutes for a full
> backup).  It is possible that the data set in question here (vast
> numbers of small files) is not good and displaying the benefit of
> checksum caching.

The client side still has to do a full read - probably mostly constrained by 
directory accesses and seeks on small files.

> The files in this test case did not demonstrate the other pathological
> problem with BackupPC's rsync algorithm, that of taking 10+ hours to
> back up a changed 25GB file.  Had such a file been involved, the tar
> backup would certainly have been many orders of magnitude faster than rsync.

You might do better with the --whole-file option on rsync if your server is 
slow 
doing the delta/merge operations.

>    * The rsync backend for BackupPC is probably not useful unless
> Internet backups or small backup sets to fast disks are involved.

That's perhaps an overstatement. But you do need relatively fast hardware on 
the 
server side.

>    * The "limitations" of the tar backend have been exaggerated, at least
> for backing up Linux systems with using POSIX-obeying filesystems with
> GNU tar.  (vfat under Linux may still exhibit the limitations
> documented, for instance.)

I think the docs combine the smb/tar description for this, and you would see 
the 
effect as described when using smb or tar on a client filesystem that doesn't 
support different mtime/ctime values.

> Does it point to any issues in BackupPC that are easily fixable?

Not unless someone wants to tackle what would have to change to not add the 
--ignore-times option on all full rsync runs.

-- 
    Les Mikesell
     lesmikesell AT gmail DOT com


------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT data 
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>