BackupPC-users

Re: [BackupPC-users] trying to improve the speed at which backuppc rsync back up processes a large binary file in incremental backups

2012-01-08 17:02:10
Subject: Re: [BackupPC-users] trying to improve the speed at which backuppc rsync back up processes a large binary file in incremental backups
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Sun, 8 Jan 2012 16:00:02 -0600
On Sun, Jan 8, 2012 at 4:48 AM, John Habermann
<jhabermann AT cook.qld.gov DOT au> wrote:
>
> You can see that the backup of the /opt share takes nearly the total
> time of the incremental taking about 8 and half hours to complete while
> the backup of the /opt rsync share in the full backup takes about 3 and
> half hours. The full backup is slightly longer than what it takes if I
> just do a rsync over ssh copy of the file from the client server to the
> backup server.
>
> I have found that rsync seems to always transfer the whole file when
> copying this file from the client server to the backup server:
>
> # rsync -avzh --progress -e ssh
> administrator@isabella:ExchangeDailyBackup.bkf ExchangeDailyBackup.bkf
> Password:
> receiving incremental file list
> ExchangeDailyBackup.bkf
>      54.44G 100%   10.66MB/s    1:21:10 (xfer#1, to-check=0/1)4
>
> sent 3.31M bytes  received 3.27G bytes  486.33K bytes/sec
> total size is 54.44G  speedup is 16.65.

Note that you have used the -z option with native rsync, which
backuppc doesn't support.  You can add the -C option to ssh to get
compression at that layer when you run rsync over ssh, though.

> My questions for the list are:
> 1. Is it reasonable for rsync to transfer the whole file when copying a
> large ntbackup file?

Yes, those files may have little or nothing in common with the
previous copy.   If compression or encryption are used they will
ensure that no blocks match and even if they aren't, the common blocks
may be skewed enough that rsync can't match them up.

> 2. Why does an incremental backup of this file take so much longer than
> a full backup of it or a plain rsync of this file?

That doesn't make sense to me either.  Are you sure that is consistent
and not related to something else that might have been using the link
concurrently?

-- 
   Les Mikesell
     lesmikesell AT gmail DOT com

------------------------------------------------------------------------------
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution, easily deploy virtual 
desktops for less than the cost of PCs and save 60% on VDI infrastructure 
costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/