BackupPC-users

Re: [BackupPC-users] trying to improve the speed at which backuppc rsync back up processes a large binary file in incremental backups

2012-01-10 20:19:42
Subject: Re: [BackupPC-users] trying to improve the speed at which backuppc rsync back up processes a large binary file in incremental backups
From: John Habermann <jhabermann AT cook.qld.gov DOT au>
To: backuppc-users AT lists.sourceforge DOT net
Date: Wed, 11 Jan 2012 11:20:17 +1000
Hi Pedro

On Wed, 11 Jan 2012 00:53:33 +0000
"Pedro M. S. Oliveira" <pmsoliveira AT gmail DOT com> wrote:

> Are you sure that the wireless connection is capable of 11 MB, isn't
> it 11Mb?

No it is 11MB, the connection is a 5.8 Ghz point to point wireless link
which according to the manufacturers has a 150 Mbps capacity. 

> 
> Sent from my galaxy tab 10.1.
> On Jan 8, 2012 10:05 PM, "Les Mikesell" <lesmikesell AT gmail DOT com> wrote:
> 
> > On Sun, Jan 8, 2012 at 4:48 AM, John Habermann
> > <jhabermann AT cook.qld.gov DOT au> wrote:
> > >
> > > You can see that the backup of the /opt share takes nearly the
> > > total time of the incremental taking about 8 and half hours to
> > > complete while the backup of the /opt rsync share in the full
> > > backup takes about 3 and half hours. The full backup is slightly
> > > longer than what it takes if I just do a rsync over ssh copy of
> > > the file from the client server to the backup server.
> > >
> > > I have found that rsync seems to always transfer the whole file
> > > when copying this file from the client server to the backup
> > > server:
> > >
> > > # rsync -avzh --progress -e ssh
> > > administrator@isabella:ExchangeDailyBackup.bkf
> > > ExchangeDailyBackup.bkf Password:
> > > receiving incremental file list
> > > ExchangeDailyBackup.bkf
> > >      54.44G 100%   10.66MB/s    1:21:10 (xfer#1, to-check=0/1)4
> > >
> > > sent 3.31M bytes  received 3.27G bytes  486.33K bytes/sec
> > > total size is 54.44G  speedup is 16.65.
> >
> > Note that you have used the -z option with native rsync, which
> > backuppc doesn't support.  You can add the -C option to ssh to get
> > compression at that layer when you run rsync over ssh, though.
> >
> > > My questions for the list are:
> > > 1. Is it reasonable for rsync to transfer the whole file when
> > > copying a large ntbackup file?
> >
> > Yes, those files may have little or nothing in common with the
> > previous copy.   If compression or encryption are used they will
> > ensure that no blocks match and even if they aren't, the common
> > blocks may be skewed enough that rsync can't match them up.
> >
> > > 2. Why does an incremental backup of this file take so much
> > > longer than a full backup of it or a plain rsync of this file?
> >
> > That doesn't make sense to me either.  Are you sure that is
> > consistent and not related to something else that might have been
> > using the link concurrently?
> >
> > --
> >   Les Mikesell
> >     lesmikesell AT gmail DOT com
> >
> >
> > ------------------------------------------------------------------------------
> > Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a
> > complex infrastructure or vast IT resources to deliver seamless,
> > secure access to virtual desktops. With this all-in-one solution,
> > easily deploy virtual desktops for less than the cost of PCs and
> > save 60% on VDI infrastructure costs. Try it free!
> > http://p.sf.net/sfu/Citrix-VDIinabox
> > _______________________________________________ BackupPC-users
> > mailing list BackupPC-users AT lists.sourceforge DOT net
> > List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
> > Wiki:    http://backuppc.wiki.sourceforge.net
> > Project: http://backuppc.sourceforge.net/
> >


-- 
John Habermann
IT Officer
Cook Shire Council
10 Furneaux St
Cooktown 4895
ph 40820577

------------------------------------------------------------------------------
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution, easily deploy virtual 
desktops for less than the cost of PCs and save 60% on VDI infrastructure 
costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/