-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/01/12 09:00, Les Mikesell wrote:
> On Sun, Jan 8, 2012 at 4:48 AM, John Habermann 
> <jhabermann AT cook.qld.gov DOT au> wrote:
>> 
>> You can see that the backup of the /opt share takes nearly the
>> total time of the incremental taking about 8 and half hours to
>> complete while the backup of the /opt rsync share in the full
>> backup takes about 3 and half hours. The full backup is slightly
>> longer than what it takes if I just do a rsync over ssh copy of
>> the file from the client server to the backup server.
>> 
>> I have found that rsync seems to always transfer the whole file
>> when copying this file from the client server to the backup
>> server: My questions for the list are: 1. Is it reasonable for
>> rsync to transfer the whole file when copying a large ntbackup
>> file?
> 
> Yes, those files may have little or nothing in common with the 
> previous copy.   If compression or encryption are used they will 
> ensure that no blocks match and even if they aren't, the common
> blocks may be skewed enough that rsync can't match them up.

Definitely, I find that transferring (for example) uncompressed
mysqldump files will transfer less bytes than transferring the
compressed files. (Obviously because the compressed file will transfer
100%, while the uncompressed will only transfer the changes).

>> 2. Why does an incremental backup of this file take so much
>> longer than a full backup of it or a plain rsync of this file?
> That doesn't make sense to me either.  Are you sure that is
> consistent and not related to something else that might have been
> using the link concurrently?

I'm not exactly sure, but I've found the best way to approach these
types of files (any dump/export from another application, MS SQL,
MySQL, custom applications, etc), is to:
1) Ensure the file is uncompressed if it is in plain text format of
some sort, which may mean uncompressing it after the other application
creates it, and before backuppc will see it. The size of chunks can
depend on transfer speeds available, total file size, etc...

2) Any file over about 100M, I split into chunks (using split) of
anything from 20M to 100M chunks. I've found that rsync will transfer
these chunks much more quickly than transferring the whole file. Also,
if the transfer is interrupted (on a full backup), it will save the
backup as a partial, and continue from the last chunk (as opposed to
re-starting that 5G file, you only lose a maximum of 100M).

Regards,
Adam

- -- 
Adam Goryachev
Website Managers
www.websitemanagers.com.au
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8PqjwACgkQGyoxogrTyiVi6ACdGNld8qhmEeLqvX6HSQMwLhcS
amQAmwegtCByLBo4dXrXEbKpMqrHuSJO
=tBKs
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
RSA(R) Conference 2012
Mar 27 - Feb 2
Save $400 by Jan. 27
Register now!
http://p.sf.net/sfu/rsa-sfdev2dev2
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/