BackupPC-users

Re: [BackupPC-users] Why is my rsync so much slower to do an incremental backup than tar over ssh?

2009-07-15 11:56:00
Subject: Re: [BackupPC-users] Why is my rsync so much slower to do an incremental backup than tar over ssh?
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Wed, 15 Jul 2009 10:49:48 -0500
Carl Wilhelm Soderstrom wrote:
> 
>> The slow part of the process should be creating the new 
>> directory tree and file structure.  Rsync shouldn't do a lot of 
>> calculating when the target is empty, but it does (at least the old 
>> versions) read and transfer the entire source directory tree before 
>> starting any file transfers which can add some time to the operation 
>> particularly where there are a lot of small files.
> 
> Interesting. I might have vaguely known some of that but never correlated
> it. Thank you. I think I can see how it might be a bit slower to create the
> directory trees then transfer the files, rather than create the files and
> indices in a more sequential fashion.

I don't think rsync actually creates the directory tree ahead of the 
transfer - in fact I'm pretty sure it doesn't based on what is left 
after an incomplete transfer.   It just loads the remote directory 
structure into RAM before starting to walk through it doing the 
comparison to what is already there.  If nothing already exists, the 
comparison won't actually be needed but it still waits to get the whole 
tree before starting to request files where tar would start writing as 
soon as the first thing comes over.

-- 
   Les Mikesell
    lesmikesell AT gmail DOT com


------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time, 
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/