BackupPC-users

Re: [BackupPC-users] Matching files against the pool remotely.

2009-12-18 10:16:46
Subject: Re: [BackupPC-users] Matching files against the pool remotely.
From: Malik Recoing. <malik AT lawek DOT com>
To: backuppc-users AT lists.sourceforge DOT net
Date: Fri, 18 Dec 2009 15:12:11 +0000 (UTC)
Tim Connors <tim.w.connors <at> gmail.com> writes:
> 
> On Fri, 18 Dec 2009, Malik Recoing. wrote:
> 
> > I know a file will be skiped if it is present in the previous backup, but
> > what
> > appens if the file have been backed up for another host ?
> 
> It is required to be uploaded first as otherwise there's nothing to
> compare it to (yeah, I know, that's a pain[1]).
> 
> It might theoretically be sufficient to let the remote side calculate a
> hash and compare it against the files in the pool with matching hashes,
> and then let rsync do full compares against all the matching hashes in the
> pool (since hash collisions happen), but I don't believe anyone has tried
> to code this up yet, and it would only be of limited uses in systems that
> were network bandwidth constrained rather than disk bandwidth constrained.

I'm quite sure it will be an improvement for both. Globaly there will be no
overhead. More : the hash calculation will be kind of "clustered" delegating it
to the client. The matching of identical hash is anyway done by BackupPC_Link.
Thus BackupPC_Link will became pointless in a "rsync-only" configuration. The
disk and the network trafic will be reduced as many files won't be transfered at
all.

If such a feature exists, it will give BackupPC a "magic" touch, backing up a
wole tree of well known files in a minute even over a slow network. 

What a pity I'm not fluent with perl...


> [1] I just worked around this myself by copying a large set of files onto
> sneakernet (my USB key), copying them onto a directory on the local backup
> server, backing that directory up, then moving the corresponding directory
> in the backup tree into the previous backup of the remote system, so it
> will be picked up and compared against the same files when that remote
> system is next backed up.  I find out tomorrow whether that actually
> worked :)
> 

I tougth of a similar solution. When your client are mostly "full system tree"
backups, you may have ready-to-copy backups of the differents OS tree. When a
new client is added, you copy the corresponding OS directory as it was the first
full backup.

Malik.





------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev 
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/