BackupPC-users

Re: [BackupPC-users] more efficient: dump archives over the internet or copy the whole pool?

2010-11-02 17:50:56
Subject: Re: [BackupPC-users] more efficient: dump archives over the internet or copy the whole pool?
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: backuppc-users AT lists.sourceforge DOT net
Date: Tue, 02 Nov 2010 16:48:55 -0500
On 11/2/2010 4:22 PM, Frank J. Gómez wrote:
> Thanks for your response, Les.
>
> Regarding the hardlinks, I was thinking (perhaps incorrectly) that since
> I'd be putting an encrypted tar.gz on S3 (rather than all the individual
> files) that the hardlinking wouldn't be an issue and that the
> non-redundancy would be preserved in the tar.

The part that has to be preserved are the hardlinks from the /pc/ tree 
to the files in the (c)pool.  You'd have to make sure nothing changed 
for the duration of the copy and take the whole thing at once.  Plus, 
tar is not particularly efficient at handling hardlinks and worse when 
restoring since it has to end up doing a linear scan of the inode table 
to match each one up.

> I don't see anything in the Brackup documentation about how incremental
> backups with encryption are supported -- did you?  I've found that
> attempts to use encryption with incrementals generally fail because
> the md5sum of your original files don't match the md5sum of the
> encrypted files, so it's impossible to know what has and hasn't changed.

I haven't looked at the implementation at all, but since it does have an 
incremental concept I assume it keeps some sort of mapping of hash 
values to file chunks that it uses locally to know what is already 
stored (and hopefully can reconstruct them if you lose the local map and 
need that disaster recovery).

> If Brackup does somehow does solve this problem, then it's well ahead of
> my hacking, and I'll take a shot at pushing the whole pool to Amazon
> that way.  (I guess I'd have to blackout backups for the period during
> which the transfer is taking place...)  If Brackup hasn't solved this
> problem, then I might as well continue down my original path.

My impression from the docs and author's blog is that it has solved that 
problem even with compression/encryption of the content, but I'd want to 
test and understand it a bit before completely trusting it.

-- 
   Les Mikesell
    lesmikesell AT gmail DOT com

------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store 
http://p.sf.net/sfu/nokia-dev2dev
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/