BackupPC-users

Re: [BackupPC-users] Amazon S3 and/or EC2 or other off-site storage ideas

2009-06-27 09:19:59
Subject: Re: [BackupPC-users] Amazon S3 and/or EC2 or other off-site storage ideas
From: Leen Besselink <leen AT consolejunky DOT net>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Sat, 27 Jun 2009 14:19:48 +0200
Pieter Wuille wrote:
> On Sat, Jun 27, 2009 at 11:05:23AM +0200, Tino Schwarze wrote:
>> Hi Mark,
>>

Hi Mark, Tino & Pieter,

>> On Fri, Jun 26, 2009 at 08:28:32AM -0700, Mark Phillips wrote:
>>> I am looking for inexpensive off-site storage that is compatible with
>>> backuppc.
>> In my opinion, off-site storage and backup don't go together. After all,
>> your backups contain all of your important (and possibly secret) data. I
>> wouldn't want to store these on some online-service.
> 
> Therefore you should always have offsite backups encrypted using a key that
> is not stored on the offsite backup. Ideally, the offsite server (assuming
> you don't control the physical access to it), shouldn't even ever see the
> unencrypted data.
> 

My personal opinion is, backup is for restoring files and has history, but
offsite storage is for when the building burns down. Some people would probably
call that disaster recovery.

> You can do this at either filesystem-level or blockdevice-level. The first one
> is (in the case of backuppc, because of massive hardlinking and millions of
> small files), very cpu and ram intensive, and more difficult to encrypt.
> Using a blockdevice-approach is a lot less stressing for you system, but
> probably a serious waste of bandwidth unless you have a way to send
> incrementals updates.
> 

I agree on the encryption part and I always though http://duplicity.nongnu.org/
looks pretty cool.

My guess is it doesn't do hardlinks, I haven't checked, but maybe you don't
really need history (what the hardlinks are for) when disaster strikes ?

That's obviously up to you.

>>> Is anyone using backuppc to backup files to Amazon S3? I have googled for
>>> some articles on this topic, and all I have found are old ones. It seems
>>> 1. S3 does not "allow" hard links and use of rsync. There is a s3sync
>>> option, but I haven't looked a it.
>>> 2. Using an EC2 front end running backuppc might work, and then storing to
>>> s3. Haven't found any backuppc articles about this.
>> I'm not familiar with S3 or EC2. Currently, BackupPC requires direct
>> access to a posix file system with hardlink support. And it does quite
>> stress the file system!
> 
> Neither am i familiar with S3 or EC2, and have no idea what kind of ways
> there are for synchronisation, but maybe you can synchronize the backup
> volume (blockdevice level) or backup filesystem with something on S3/EC2.
> I would advise against using backuppc to do offsite-backups of a backuppc
> pool. It will require many resources and increases the difficulty for
> doing a restore.
> 

I think duplicity does do so, but again I haven't checked.

But anyway, the most important thing you should do with any tooling like
this, is a restore test. And have a working plan.

------------------------------------------------------------------------------
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/