BackupPC-users

Re: [BackupPC-users] Using rsync for blockdevice-level synchronisation of BackupPC pools

2009-09-02 11:43:43
Subject: Re: [BackupPC-users] Using rsync for blockdevice-level synchronisation of BackupPC pools
From: Pieter Wuille <sipa AT users.sourceforge DOT net>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Wed, 2 Sep 2009 17:40:19 +0200
On Wed, Sep 02, 2009 at 10:14:05AM -0500, Les Mikesell wrote:
> Pieter Wuille wrote:
> > In our case, the BackupPC pool is stored on an XFS filesystem on an LVM
> > volume, allowing a xfsfreeze/sync/snapshot/xfsunfreeze, and using
> > devfiles.pl on the snapshot. Instead of xfsfreeze+unfreeze, a backuppc
> > stop/umount + mount/backuppc start is also possible. If no system for making
> > snapshots is available, you would need to suspend backuppc during the whole
> > synchronisation.
> > In fact, the BackupPC volume is already encrypted on our backup server
> > itself, allowing very cheap encrypted offsite backups (simply not sending
> > the keyfile to the remote side is enough...)
> > 
> > The result: offsite backups of our 400GiB pool, containing 350GiB data, of
> > which about 2GiB changes daily, is synchronised 5 times a week with offsite
> > backup in 12-15 hours, requiring nearly no bandwidth. This seems mostly
> > limited by the slow disk I/O on the receiver side (25MiB/s).
> > 
> > Hope you find this interesting/useful,
> 
> The one thing that would bother me about this approach is that you would 
> have a fairly long window of time while the remote filesystem chunks are 
> being updated.  While rsync normally creates a copy of an individual 
> file and does not delete the original until the copy is complete, a 
> mis-matched set of filesystem chunks would likely not be usable.  Since 
> disasters always happen at the worst possible time, I'd want to be sure 
> you could recover from losing the primary filesystem (site?) in the 
> middle of a remote copy.  This might be done by keeping a 2nd copy of 
> the files at the remote location, keeping them on an LVM with a snapshot 
> taken before each update, or perhaps catting them together onto a 
> removable device for fast access after the chunks update.

You're very right, and i thought about it too. Instead of using a RAID1 on
the offsite backup, there are two separate backups on the offsite machine,
and synchronisation switches between them. This also enables the use of
rsync's --inplace option.

Keeping an LVM snapshot is a possibility, but it becomes somewhat complex to
manage: you get a snapshot of a volume containing a filesystem whose files
correspond to parts of a snapshot of a volume containing an (encrypted)
filesystem containing a directory that corresponds to a pool of backups...

Catting the part files together to a device after transmission isn't a
complete solution: what if the machine crashes during the catting...?

-- 
Pieter

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>