Lars,
Thanks for the interesting idea! I confess I haven't played with ZFS much
(though I've been wanting to for some time), maybe this is the excuse I need
;). Question, taking your model here, and applying it to my situation, how
well would this work:
BackupPC server, with a RAID1 zpool, with the third member being my external
fireproof drive. Rather than the rotation you described, just leave it as is
as it does its daily routine. Then, should the day come where I need to grab
the drive and go, plugging the drive into a system with ZFSonLinux & BackupPC
installed, could I mount this drive by itself?
I really like your idea of zfs send/receive for the remote copy. Do you have
any tips/pointers/docs on the best way to run it in this scenario?
Thanks,
--Mark
-----Original Message-----
From: Lars Tobias Skjong-Børsting [mailto:lists AT snota DOT no]
Sent: Friday, March 01, 2013 4:18 AM
To: backuppc-users AT lists.sourceforge DOT net
Subject: Re: [BackupPC-users] BackupPC Pool synchronization?
Hi,
On 3/1/13 12:34 AM, Les Mikesell wrote:
> On Thu, Feb 28, 2013 at 3:10 PM, Mark Campbell <mcampbell AT emediatrade DOT
> com> wrote:
>
>> So I'm trying to get a BackupPC pool synced on a daily basis from a
>> 1TB MD
>> RAID1 array to an external Fireproof drive (with plans to also sync
>> to a remote server at our collo).
>
> I'm not sure anyone has come up with a really good way to do this.
> One approach is to use a 3-member raid1 where you periodically remove
> a drive and resync a new one. If you have reasonable remote
> bandwidth and enough of a backup window, it is much easier to just run
> another instance of backuppc hitting the same targets independently.
I have come up with a IMHO good way to do this using ZFS (ZFSonLinux).
Description:
* uses 3 disks.
* at all times, keep 1 mirrored disk in a fire safe.
* periodically swap the safe disk with mirror in server.
1. create a zpool with three mirrored members.
2. create a filesystem on it and mount at /var/lib/backuppc.
3. do some backups.
4. detach one disk and put in safe.
5. do more backups.
6. detach one disk and swap with the other disk in the safe.
7. attach and online the disk from the safe.
8. watch it sync up.
I am currently using 2TB disks, and swap period of 1 month. Because of ZFS it
doesn't need to sync all the blocks, but only the changed blocks since 1 month
ago. For example, with 10GB changed it will sync in less than 25 minutes
(approx. 7 MB/s speed). That's a lot faster than anything I got with mdraid
which syncs every block.
ZFS also comes with benefits of checksumming and error correction of file
content and file metadata. BackupPC also supports error correction through
par2, and this gives an extra layer of data protection.
Backing up large numbers of files can take a very long time because of harddisk
seeking. This can be alleviated by using a SSD cache drive for ZFS. This
support for read (ZFS L2ARC) and write (ZFS ZIL) caching on a small SSD (30 GB)
cuts incremental time down to half for some shares.
As for remote sync, you can use "zfs send" on the backup server and "zfs
receive" on the offsite server. This will only send the differences since last
sync (like rsync), and will be probably be significantly faster than rsync that
in addition has to resolve all the hardlinks.
--
Best regards,
Lars Tobias
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics Download AppDynamics Lite for free
today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
