BackupPC-users

Re: [BackupPC-users] BackupPC 3.x Pool on BTRFS?

2016-02-18 11:45:55
Subject: Re: [BackupPC-users] BackupPC 3.x Pool on BTRFS?
From: Russell R Poyner <russell.poyner AT wisc DOT edu>
To: cvoelker AT knebb DOT de, "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Thu, 18 Feb 2016 10:44:28 -0600
Christian,

I've run BackupPC on zfs for several years. As you point out there is a 
lot of overlap between the features of BackupPC and COW file systems. 
However, BackupPC is not designed to take advantage of COW filesystems 
specifically.

My observations:

1. I saw minimal benefit from zfs deduplication when backing up desktop 
machines. It seems that BackupPC's hard-link based dedupe get's most of 
available benefit, with very little duplicated data for zfs to do dedupe 
on. Also with zfs anyway dedupe doesn't work well, and is widely 
discouraged. I've no experience with dedupe on btrfs.

2. Using filesystem compression instead of BackupPC compression works 
fine for me. I haven't really compared it to using BackupPC compression. 
You only want one compression system though. If you use BackupPC 
compression, your file system compression should be off and if you use 
the filesystem's compression BackupPC's compression should be off.

3. I've not done physical pool migration, but zfs send can do some cool 
things. Presumably btrfs send can do similar things.

4. COW filesystems have stability advantages in terms of checksums and 
the possibility to recover from a snapshot if you make a mistake. They 
also can have performance problems due to the large number of seeks that 
can be needed as the file system fragments. It becomes a personal 
decision as to what you want to use.

If you are doing this for a production set up I'd recommend using a 
filesystem you are comfortable and confident with. Learning BackupPC and 
a new filesystem could become an adventure ;-)

RP

On 14/02/16 04:01, Christian Völker wrote:
> Hi all,
>
> I did find only a single reference to the topic- but already some years ago.
>
> So are there any implications someone need to take care of when using
> BackupPC pool on a BTRFS filesystem?
>
> Some features of BTRFS seems to be perfect for BackupPC but I am unsure
> if they have advantages.
>
> Deduplication feature:
> BackupPC hadrlinks identical files to save storage space which is
> already deduplication. But it uses hardlinks to do so- which frequently
> causes issues as you can see in this mailing list. So possibly diasble
> hardlinking in BackupPC (if possible) and let BTRFS do the work with
> deduplication? Disadvantage for BTRFS: it is only a "out-of-band"
> deduplication- so you have to perform the dupe detection by cron or so.
>
> Online-Compression:
> BTRFS uses online-compression- I am unsure if there would be an
> advantage if compression is done by filesystem. At least IMHO it would
> be easier to restore directly from the pool instead of cpool through bash?
>
> Pool-migration/ move:
> If you decide to move your pool to a different location you are happy if
> you have it on something like LVM (or completely virtualized). Then you
> can move the underlying device. Disadvantage: LVM and other tools move
> the full device. If the pool is just filled up to 70% it even transfers
> the 30% of empty space. With btrfs receive and btrfs send the filesystem
> will only transfer the used blocks.
>
> COW-Feature:
> With copy-on-write btrfs writes changed block on a different location
> and then refers to this new location. Has advantages for snapshots and
> so on. But disadvantages for larger files (like virtual disk images) as
> they spread among all devices which causes lots of seeks on rotating
> devices like HDDs. How about pool? Am I right this does not matter for
> backupPC as it always writes full files only? Even rsync-based backups?
>
> Besides of BackupPC someone havving experiences in using BTRFS?
>
> Thanks!
>
> Christian
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> _______________________________________________
> BackupPC-users mailing list
> BackupPC-users AT lists.sourceforge DOT net
> List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:    http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/



------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>