BackupPC-users

Re: [BackupPC-users] Looking for some comments on sizing.

2017-02-08 22:02:09
Subject: Re: [BackupPC-users] Looking for some comments on sizing.
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Wed, 8 Feb 2017 21:01:44 -0600
On Wed, Feb 8, 2017 at 8:24 PM, Scott Walker
<scott.walker AT framestore DOT com> wrote:
> Individual files range from k text files, to100GB+ in size. Everything from
> images to movies files, to text, to VM images. (We are a VFX studio).
>
> And mainly unique files. so deduplication I'm not honestly concerned about.
> I just like BackupPC UI and it's easy of use and I'm wondering if this is a
> possible fit.
>

I don't think there should be any real surprises in terms of capacity
if you have hardware to handle it.   There are a few things that
aren't exactly optimal, at least in v3.   One is that the entire
directory tree is transmitted before the comparison starts and it is
held in RAM on both sides.  This can be a problem where there are
millions of files on a target host.  Another is that very large files
with changes can be slow to handle since the server has to uncompress
and copy the previous version while merging the differences and then
recompress again. And, the archive storage must be a single volume for
the hardlinks to work, although you can use LVM to work around
physical limits.

However, the price is right...  The nice thing about free software is
that if you need multiple instances you don't pay any more for the
extra licenses. I'd give it a try and if you need more than one
server, try to group the most similar targets together.   You may need
to add the hosts a few at a time to skew the days that the full
backups will run.   Also, note that backups become faster after the
2nd full run.  If you use the --checksum-seed option the rsync block
checksums are cached on the server so the archive copy no longer has
to be uncompressed for the rsync comparisons.

-- 
   Les Mikesell
     lesmikesell AT gmail DOT com

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/