BackupPC-users

Re: [BackupPC-users] Looking for some comments on sizing.

2017-02-08 19:24:17
Subject: Re: [BackupPC-users] Looking for some comments on sizing.
From: Adam Goryachev <mailinglists AT websitemanagers.com DOT au>
To: backuppc-users AT lists.sourceforge DOT net
Date: Thu, 9 Feb 2017 11:23:40 +1100
On 09/02/17 09:52, Scott Walker wrote:
> Has anyone used BackupPC in an enterprise environment?
>
> I'm talking PBs of data, 100's of servers, hybrid environment. Mac, 
> Solaris, BSD, Linux, Windows.
>
> Did it work well? Any gotcha's? When you see PB of data does it make 
> your gut feeling go uhh yeah no.
>
> I'm just fact finding and investigating. I find a lot of people 
> talking about using it for their home network or small business but 
> not much about the enterprise.

It would be interesting to hear if anyone is using it at that scale...

I expect that if you were going to try that, then you should use BPC v4 
(because it has the ability to avoid transferring a file if another 
client has already backed it up) and assuming you are going to use rsync 
everywhere.

I would also think that you will need to use multiple servers, otherwise 
you will spend a huge amount on getting enough CPU performance in the 
machine (all that encryption, compression, etc takes CPU), and RAM 
(storing lots of data and cache in RAM) and also the massive IOPS 
required. Consider the number of clients, you say 100s, so assume 300. 
You need a backup every day, and assume each backup takes 45 minutes. 
You need a concurrency of at least 30 to complete all the backups in 
under 8 hours (assuming an 8 hour backup window). OTOH, you could have 6 
servers each handling 5 concurrent backups, and hence have 50 clients on 
each server. This loses some of the de-dupe, but it makes everything a 
lot easier.

Personally, you could extend this further (depending on budget, risk, 
etc) and have 12 servers, such that each client is backed up by two 
different servers. A failure of one server or its underlying storage/etc 
will mean you do not lose all the backups.

Someone (on the list earlier) suggested that it would be possible to 
have BPC v4 use an object storage backend, so as long as that backend 
could perform fast enough, then it may be one way to get a larger "farm" 
of BPC servers but still keep de-dupe across all of them.

Regards,
Adam

-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/