Adam Goryachev wrote:
>
>> We're a mid-sized university academic department and are fairly happy 
>> with our current BackupPC setup.  We currently backup an 800 GB 
>> fileserver (which has an iSCSI-attached drive), a few other sub-300 GB 
>> fileservers, a bunch of XP desktops.  The 800 GB fileserver takes a long 
>> time to back up...almost a full day, and I think this is normal for 
>> BackupPC.  We'd like to use BackupPC to backup some of our heftier linux 
>> servers -- moving into the multiple terabyte range (5-10 TB).  We're 
>> considering a ZFS filesystem over gigabit for our backup target, but 
>> obviously are concerned that backing up 5 TB of data would take a week.
>>
>> Is this where we should consider multiple BackupPC servers to break up 
>> the backup time?  Should we move to a solution with less overhead (if 
>> there is one)?  Thanks for any input or experiences.
> 
> You haven't specified your backup method, but I'll make a couple of
> assumptions:
> 
> * you are using rsync over SSH
> 
> If your 800G file server is taking 24 hours per backup, there are
> probably some optimisations you can make. Firstly, check if either your
> backup server or file server are consuming all available memory and
> swapping during the backup. Adding some extra RAM will drastically
> reduce the backup time. The other issue is the size of the files on your
> fileserver, lots of small files will take longer than lots of small files.

Also, incremental runs should be substantially faster than fulls, even 
when using rsync where the data tranfer amount is similar.  If all the 
fulls are happening at the same time, force some to run on a different 
day to skew them.  You might push the larger fulls to Friday night if it 
is OK to run into the weekend.

> Other than that, you can look at IO etc on both fileserver and backup
> server. Probably your bigger fileservers are faster, but you backup
> server is only so fast... Examine where the bottlenecks are, and then
> you can either remove/improve those bottlenecks, or else add additional
> backuppc servers (if the backup server is the bottleneck and removing it
> is too costly).

Some files are just not backup-friendly, like unix mailbox format or big 
databases.  You may need to handle them some other way and exclude them 
from the backuppc runs.

Raid5 is generally a bad idea performance-wise unless you have 12 or 
more disks in a set. It is usually much more expensive to go above 
'commodity-size' servers than to add more so you may be better off 
splitting the backups to different servers, especially if you can group 
similar targets for better pooling and knowing where to look when you 
need to restore.

-- 
   Les Mikesell
    lesmikesell AT gmail DOT com



------------------------------------------------------------------------------
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/