BackupPC-users

Re: [BackupPC-users] Large Amounts of Data

2009-01-05 19:17:38
Subject: Re: [BackupPC-users] Large Amounts of Data
From: Adam Goryachev <mailinglists AT websitemanagers.com DOT au>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Tue, 06 Jan 2009 11:15:35 +1100
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Christopher Derr wrote:
> We're a mid-sized university academic department and are fairly happy 
> with our current BackupPC setup.  We currently backup an 800 GB 
> fileserver (which has an iSCSI-attached drive), a few other sub-300 GB 
> fileservers, a bunch of XP desktops.  The 800 GB fileserver takes a long 
> time to back up...almost a full day, and I think this is normal for 
> BackupPC.  We'd like to use BackupPC to backup some of our heftier linux 
> servers -- moving into the multiple terabyte range (5-10 TB).  We're 
> considering a ZFS filesystem over gigabit for our backup target, but 
> obviously are concerned that backing up 5 TB of data would take a week.
> 
> Is this where we should consider multiple BackupPC servers to break up 
> the backup time?  Should we move to a solution with less overhead (if 
> there is one)?  Thanks for any input or experiences.

You haven't specified your backup method, but I'll make a couple of
assumptions:

* you are using rsync over SSH

If your 800G file server is taking 24 hours per backup, there are
probably some optimisations you can make. Firstly, check if either your
backup server or file server are consuming all available memory and
swapping during the backup. Adding some extra RAM will drastically
reduce the backup time. The other issue is the size of the files on your
fileserver, lots of small files will take longer than lots of small files.

Other than that, you can look at IO etc on both fileserver and backup
server. Probably your bigger fileservers are faster, but you backup
server is only so fast... Examine where the bottlenecks are, and then
you can either remove/improve those bottlenecks, or else add additional
backuppc servers (if the backup server is the bottleneck and removing it
is too costly).

Gotta go, hopefully someone else will add some other suggestions.

Regards,
Adam
Adam Goryachev
Website Managers
Ph: +61 2 8304 0000                            adam AT websitemanagers.com DOT 
au
Fax: +61 2 8304 0001                            www.websitemanagers.com.au
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAklioqQACgkQGyoxogrTyiUlnQCgqrjMs61hcvGJ1Em1/feWvS6C
sIQAoKZPGdKqdQUzErA2/A4PQiNulS9v
=t4NI
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>