Bacula-users

Re: [Bacula-users] Copying huge number of files - very slow

2010-03-02 12:54:17
Subject: Re: [Bacula-users] Copying huge number of files - very slow
From: Phil Stracchino <alaric AT metrocast DOT net>
To: bacula-users AT lists.sourceforge DOT net
Date: Tue, 02 Mar 2010 12:34:17 -0500
On 03/02/10 12:10, Sean Carolan wrote:
> Hello bacula users:
> 
> We have a server that uses BackupPC to handle network-based backups of
> a couple hundred hosts on our network.  If you're not familiar with
> BackupPC, it uses rsync and a clever pooling technique to optimize
> storage space.  The downside of this is that you can end up with
> millions of obscure files in the backup partition and tons of hard
> links pointing to the various files.  Currently our backup partition
> has around 1.2 terabytes of data on it.
> 
> I'd like to use bacula to copy this data to tapes for off-site
> storage.  We have an HP d2d device that gets speeds from 30-60
> megabytes per second during testing with btape.  When I try to run a
> bacula job for our backuppc partition however I only get around 25MB
> per *minute*.  At this rate it will take days to back up the entire
> partition.
> 
> Any suggestions on how we might speed this up?


I confess my first reaction is that any time your answer involves two
different backup systems working in series, you probably asked the wrong
question in the first place.  If you're just copying all this data to
tape for an offsite archive copy (which, one presumes, you're going to
date and periodically rotate), why don't you just dump it with, say,
cpio/afio?


-- 
  Phil Stracchino, CDK#2     DoD#299792458     ICBM: 43.5607, -71.355
  alaric AT caerllewys DOT net   alaric AT metrocast DOT net   phil AT 
co.ordinate DOT org
         Renaissance Man, Unix ronin, Perl hacker, Free Stater
                 It's not the years, it's the mileage.

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users