Bacula-users

Re: [Bacula-users] Tuning for large (millions of files) backups?

2010-11-13 07:08:21
Subject: Re: [Bacula-users] Tuning for large (millions of files) backups?
From: Gavin McCullagh <gavin.mccullagh AT gcd DOT ie>
To: bacula-users AT lists.sourceforge DOT net
Date: Sat, 13 Nov 2010 12:05:27 +0000
Hi,

On Fri, 12 Nov 2010, Bob Hetzel wrote:

> I'm starting to think the issue might be linked to some kernels or linux 
> distros.  I have two bacula servers here.  One system is a year and a half 
> old (12 GB RAM), has with a File table having approx 40 million File 
> records.  That system has had the slowness issue (building the directory 
> tree on restores took about an hour) running first Ubuntu 9.04 or 9.10 and 
> now RedHat 6 beta.  The kernel currently is at 2.6.32-44.1.el6.x86_64.  I 
> haven't tried downgrading, instead I tweaked the source code to use the old 
> 3.0.3 query and recompiled--I don't use Base jobs or Accurate backups so 
> that's safe for me.
> 
> The other system is 4 yrs or so old, with less memory (8GB), slower cpus, 
> slower hard drives, etc., and in fairness only 35 million File records. 
> This one builds the directory tree in approx 10 seconds, but is running 
> Centos 5.5.  The kernel currently is at 2.6.18-194.11.3.el5.

That's an interesting thought.  It would be interesting to make an exact
comparison, something like:

 - run a restore with the slow query log on, capture the query text 
 - run the query manually in mysql
 - dump the mysql database
 - restore the mysql database on the older server
 - run the query there

It sounds like your database is quite large so this might be too
awkward in practice?  Strictly speaking the freshly sequentially written
database might have a slight unfair advantage, but if the results are
radically different then that would be useful to know.

Gavin


------------------------------------------------------------------------------
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users