Bacula-users

Re: [Bacula-users] Tuning for large (millions of files) backups?

2010-10-08 02:10:55
Subject: Re: [Bacula-users] Tuning for large (millions of files) backups?
From: Bruno Friedmann <bruno AT ioda-net DOT ch>
To: bacula-users AT lists.sourceforge DOT net
Date: Fri, 08 Oct 2010 08:07:41 +0200
On 10/07/2010 11:03 PM, Mingus Dew wrote:
> All,
>      I am running Bacula 5.0.1 on Solaris 10 x86. I'm currently running
> MySQL 4.1.22 for the database server. I do plan on upgrading to a compatible
> version of MySQL 5, but migrating to PostgreSQL isn't an option at this
> time.
> 
>      I am trying to backup to tape a very large number of files for a
> client. While the data size is manageable at around 2TB, the number of files
> is incredibly large.
> The first of the jobs had 27 million files and initially failed because the
> batch table became "Full". I changed the myisam_data_pointer size to a value
> of 6 in the config.
> This job was then able to run successfully and did not take too long.
> 
>     I have another job which has 42 million files. I'm not sure what that
> equates to in rows that need to be inserted, but I can say that I've not
> been
> able to successfully run the job, as it seems to hang for over 30 hours in a
> "Dir inserting attributes" status. This causes other jobs to backup in the
> queue and
> once canceled I have to restart Bacula.
> 
>     I'm looking for way to boost performance of MySQL or Bacula (or both) to
> get this job completed.
> 
> Thanks,
> Shon
> 
Rude answer :

If you really want to use Mysql drop the myisam to innodb.
But you don't want to use mysql for that job, just use Postgresql fine tuned 
with batch insert enabled.

:-)

-- 

Bruno Friedmann (irc:tigerfoot)
Ioda-Net Sàrl www.ioda-net.ch
 openSUSE Member
    User www.ioda.net/r/osu
    Blog www.ioda.net/r/blog
  fsfe fellowship www.fsfe.org
GPG KEY : D5C9B751C4653227
vcard : http://it.ioda-net.ch/ioda-net.vcf

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users