|
Re: [Bacula-users] Tuning for large (millions of files) backups?
2010-10-08 08:39:24
Bruno,
Not so rude at all :) You've made me think of 2 questions
How difficult is it (or procedure for) converting to InnoDB and what exactly will this gain in performance increase?
Also, you mention Postgresql and batch inserts. Does Bacula not use batch inserts with MySQL by default?
I'm assuming I'm using batch inserts because Bacula uses a table called 'batch'
-Shon
On Fri, Oct 8, 2010 at 2:07 AM, Bruno Friedmann <bruno AT ioda-net DOT ch> wrote:
On 10/07/2010 11:03 PM, Mingus Dew wrote:
> All,
> I am running Bacula 5.0.1 on Solaris 10 x86. I'm currently running
> MySQL 4.1.22 for the database server. I do plan on upgrading to a compatible
> version of MySQL 5, but migrating to PostgreSQL isn't an option at this
> time.
>
> I am trying to backup to tape a very large number of files for a
> client. While the data size is manageable at around 2TB, the number of files
> is incredibly large.
> The first of the jobs had 27 million files and initially failed because the
> batch table became "Full". I changed the myisam_data_pointer size to a value
> of 6 in the config.
> This job was then able to run successfully and did not take too long.
>
> I have another job which has 42 million files. I'm not sure what that
> equates to in rows that need to be inserted, but I can say that I've not
> been
> able to successfully run the job, as it seems to hang for over 30 hours in a
> "Dir inserting attributes" status. This causes other jobs to backup in the
> queue and
> once canceled I have to restart Bacula.
>
> I'm looking for way to boost performance of MySQL or Bacula (or both) to
> get this job completed.
>
> Thanks,
> Shon
>
If you really want to use Mysql drop the myisam to innodb.
But you don't want to use mysql for that job, just use Postgresql fine tuned with batch insert enabled.
:-)
--
Bruno Friedmann (irc:tigerfoot)
Ioda-Net Sàrl www.ioda-net.ch
openSUSE Member
User www.ioda.net/r/osu
Blog www.ioda.net/r/blog
fsfe fellowship www.fsfe.org
GPG KEY : D5C9B751C4653227
vcard : http://it.ioda-net.ch/ioda-net.vcf
------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb _______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
|
|
