On Fri, Oct 8, 2010 at 3:30 PM, Henrik Johansen <henrik AT scannet DOT dk> wrote:

'Mingus Dew' wrote:

All,
I am running Bacula 5.0.1 on Solaris 10 x86. I'm currently running
MySQL 4.1.22 for the database server. I do plan on upgrading to a
compatible version of MySQL 5, but migrating to PostgreSQL isn't an
option at this time.

I am trying to backup to tape a very large number of files for a
client. While the data size is manageable at around 2TB, the number of
files is incredibly large.
The first of the jobs had 27 million files and initially failed because
the batch table became "Full". I changed the myisam_data_pointer size
to a value of 6 in the config.

This job was then able to run successfully and did not take too long.

I have another job which has 42 million files. I'm not sure what that
equates to in rows that need to be inserted, but I can say that I've
not been able to successfully run the job, as it seems to hang for
over 30 hours in a "Dir inserting attributes" status. This causes
other jobs to backup in the queue and once canceled I have to restart
Bacula.

I'm looking for way to boost performance of MySQL or Bacula (or both)
to get this job completed.

You *really* need to upgrade to MySQL 5 and change to InnoDB - there is no
way in hell that MySQL 4 + MyISAM is going to perform decent in your
situation.
Solaris 10 is a Tier 1 platform for MySQL so the latest versions are
always available from www.mysql.com in the native pkg format so there really
is no excuse.

We run our Bacula Catalog MySQl servers on Solaris (OpenSolaris) so
perhaps I can give you some pointers.

Our smallest Bacula DB is currently ~70 GB (381,230,610 rows).

Since you are using Solaris 10 I assume that you are going to run MySQL
off ZFS - in that case you need to adjust the ZFS recordsize for the
filesystem that is going to hold your InnoDB datafiles to match the
InnoDB block size.

If you are using ZFS you should also consider getting yourself a fast
SSD as a SLOG (or to disable the ZIL entirely if you dare) - all InnoDB
writes to datafiles are O_SYNC and benefit *greatly* from an SSD in
terms of write / transaction speed.

If you have enough CPU power to spare you should try turning on
compression for the ZFS filesystem holding the datafiles - it also can
accelerate DB writes / reads but YMMV.

Lastly, our InnoDB related configuration from my.cnf :

# InnoDB options skip-innodb_doublewrite
innodb_data_home_dir = /tank/db/
innodb_log_group_home_dir = /tank/logs/
innodb_support_xa = false
innodb_file_per_table = true
innodb_buffer_pool_size = 20G
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 128M
innodb_log_file_size = 512M
innodb_log_files_in_group = 2
innodb_max_dirty_pages_pct = 90

Thanks,
Shon

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

--

Med venlig hilsen / Best Regards

Henrik Johansen
henrik AT scannet DOT dk
Tlf. 75 53 35 00

ScanNet Group
A/S ScanNet

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users