Re: [Bacula-users] Database inserts and performance issues

Maximum concurrent Jobs is set to 50 on each storage deamon. I've set the
Director to Maximum concurrent jobs of 110.

I'm running 64 bit Gentoo on the Director, MySQL server, and the Storage
daemons. Up until moving the Mysql server to SSD drives on different box I
was running both the Director and the Bacula catalog on the same server. 

The director is running the following processor:

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 55
model name      : AMD Athlon(tm) 64 Processor 3700+
stepping        : 2
cpu MHz         : 2210.117
cache size      : 1024 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
bogomips        : 4426.68
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

The mysql server has the following specs:

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 39
model name      : AMD Athlon(tm) 64 Processor 3700+
stepping        : 1
cpu MHz         : 2211.287
cache size      : 1024 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
bogomips        : 4422.57
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stc

The disk subsystem in the Director is a software raid 5 which consists of
7200 rpm sata drive.

Each storage deamon has the following processor:

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 95
model name      : AMD Athlon(tm) 64 Processor 3800+
stepping        : 2
cpu MHz         : 2411.006
cache size      : 512 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
bogomips        : 4858.21
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stc

and 1GB of ram. The disk subsystem on five of the storage deamons were hand
me downs each running Hardware RAID 5 with old 200 GB IDE drives on a 3ware
Model 7506-8. My spool drive is a 200GB 7200 RPM sata drive connected to the
on board sata. 

The sixth storage deamon was added later and has 500 GB 7200 drives in a
RAID 5 configuration on a 3ware model 9590SE-8ML.

All servers have single Gigabit connection.

According to Munin the director had a Max of committed memory of 2.95G for
the week and last night a max of 2.51G of memory. CPU utilization never
peaks above 8%.

My storage deamons are showing around 300-400 MB max committed memory
throughout the week each night during the backup cycle. CPU utilization
never peaks above 30%. The IOStat statistics show a peak of between 56k-59k
blocks per seconds written on the spool drives. On the older IDE RAID 5
systems I'm seeing max blocks per second written from 49k-54k. The Iostat
data on the newer RAID 5 with 500 GB drives hit a max blocks per second
written for the week of 143.42k.

The mysql server:

I using an InnoDB buffer pool size of 1500M. 
CPU utilization: Max for the week hit 97.30. But overall for the week I'm
seeing peaks never rise above 50%.
IOstat: for the week peaked at 61.94k blocks per second written. 
Mysql queries per week: Inserts per second 456.83 Max with an average 18.98.

Brian
-----Original Message-----
From: Jonas Björklund [mailto:jb AT soe DOT se] 
Sent: Thursday, April 16, 2009 5:02 AM
To: Kelly, Brian
Cc: bacula-users AT lists.sourceforge DOT net
Subject: Re: [Bacula-users] Database inserts and performance issues

On Tue, 14 Apr 2009, Kelly, Brian wrote:

> I've seen posts regarding attribute inserts as a performance issue even
when
> modern 10K 15K drives are being used. I backup 800 clients every night.

Interesting, does anyone know a bacula system with more clients?

How many concurrent clients do you allow to each SD, how many concurrent 
clients to the directory?

Do you want to tell us a little bit more about the setup? Are the 
directory and MySQL on the same server? What operating system do you use 
at the servers? What about the load on the directory?

/Jonas

smime.p7s
Description: S/MIME cryptographic signature

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and 
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today. 
Use priority code J9JMT32. http://p.sf.net/sfu/p

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users