Hello,

If you have Attribute Spooling turned off, you will essentially insert 
records into the database one at a time.  The database is locked around 
each insert so even though multiple threads are running they cannot run 
at the same time.  If you have a lot of files that are being backed up 
by each job, the DB insertions of attributes will be very slow.

If you have Attribute Spooling turned on, multiple simultaneous threads 
will be used to insert attributes (metadata) into the database, and that 
will be done in big batches.  Providing the Jobs are not referencing the 
same data (most likely) there will be no global locks so multiple DB 
inserts can run at the same time and they can be quite efficient.  For 
best performance the DB must be tuned as the default MySQL values are 
not optimal for Bacula.

Best regards,
Kern

On 15-03-24 11:18 AM, Phil Stracchino wrote:
> On 03/24/15 11:47, Robert Heinzmann wrote:
>> Hello,
>>
>> we have a rather large Bacula setup - ~600 Clients, database for catalog
>> is 19GB large. File table is ~65 million records.
>>
>> Right now, bacula director can not run backups as scheduled and delays
>> the jobs for up to 2 hours, as it seems the catalog is not keeping up.
>>
>> SD has not hit it’s storage IO limit, as I/O Wait is almost zero (24x
>> 10k spindles in Hardware Raid, doing ~50MB/s avg).
>>
>> Also we have defined 20 parallel backup drives doing backups, so drives
>> is not a blocker I guess.
>>
>> It seems that the bacula catalog is CPU bound and that only a single
>> core is used (of 4).
>>
>> How can I parallelize database access with Bacula Director to speed up
>> the backup process.
> If you have queries being executed serially by only a single connection,
> then they cannot be parallelized across multiple cores, because MySQL
> can use only a single core on a single query (trying to parallelize
> below the query level doesn't work out well).  I don't know whether the
> Bacula director's DB accesses *could* be parallelized, but I suspect
> many of them cannot.
>
>


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users