Bacula-users

Re: [Bacula-users] Fwd: Spooling attrs takes forever

2013-11-15 09:38:55
Subject: Re: [Bacula-users] Fwd: Spooling attrs takes forever
From: Kern Sibbald <kern AT sibbald DOT com>
To: Josh Fisher <jfisher AT pvct DOT com>, bacula-users AT lists.sourceforge DOT net
Date: Fri, 15 Nov 2013 15:35:21 +0100
Hello,

I would suggest one minor modification to what you
wrote.  See below ...

On 11/14/2013 04:29 PM, Josh Fisher wrote:

On 11/12/2013 7:09 PM, Charles Douglass wrote:
This appears to have never posted.  My apologies if I'm wrong and this is duplicate.


-------- Original Message --------
Subject: Spooling attrs takes forever
Date: Mon, 11 Nov 2013 08:30:59 -0800
From: Charles Douglass <chas.douglass AT gmail DOT com>
To: bacula-users AT lists.sourceforge DOT net


I am running :
Linux Mint 15 (based on Ubuntu 13.04)
Bacula "Version: 5.2.6 (21 February 2012)"
MySQL version "14.14 Distrib 5.5.32, for debian-linux-gnu (x86_64) using
readline 6.2"

It's a network installation with several clients.

When it does a full backup of one client (also running Mint 15) of
around 40G it gets stuck on the "Sending spooled attrs to the Director.
Despooling 151,437,267 bytes ...".  This one step takes over eight hours
to complete.  For instance it started at 03:10 my time today and it's
now 08:20 and it's still running.

If backups are being written to fast local disk storage, then you would likely be better off with no spooling ( SpoolData=no and SpoolAttributes=no in bacula-dir.conf ).

Yes, for disk storage, it does not make much sense to have data spooling turned off.
I would suggest to always turn attribute spooling on (default off) so that attributes
will be inserted in batch mode (much faster), and if possible ensure that the
working directory, where attributes are spooled is on a different drive from the
Archive Directory.  Of course this last suggestion is most often not possible.

Kern


In any case, when spooling attributes, Bacula writes the attribute info to a temporary file in the Bacula work directory, then despools when the job is finished receiving data from the client by reading the temp file and updating the DB as a batch. My guess is that the problem you are having is the result of having the Bacula work directory and the DB storage on the same physical drive(s). Unless you have SSD drives, the constant movement of the read/write heads is killing performance (ie. disk thrashing).

There is no recent mention of this problem that I could find.  One
suggestion from 2012 was to reinitialize the DB so I did that, with no
change to the problem.

Job 1, which is a local backup of about 20G shows this on the console:
11-Nov 00:35 maple-sd JobId 1: Sending spooled attrs to the Director.
Despooling 9,801,461 bytes ...
11-Nov 01:11 maple-dir JobId 1: Bacula maple-dir 5.2.6 (21Feb12):

Which seems long, but is way faster than Job 2.

Chas Douglass





------------------------------------------------------------------------------
DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps
OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
Free app hosting. Or install the open source package on any LAMP server.
Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native!
http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk


_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users



------------------------------------------------------------------------------
DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps
OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
Free app hosting. Or install the open source package on any LAMP server.
Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native!
http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk


_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

------------------------------------------------------------------------------
DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps
OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
Free app hosting. Or install the open source package on any LAMP server.
Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native!
http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users