Bacula-users

Re: [Bacula-users] restore / slow directory tree build process on 5.0.x / mysql

2010-07-19 07:38:02
Subject: Re: [Bacula-users] restore / slow directory tree build process on 5.0.x / mysql
From: Dan Langille <dan AT langille DOT org>
To: Nick Hilliard <nick AT foobar DOT org>
Date: Mon, 19 Jul 2010 07:33:20 -0400
On 7/19/2010 7:04 AM, Nick Hilliard wrote:
> On 16/07/2010 16:48, Nick Hilliard wrote:
>>> +--------+-------+----------+----------------+---------------------+-------------------------+
>>> | JobId  | Level | JobFiles | JobBytes       | StartTime           | 
>>> VolumeName              |
>>> +--------+-------+----------+----------------+---------------------+-------------------------+
>>> | 10,087 | F     |  401,368 | 56,963,986,536 | 2010-07-04 02:05:01 | 
>>> Pool-Full-Office-0597   |
>>> | 10,178 | D     |    5,768 |  1,804,433,105 | 2010-07-11 02:05:01 | 
>>> Pool-Diff-Office-0009   |
>>> | 10,217 | I     |    3,189 |    628,898,394 | 2010-07-14 02:05:01 | 
>>> Pool-Increm-Office-0013 |
>>> | 10,230 | I     |    1,689 |    250,775,502 | 2010-07-15 02:05:00 | 
>>> Pool-Increm-Office-0051 |
>>> +--------+-------+----------+----------------+---------------------+-------------------------+
>>> You have selected the following JobIds: 10087,10178,10217,10230
>>>
>>> Building directory tree for JobId(s) 10087,10178,10217,10230 ...
>>
>> At this stage, bconsole has been sitting there for 3 hours waiting for
>> mysql to complete the query.  the mysqld process has been pegged at 100%
>> all this time.
>
> Just following up on this, it took 12 hours, 40 minutes before the
> directory tree was built and bconsole returned to the restore prompt.
> mysqld CPU usage was pegged at 100% during this time.
>
> This seems to be a severe regression on bacula 3.x, and it makes restoring
> single files completely unfeasible.

You are probably missing an index.  I'm pretty sure this has been 
discussed previously on this list. Sorry, I don't know which index is 
missing but I'm pretty sure it will be easy to find with a search.

>
> Nick
>
>>> SELECT
>>>          Path.Path,
>>>          Filename.Name,
>>>          Temp.FileIndex,
>>>          Temp.JobId,
>>>          LStat,
>>>          MD5
>>>      FROM (
>>>          SELECT
>>>                  FileId,
>>>                  Job.JobId AS JobId,
>>>                  FileIndex,
>>>                  File.PathId AS PathId,
>>>                  File.FilenameId AS FilenameId,
>>>                  LStat,
>>>                  MD5
>>>              FROM
>>>                  Job,
>>>                  File,
>>>                  (
>>>                      SELECT
>>>                              MAX(JobTDate) AS JobTDate,
>>>                              PathId,
>>>                              FilenameId
>>>                          FROM (
>>>                              SELECT
>>>                                      JobTDate,
>>>                                      PathId,
>>>                                      FilenameId
>>>                                  FROM
>>>                                      File
>>>                                  JOIN
>>>                                      Job USING (JobId)
>>>                                  WHERE
>>>                                      File.JobId IN (10087)
>>>                                  UNION ALL SELECT
>>>                                          JobTDate,
>>>                                          PathId,
>>>                                          FilenameId
>>>                                      FROM BaseFiles
>>>                                      JOIN File USING (FileId)
>>>                                      JOIN Job  ON    (BaseJobId = Job.JobId)
>>>                                          WHERE BaseFiles.JobId IN (10087)
>>>                          ) AS tmp GROUP BY PathId, FilenameId
>>>                  ) AS T1
>>>                  WHERE (
>>>                          Job.JobId IN (
>>>                              SELECT DISTINCT BaseJobId FROM BaseFiles WHERE 
>>> JobId IN (10087)
>>>                          ) OR
>>>                          Job.JobId IN (10087)
>>>                      )
>>>                  AND T1.JobTDate = Job.JobTDate
>>>                  AND Job.JobId = File.JobId
>>>                  AND T1.PathId = File.PathId
>>>                  AND T1.FilenameId = File.FilenameId
>>>      )
>>>      AS Temp
>>>      JOIN Filename ON (Filename.FilenameId = Temp.FilenameId)
>>>      JOIN Path ON (Path.PathId = Temp.PathId)
>>>      WHERE FileIndex>  0
>>>      ORDER BY Temp.JobId, FileIndex ASC
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>


-- 
Dan Langille - http://langille.org/

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users