Bacula-users

Re: [Bacula-users] restore / slow directory tree build process on 5.0.x / mysql

2010-07-19 07:08:28
Subject: Re: [Bacula-users] restore / slow directory tree build process on 5.0.x / mysql
From: Nick Hilliard <nick AT foobar DOT org>
To: bacula-users AT lists.sourceforge DOT net
Date: Mon, 19 Jul 2010 12:04:23 +0100
On 16/07/2010 16:48, Nick Hilliard wrote:
>> +--------+-------+----------+----------------+---------------------+-------------------------+
>> | JobId  | Level | JobFiles | JobBytes       | StartTime           | 
>> VolumeName              |
>> +--------+-------+----------+----------------+---------------------+-------------------------+
>> | 10,087 | F     |  401,368 | 56,963,986,536 | 2010-07-04 02:05:01 | 
>> Pool-Full-Office-0597   |
>> | 10,178 | D     |    5,768 |  1,804,433,105 | 2010-07-11 02:05:01 | 
>> Pool-Diff-Office-0009   |
>> | 10,217 | I     |    3,189 |    628,898,394 | 2010-07-14 02:05:01 | 
>> Pool-Increm-Office-0013 |
>> | 10,230 | I     |    1,689 |    250,775,502 | 2010-07-15 02:05:00 | 
>> Pool-Increm-Office-0051 |
>> +--------+-------+----------+----------------+---------------------+-------------------------+
>> You have selected the following JobIds: 10087,10178,10217,10230
>>
>> Building directory tree for JobId(s) 10087,10178,10217,10230 ...
>
> At this stage, bconsole has been sitting there for 3 hours waiting for
> mysql to complete the query.  the mysqld process has been pegged at 100%
> all this time.

Just following up on this, it took 12 hours, 40 minutes before the 
directory tree was built and bconsole returned to the restore prompt. 
mysqld CPU usage was pegged at 100% during this time.

This seems to be a severe regression on bacula 3.x, and it makes restoring 
single files completely unfeasible.

Nick

>> SELECT
>>         Path.Path,
>>         Filename.Name,
>>         Temp.FileIndex,
>>         Temp.JobId,
>>         LStat,
>>         MD5
>>     FROM (
>>         SELECT
>>                 FileId,
>>                 Job.JobId AS JobId,
>>                 FileIndex,
>>                 File.PathId AS PathId,
>>                 File.FilenameId AS FilenameId,
>>                 LStat,
>>                 MD5
>>             FROM
>>                 Job,
>>                 File,
>>                 (
>>                     SELECT
>>                             MAX(JobTDate) AS JobTDate,
>>                             PathId,
>>                             FilenameId
>>                         FROM (
>>                             SELECT
>>                                     JobTDate,
>>                                     PathId,
>>                                     FilenameId
>>                                 FROM
>>                                     File
>>                                 JOIN
>>                                     Job USING (JobId)
>>                                 WHERE
>>                                     File.JobId IN (10087)
>>                                 UNION ALL SELECT
>>                                         JobTDate,
>>                                         PathId,
>>                                         FilenameId
>>                                     FROM BaseFiles
>>                                     JOIN File USING (FileId)
>>                                     JOIN Job  ON    (BaseJobId = Job.JobId)
>>                                         WHERE BaseFiles.JobId IN (10087)
>>                         ) AS tmp GROUP BY PathId, FilenameId
>>                 ) AS T1
>>                 WHERE (
>>                         Job.JobId IN (
>>                             SELECT DISTINCT BaseJobId FROM BaseFiles WHERE 
>> JobId IN (10087)
>>                         ) OR
>>                         Job.JobId IN (10087)
>>                     )
>>                 AND T1.JobTDate = Job.JobTDate
>>                 AND Job.JobId = File.JobId
>>                 AND T1.PathId = File.PathId
>>                 AND T1.FilenameId = File.FilenameId
>>     )
>>     AS Temp
>>     JOIN Filename ON (Filename.FilenameId = Temp.FilenameId)
>>     JOIN Path ON (Path.PathId = Temp.PathId)
>>     WHERE FileIndex > 0
>>     ORDER BY Temp.JobId, FileIndex ASC


------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users