Bacula-users

[Bacula-users] Issue with recycling after moving from 2.0 to 2.4

2009-02-13 05:22:41
Subject: [Bacula-users] Issue with recycling after moving from 2.0 to 2.4
From: Yann Cézard <yann.cezard AT univ-pau DOT fr>
To: bacula-users AT lists.sourceforge DOT net
Date: Fri, 13 Feb 2009 11:19:40 +0100
Hi list,

I have recently moved to bacula 2.4.4 (2.4.3 at the start, I upgraded
to 2.4.4 last week), from 2.0.3 (totally new installation : new server,
new databases, the only thing that I kept from the old install is my
client/jobs/pools/... configuration files) and I am facing a strange
issue in volume recycling.
Operating system is Debian Lenny (bacula packages are the "official"
Debian ones).

My backups are configured that way :
- 1 Full job per week, Incremental the other days
- jobs are kept 13 days (full or incr)
- volume are 13 days retention period, recycling is on, autoprune
  is on, each job has one Full Pool (2 volumes) and one Incremental
  Pool (12 volumes)
- all backups are done on disk (no tape)
- multiple databases (not really one for each client, but not far - one
  database by client profile)

Everything was working fine with 2.0.3, but I noticed that with 2.4.4
recycling is sometime happenning more lately, or not when wanted...

Exple :
Pools looks like this in database (sorry for the formatting) :
+--------+--------------+---------+---------+---------+------------+-----------------+--------------+----------------+------------+-------------+-------------+-----------+---------+----------+-----------+-------------+---------+---------------+---------------+------------+--------------------+-------------------+---------------+
| PoolId | Name         | NumVols | MaxVols | UseOnce | UseCatalog | 
AcceptAnyVolume | VolRetention | VolUseDuration | MaxVolJobs | 
MaxVolFiles | MaxVolBytes | AutoPrune | Recycle | PoolType | LabelType | 
LabelFormat | Enabled | ScratchPoolId | RecyclePoolId | NextPoolId | 
MigrationHighBytes | MigrationLowBytes | MigrationTime |
+--------+--------------+---------+---------+---------+------------+-----------------+--------------+----------------+------------+-------------+-------------+-----------+---------+----------+-----------+-------------+---------+---------------+---------------+------------+--------------------+-------------------+---------------+
|     60 | PoolSso-Full |       2 |       2 |       1 |          1 
|               0 |      1123200 |              0 |          0 
|           0 |           0 |         1 |       1 | Backup   |         0 
| sso-full-   |       1 |    0 |             0 |          0 
|                  0 |                 0 |             0 |
|     61 | PoolSso-Incr |      12 |      12 |       1 |          1 
|               0 |      1123200 |              0 |          0 
|           0 |           0 |         1 |       1 | Backup   |         0 
| sso-incr-   |       1 |    0 |             0 |          0 
|                  0 |                 0 |             0 |
+--------+--------------+---------+---------+---------+------------+-----------------+--------------+----------------+------------+-------------+-------------+-----------+---------+----------+-----------+-------------+---------+---------------+---------------+------------+--------------------+-------------------+---------------+

Media :
+---------+---------------+--------+-----------+-------------+-----------+---------------------+---------------------+---------------------+---------+----------+-----------+-----------+-----------+
           

| MediaId | VolumeName    | PoolId | MediaType | MediaTypeId | LabelType 
| FirstWritten        | LastWritten         | LabelDate           | 
VolJobs | VolFiles | VolBlocks | VolMounts | VolStatus |          
+---------+---------------+--------+-----------+-------------+-----------+---------------------+---------------------+---------------------+---------+----------+-----------+-----------+-----------+
           

|       2 | sso-full-0002 |     60 | File-sso  |           0 |         0 
| 2009-02-10 22:40:13 | 2009-02-10 22:46:00 | 2009-02-10 22:40:13 
|       1 |        0 |     21810 |         2 | Used      |          
|       5 | sso-incr-0005 |     61 | File-sso  |           0 |         0 
| 2009-02-11 22:37:30 | 2009-02-11 22:39:19 | 2009-02-11 22:37:30 
|       1 |        0 |      1914 |         2 | Used      |          
|       7 | sso-incr-0007 |     61 | File-sso  |           0 |         0 
| 2009-01-29 22:42:26 | 2009-01-29 22:43:13 | 2009-01-29 22:42:26 
|       1 |        0 |      1722 |         1 | Used      |          
|      10 | sso-incr-0010 |     61 | File-sso  |           0 |         0 
| 2009-01-30 22:34:40 | 2009-01-30 22:35:13 | 2009-01-30 22:34:40 
|       1 |        0 |      1737 |         1 | Used      |          
|      13 | sso-incr-0013 |     61 | File-sso  |           0 |         0 
| 2009-01-31 21:31:26 | 2009-01-31 21:32:00 | 2009-01-31 21:31:26 
|       1 |        0 |      1744 |         1 | Used      |          
|      16 | sso-incr-0016 |     61 | File-sso  |           0 |         0 
| 2009-02-01 22:31:09 | 2009-02-01 22:31:43 | 2009-02-01 22:31:09 
|       1 |        0 |      1753 |         1 | Used      |          
|      19 | sso-incr-0019 |     61 | File-sso  |           0 |         0 
| 2009-02-02 22:33:27 | 2009-02-02 22:34:02 | 2009-02-02 22:33:27 
|       1 |        0 |      1772 |         1 | Used      |          
|      22 | sso-full-0022 |     60 | File-sso  |           0 |         0 
| 2009-02-03 22:31:50 | 2009-02-03 22:36:31 | 2009-02-03 22:31:50 
|       1 |        0 |     21701 |         1 | Used      |          
|      24 | sso-incr-0024 |     61 | File-sso  |           0 |         0 
| 2009-02-04 22:35:47 | 2009-02-04 22:37:51 | 2009-02-04 22:35:47 
|       1 |        0 |      1807 |         1 | Used      |          
|      27 | sso-incr-0027 |     61 | File-sso  |           0 |         0 
| 2009-02-05 22:38:27 | 2009-02-05 22:39:05 | 2009-02-05 22:38:27 
|       1 |        0 |      1825 |         1 | Used      |          
|      30 | sso-incr-0030 |     61 | File-sso  |           0 |         0 
| 2009-02-06 22:37:04 | 2009-02-06 22:37:44 | 2009-02-06 22:37:04 
|       1 |        0 |      1842 |         1 | Used      |          
|      33 | sso-incr-0033 |     61 | File-sso  |           0 |         0 
| 2009-02-07 21:31:22 | 2009-02-07 21:31:58 | 2009-02-07 21:31:22 
|       1 |        0 |      1852 |         1 | Used      |          
|      36 | sso-incr-0036 |     61 | File-sso  |           0 |         0 
| 2009-02-08 22:31:47 | 2009-02-08 22:32:25 | 2009-02-08 22:31:47 
|       1 |        0 |      1858 |         1 | Used      |          
|      39 | sso-incr-0039 |     61 | File-sso  |           0 |         0 
| 2009-02-09 22:33:36 | 2009-02-09 22:34:14 | 2009-02-09 22:33:36 
|       1 |        0 |      1877 |         1 | Used      |          
+---------+---------------+--------+-----------+-------------+-----------+---------------------+---------------------+---------------------+---------+----------+-----------+-----------+-----------+
   


And Jobs :
+-------+----------------------------+------+------+-------+----------+-----------+---------------------+---------------------+---------------------+---------------------+------------+--------------+----------------+----------+------------+-----------+-----------------+--------+-----------+------------+-------------+---------+
                                                                                
                                                                  

| JobId | Job                        | Name | Type | Level | ClientId | 
JobStatus | SchedTime           | StartTime           | 
EndTime             | RealEndTime         | JobTDate   | VolSessionId | 
VolSessionTime | JobFiles | JobBytes   | JobErrors | JobMissingFiles | 
PoolId | FileSetId | PriorJobId | PurgedFiles | HasBase 
|                                                                               
                                                                   

+-------+----------------------------+------+------+-------+----------+-----------+---------------------+---------------------+---------------------+---------------------+------------+--------------+----------------+----------+------------+-----------+-----------------+--------+-----------+------------+-------------+---------+
                                                                                
                                                                  

|    50 | sso.2009-02-12_22.30.01.30 | sso  | B    | I     |        2 | 
A         | 2009-02-12 22:30:01 | 2009-02-12 22:38:25 | 2009-02-13 
10:34:51 | 2009-02-13 10:34:51 | 1234517691 |           14 |     
1234454882 |        0 |          0 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    47 | sso.2009-02-11_22.30.01.57 | sso  | B    | I     |        2 | 
T         | 2009-02-11 22:30:01 | 2009-02-11 22:37:30 | 2009-02-11 
22:39:19 | 2009-02-11 22:39:19 | 1234388359 |          145 |     
1234171252 |       16 |  123169733 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|     8 | sso.2009-01-29_22.30.00.25 | sso  | B    | I     |        2 | 
T         | 2009-01-29 22:30:00 | 2009-01-29 22:42:26 | 2009-01-29 
22:43:13 | 2009-01-29 22:43:13 | 1233265393 |           14 |     
1233253608 |       17 |  110767822 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    11 | sso.2009-01-30_22.30.00.28 | sso  | B    | I     |        2 | 
T         | 2009-01-30 22:30:00 | 2009-01-30 22:34:40 | 2009-01-30 
22:35:13 | 2009-01-30 22:35:13 | 1233351313 |           14 |     
1233324758 |       16 |  111759453 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    14 | sso.2009-01-31_21.30.00.29 | sso  | B    | I     |        2 | 
T         | 2009-01-31 21:30:00 | 2009-01-31 21:31:26 | 2009-01-31 
21:32:00 | 2009-01-31 21:32:00 | 1233433920 |           14 |     
1233423786 |       10 |  112214492 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    17 | sso.2009-02-01_22.30.01.30 | sso  | B    | I     |        2 | 
T         | 2009-02-01 22:30:01 | 2009-02-01 22:31:09 | 2009-02-01 
22:31:43 | 2009-02-01 22:31:43 | 1233523903 |           14 |     
1233502959 |       16 |  112753840 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    20 | sso.2009-02-02_22.30.00.30 | sso  | B    | I     |        2 | 
T         | 2009-02-02 22:30:00 | 2009-02-02 22:33:27 | 2009-02-02 
22:34:02 | 2009-02-02 22:34:02 | 1233610442 |           14 |     
1233587212 |       16 |  114005856 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    23 | sso.2009-02-03_22.30.01.30 | sso  | B    | F     |        2 | 
T         | 2009-02-03 22:30:01 | 2009-02-03 22:31:50 | 2009-02-03 
22:36:35 | 2009-02-03 22:36:35 | 1233696995 |           14 |     
1233675432 |    44656 | 1388748407 |         0 |               0 |     
60 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    26 | sso.2009-02-04_22.30.00.30 | sso  | B    | I     |        2 | 
T         | 2009-02-04 22:30:00 | 2009-02-04 22:35:47 | 2009-02-04 
22:37:51 | 2009-02-04 22:37:51 | 1233783471 |           14 |     
1233752014 |       13 |  116268405 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    29 | sso.2009-02-05_22.30.00.33 | sso  | B    | I     |        2 | 
T         | 2009-02-05 22:30:00 | 2009-02-05 22:38:27 | 2009-02-05 
22:39:06 | 2009-02-05 22:39:06 | 1233869946 |           14 |     
1233842441 |       22 |  117411605 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    32 | sso.2009-02-06_22.30.01.31 | sso  | B    | I     |        2 | 
T         | 2009-02-06 22:30:01 | 2009-02-06 22:37:04 | 2009-02-06 
22:37:45 | 2009-02-06 22:37:45 | 1233956265 |           14 |     
1233944766 |       16 |  118484893 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    35 | sso.2009-02-07_21.30.00.39 | sso  | B    | I     |        2 | 
T         | 2009-02-07 21:30:00 | 2009-02-07 21:31:22 | 2009-02-07 
21:31:58 | 2009-02-07 21:31:58 | 1234038718 |           80 |     
1233944766 |       13 |  119164252 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    38 | sso.2009-02-08_22.30.00.47 | sso  | B    | I     |        2 | 
T         | 2009-02-08 22:30:00 | 2009-02-08 22:31:47 | 2009-02-08 
22:32:26 | 2009-02-08 22:32:26 | 1234128746 |          148 |     
1233944766 |       13 |  119515925 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    41 | sso.2009-02-09_22.30.00.30 | sso  | B    | I     |        2 | 
T         | 2009-02-09 22:30:00 | 2009-02-09 22:33:36 | 2009-02-09 
22:34:14 | 2009-02-09 22:34:14 | 1234215254 |           14 |     
1234171252 |       13 |  120764165 |         0 |               0 |     
61 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

|    44 | sso.2009-02-10_22.30.00.43 | sso  | B    | F     |        2 | 
T         | 2009-02-10 22:30:00 | 2009-02-10 22:40:13 | 2009-02-10 
22:46:29 | 2009-02-10 22:46:29 | 1234302389 |           75 |     
1234171252 |    44668 | 1395784285 |         0 |               0 |     
60 |         1 |          0 |           0 |       0 
|                                                                               
                                                                   

+-------+----------------------------+------+------+-------+----------+-----------+---------------------+---------------------+---------------------+---------------------+------------+--------------+----------------+----------+------------+-----------+-----------------+--------+-----------+------------+-------------+---------+
 


Today I had to cancel the incremental job :

12-fév 22:38 backuppa-dir JobId 50: Start Backup JobId 50, 
Job=sso.2009-02-12_22.30.01.30
12-fév 22:38 backuppa-dir JobId 50: Using Device "SAVE-SSO"
12-fév 22:38 backuppa-sd JobId 50: Job sso.2009-02-12_22.30.01.30 waiting. 
Cannot find any appendable volumes.
Please use the "label"  command to create a new Volume for:
    Storage:      "SAVE-SSO" (/save/sso)
    Pool:         PoolSso-Incr
    Media type:   File-sso
12-fév 23:38 backuppa-sd JobId 50: Job sso.2009-02-12_22.30.01.30 waiting. 
Cannot find any appendable volumes.
Please use the "label"  command to create a new Volume for:
    Storage:      "SAVE-SSO" (/save/sso)
    Pool:         PoolSso-Incr
    Media type:   File-sso
13-fév 01:38 backuppa-sd JobId 50: Job sso.2009-02-12_22.30.01.30 waiting. 
Cannot find any appendable volumes.
Please use the "label"  command to create a new Volume for:
    Storage:      "SAVE-SSO" (/save/sso)
    Pool:         PoolSso-Incr
    Media type:   File-sso
13-fév 05:38 backuppa-sd JobId 50: Job sso.2009-02-12_22.30.01.30 waiting. 
Cannot find any appendable volumes.
Please use the "label"  command to create a new Volume for:
    Storage:      "SAVE-SSO" (/save/sso)
    Pool:         PoolSso-Incr
    Media type:   File-sso
13-fév 10:34 backuppa-sd JobId 50: Job sso.2009-02-12_22.30.01.30 marked to be 
canceled.
13-fév 10:34 backuppa-sd JobId 50: Job sso.2009-02-12_22.30.01.30 canceled 
while waiting for mount on Storage Device ""SAVE-SSO" (/save/sso)".


This happened several times on differents jobs, but not always (that's 
why i am a little
confused...).

My question is why in this case the volume labeled sso-incr-0007 didn't 
have been recycled ?
It was last written on "2009-01-29 22:42:26", correponding job 
RealEndTime is "2009-01-29 22:43:13",
so retention time (13 days) was over, am I right ?

And why does it seems to behave differently from 2.0.3 ?
Am i missing some changes between the 2 versions ?

This is really a problem because it causes delays in backup scheduling,
sometime more then 4 jobs are waiting for a volume, which cause 4 
processes on 6 to be
idled and delaying all my other jobs.

I can provide more information from database or config files if needed.

Thanks for any clue,

Best regards,

-- 
Yann Cézard - Administrateur Systèmes Serveurs
Centre de Ressources Informatiques    -    http://cri.univ-pau.fr
Université de Pau et des Pays de l'Adour - http://www.univ-pau.fr


------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users