Bacula-users

Re: [Bacula-users] Bacula volume (tape) rotation out of order

2008-06-01 22:32:30
Subject: Re: [Bacula-users] Bacula volume (tape) rotation out of order
From: "Mark Walkom" <markwalkom AT gmail DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Mon, 2 Jun 2008 12:24:29 +1000
Apologies for seemingly ignoring you on this Arno, I was sidetracked by some super urgent project work (is there any other?).
All of the below is handled on a single drive - no autochanger.

Here are my pool definitions;
##Pools
Pool {
  Name = Default
  Pool Type = Backup
  Recycle = yes
  AutoPrune = yes
  Volume Retention = 11 days
  Accept Any Volume = yes
  Maximum Volume Jobs = 2
}

Pool {
    Name = Incremental
    Pool Type = Backup
    Recycle = yes
    AutoPrune = yes
    Volume Retention = 11 days
    Accept Any Volume = yes
    Maximum Volume Jobs = 1
}

From digging around it looks like the backup to physical tape uses the Default pool;
*list pool
+--------+-------------+---------+---------+----------+-------------+
| PoolId | Name        | NumVols | MaxVols | PoolType | LabelFormat |
+--------+-------------+---------+---------+----------+-------------+
|      1 | Default     |       8 |       0 | Backup   | *           |
|      2 | Incremental |      11 |       0 | Backup   | *           |
+--------+-------------+---------+---------+----------+-------------+
*list media
Pool: Default
+---------+------------+-----------+----------------+----------+--------------+---------+------+-----------+-----------+---------------------+
| MediaId | VolumeName | VolStatus | VolBytes       | VolFiles | VolRetention | Recycle | Slot | InChanger | MediaType | LastWritten         |
+---------+------------+-----------+----------------+----------+--------------+---------+------+-----------+-----------+---------------------+
|      29 | WEEK2_001  | Used      | 21,163,037,475 |       22 |      950,400 |       1 |    0 |         1 | LTO2      | 2008-05-21 16:55:14 |
|      30 | WEEK1_001  | Used      | 18,964,770,004 |       20 |      950,400 |       1 |    0 |         1 | LTO2      | 2008-05-20 09:04:38 |
|      35 | WEEK2_003  | Used      | 19,263,924,395 |       20 |      950,400 |       1 |    0 |         1 | LTO2      | 2008-05-12 11:25:00 |
|      36 | WEEK2_004  | Used      | 33,530,520,557 |       35 |      950,400 |       1 |    0 |         1 | LTO2      | 2008-06-02 11:16:02 |
|      39 | WEEK1_004  | Used      | 15,837,863,999 |       17 |      950,400 |       1 |    0 |         1 | LTO2      | 2008-05-19 12:49:33 |
|      42 | WEEK1_003  | Used      | 18,127,185,237 |       19 |      950,400 |       1 |    0 |         1 | LTO2      | 2008-05-28 11:31:06 |
|      48 | WEEK1_002  | Purged    | 20,989,514,653 |       22 |      950,400 |       1 |    0 |         1 | LTO2      | 2008-05-14 09:57:38 |
|      51 | WEEK2_002  | Append    |  1,793,869,060 |        2 |      950,400 |       1 |    0 |         1 | LTO2      | 2008-06-02 11:26:59 |
+---------+------------+-----------+----------------+----------+--------------+---------+------+-----------+-----------+---------------------+
Pool: Incremental
+---------+-----------------+-----------+-----------------+----------+--------------+---------+------+-----------+-----------+---------------------+
| MediaId | VolumeName      | VolStatus | VolBytes        | VolFiles | VolRetention | Recycle | Slot | InChanger | MediaType | LastWritten         |
+---------+-----------------+-----------+-----------------+----------+--------------+---------+------+-----------+-----------+---------------------+
|      34 | WEEK2_005       | Error     |  12,762,444,081 |       13 |      950,400 |       1 |    0 |         1 | LTO2      | 2008-05-29 09:12:30 |
|      37 | Incremental_001 | Purged    |   6,286,363,193 |        1 |      950,400 |       1 |    0 |         1 | File      | 2008-05-13 21:50:08 |
|      38 | WEEK1_005       | Used      | 184,272,563,032 |      186 |      950,400 |       1 |    0 |         1 | LTO2      | 2008-05-29 20:09:31 |
|      40 | Incremental_002 | Purged    |   6,618,654,726 |        1 |      950,400 |       1 |    0 |         1 | File      | 2008-05-15 21:48:36 |
|      41 | Incremental_003 | Used      |   6,760,518,261 |        1 |      950,400 |       1 |    0 |         1 | File      | 2008-05-28 12:18:43 |
|      43 | Incremental_004 | Used      |   1,317,094,058 |        0 |      950,400 |       1 |    0 |         1 | File      | 2008-05-23 21:14:00 |
|      44 | Incremental_005 | Purged    |   7,399,456,778 |        1 |      950,400 |       1 |    0 |         1 | File      | 2008-05-19 21:43:34 |
|      45 | Incremental_006 | Purged    |   6,890,183,389 |        1 |      950,400 |       1 |    0 |         1 | File      | 2008-05-12 21:45:58 |
|      46 | Incremental_007 | Used      |   5,931,398,980 |        1 |      950,400 |       1 |    0 |         1 | File      | 2008-05-29 21:39:40 |
|      49 | Incremental_009 | Used      |   8,774,714,815 |        2 |      950,400 |       1 |    0 |         1 | File      | 2008-05-26 21:49:10 |
|      50 | Incremental_008 | Purged    |   6,336,229,434 |        1 |      950,400 |       1 |    0 |         1 | File      | 2008-05-20 21:47:38 |
+---------+-----------------+-----------+-----------------+----------+--------------+---------+------+-----------+-----------+---------------------+

By looking in the bacula-dir.conf I can see the retention period;
##Pools
Pool {
  Name = Default
  Pool Type = Backup
  Recycle = yes
  AutoPrune = yes
  Volume Retention = 11 days
  Accept Any Volume = yes
  Maximum Volume Jobs = 2
}

Pool {
    Name = Incremental
    Pool Type = Backup
    Recycle = yes
    AutoPrune = yes
    Volume Retention = 11 days
    Accept Any Volume = yes
    Maximum Volume Jobs = 1
}

I'd be guessing that things are kept in order by the volume retention?
11 days being the period from first using tape WEEK1_001 then through to WEEK1_005 which is 5 days (and then these tapes are sent off site), then WEEK2_001 to WEEK2_005 which is another 5 days. After the WEEK2 set we get back to WEEK1_001, which is 11 days since the last time we used WEEK1_001.

Pending this being correct, it looks like my retention + last written times are what is causing the issue.

Now can I manually force bacula to use specific volumes (I'd guess that would mean recycling?) to change it's last written and get back into order - which I think is what you mention, or should I manually run jobs to catch up?

Many Thanks,
Mark

2008/5/9 Arno Lehmann <al AT its-lehmann DOT de>:
Hi,

09.05.2008 01:33, Mark Walkom wrote:
> Hi All,
> I'm new to bacula and this has been dumped in my lap to magically fix
> somehow, so please excuse any lack of knowledge in my email! After
> reading the bacula documentation, lots of google and a few other mailing
> lists and forums I am still a tad confused over this issue and yet to
> find a resolution so hopefully someone can assist me.
>
> Basically my problem is that the tape rotation order has somehow gone awry.
> The rotation we use is WEEK1_001 to WEEK1_005 (Thursday to Wednesday, no
> changes on weekends) and that changes between two lots of tapes, WEEK1
> and WEEK2 which are alternatively stored off site. The tapes are
> manually swapped and mounted each morning by myself because the
> automount script 'someone' wrote doesn't always work .

Ok... for manual tape handling this is a reasonable approach. It is,
as you learned, a bit inflexible ;-)

> For some reason the rotation order has changed completely and now looks
> something like this (taken from the log files when bacula requests the
> next tape);
> 1_3
> 1_3
> 1_3
> 2_2
> 2_2
> 2_2
> 2_2
> 2_3
> 1_5
> 1_5
> 2_3, 1_1
> 1_2
> 1_4
> 2_1
>
> The job schedule hasn't changed, the tapes haven't been relabelled or
> marked as non-recyclable nor has an incorrect tape been inserted out of
> order, so it has me stumped on what caused it and what I need to do to
> fix this to get back to the correct order.

First we need to know how your Bacula should know which tapes to use.
Do you have them in different pools, or are they all in the same pool
and you rely on the retention times to ensure the proper order of
recycling?

Also, is this with using an autochanger or a single drive?

> Unfortunately the preceding sysadmin only really taught me how to mount
> and run the basic status commands

That's really not much if you have to operate Bacula...

> which means I am learning this all
> from scratch.

Yup. You'll want a copy of the manual for bedtime reading :-)

(This is not going to help you now, but I suggest you start playing
with bconsole a bit - try the different list, llist and show commands
and see if you understand what they report. Then compare that to your
configuration and see why they report what they report. And then try
to think how *you* would solve your tape rotation problem - in the end
you'll know quite a bit about how to configure Bacula.)

>
> Running bacula Version: 1.38.11 (28 June 2006) on Debian 3.1 (Linux
> 2.6.18-6-686).

This is seriously outdated. I recommend you plan an upgrade to a
current version.

> Any help is greatly appreciated.

Ok, let's see... first, we need a bit more information (see above -
which pools etc.) Then you have to check if the actual volume settings
match the pool definition.

Then you check why Bacula selects the volumes it wants, i.e. the order
in which the volumes can be recycled (defined by LastWritten plus
retention time).

If you find Bacula can use the tape you want it to use you simply
(manually) ensure it has to use these volumes. Just disable all the
others, or only load the one volume you want it to use.

When not using an autochanger this is even simpler - just load the
tape you want it to use and, if necessary, i.e. the volume is not yet
recyclable, you manually recycle it.

After a complete backup cycle the order should be correct again.

You could also manipulate the catalog data to fix all volumes in one
go, but that is definitely a more advanced technology :-)

Arno

> Regards,
> Mark

--
Arno Lehmann
IT-Service Lehmann
www.its-lehmann.de

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
<Prev in Thread] Current Thread [Next in Thread>
  • Re: [Bacula-users] Bacula volume (tape) rotation out of order, Mark Walkom <=