Bacula-users

Re: [Bacula-users] Retention not being applied correctly

2011-03-22 11:31:08
Subject: Re: [Bacula-users] Retention not being applied correctly
From: James Woodward <james.woodward AT ualberta DOT ca>
To: dermot.beirne AT dpd DOT ie
Date: Tue, 22 Mar 2011 09:28:05 -0600
Hello Dermot,

In the testing I've done I found that if I had one pool set to something 
ridiculous like 1 day then when the backup was written to that pool that 
retention period was applied to all copies of that particular job. As an 
example if I had a 3 pools with full backups being kept in a pool for 60 days 
and then a differential pool which holds backups for 7 days and a incremental 
pool that holds backups for 1 day. If I run a incremental backup then any job 
that was over a day old is pruned from the database no matter what pool it was 
in. 

I tested this by setting up all pools with a 30 day retention initially. I ran 
backups for a week with a mix of full, differential and incremental backups. I 
then modified the incremental pool to have a retention period of 1 day and 
updated the pools from the configuration. When I ran an incremental backup all 
backups except the one from the previous day were removed from the database. 

I haven't spent any more time looking at it to see what other combinations 
would do. I just put all my retention periods up to 60 days until the pruning 
algorithm is fixed. I was pretty sure I read something that said it would be 
fixed in 5.1 but I can't say for sure where I read that.

James



On 2011-03-22, at 9:05 AM, Dermot Beirne wrote:

> Hi James,
> Thanks for replying.
> If I understand you correctly, then I think that is what I'm doing.
> I have a very high File and Job retention on the clients resource (10
> years), and a volume retention on the pools that I actually do want to
> use, e.g. 13 days, 60 days, 1 year, 10 years depending on the pool.
> 
> It appears though, the files are still being pruned.  I believe they
> should not be pruned until the volume is.
> 
> Is that what you have implemented?
> 
> Dermot.
> 
> 
> 
> On 22 March 2011 14:13, James Woodward <james.woodward AT ualberta DOT ca> 
> wrote:
>> Hello,
>> 
>> I've only seen it on the list and from some testing. The retention periods 
>> are broken and I don 't think it will be fixed until 5.1. When a job is 
>> written to the pool with a lower retention all jobs and files are pruned 
>> according to that retention period. My solution was to bump everything up to 
>> the higher retention period and then I'll lower the pools later on when I've 
>> confirmed that it's fixed in 5.1 and we upgrade. This isn't much of an issue 
>> for me since everything goes to a tape library. It may not work well for you.
>> 
>> James
>> 
>> On 2011-03-22, at 7:02 AM, Dermot Beirne wrote:
>> 
>>> Hi,
>>> I use Bacula 5.0.3 on ubuntu 9.04
>>> 
>>> I backup from a number of clients (windows and linux) to a disk raid
>>> and then migrate to tape afterwards.
>>> I am concerned that my retention is not working properly.
>>> 
>>> This is my client config for one client:
>>> 
>>> 
>>> # Client (File Services) to backup
>>> Client {
>>>  Name = interlink-pdc-fd
>>>  Address = 10.1.8.11
>>>  FDPort = 9102
>>>  Catalog = MyCatalog
>>>  Password = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"      # password for 
>>> FileDaemon
>>>  Maximum Concurrent Jobs = 1
>>>  AutoPrune = yes                     # Prune expired Jobs/Files
>>>  Job Retention = 10 years
>>>  File Retention = 10 years
>>> }
>>> 
>>> When I run "query" and choose:
>>> 5: List all backups for a Client
>>> 
>>> The oldest one is:
>>> 
>>> | JobId  | Client           | FileSet               | Level |
>>> StartTime           | JobFiles  | JobBytes        | VolumeName |
>>> +--------+------------------+-----------------------+-------+---------------------+-----------+-----------------+------------+
>>> | 10,614 | interlink-pdc-fd | interlink-pdc Fileset | F     |
>>> 2010-12-05 22:00:00 | 1,580,321 | 188,248,232,869 | 000040L4
>>> 
>>> If I choose "restore" and option 3: Enter list of comma separated
>>> JobIds to select, and enter jobid 10614
>>> I get:
>>> 
>>> Select item:  (1-13): 3
>>> Enter JobId(s), comma separated, to restore: 10614
>>> You have selected the following JobId: 10614
>>> 
>>> Building directory tree for JobId(s) 10614 ...
>>> 
>>> For one or more of the JobIds selected, no files were found,
>>> so file selection is not possible.
>>> Most likely your retention policy pruned the files.
>>> 
>>> Do you want to restore all the files? (yes|no):
>>> 
>>> 
>>> The only other place where retentions are specified is in the pool.
>>> I have 4 disk pools, daily, weekly, monthly and yearly.
>>> There is a Volume Retention specified here which ranges from 2 to 4
>>> days, to ensure the disk volumes are freed up quickly after migration.
>>> 
>>> I also have 4 tape pools, where Volume Retention ranges from 13 days
>>> to 10 years.
>>> 
>>> My understanding is that the pool retention takes precedence over the
>>> client.  So keeping a high file and job retention on the clients, and
>>> then the correct required retention on the volume in the pool, should
>>> ensure the catalog file and job contents are preserved until the
>>> volume is recycled.
>>> 
>>> This does not appear to be happening.
>>> 
>>> Would anyone know why this is happening?  Thank you.
>>> 
>>> Dermot.
>>> 
>>> ------------------------------------------------------------------------------
>>> Enable your software for Intel(R) Active Management Technology to meet the
>>> growing manageability and security demands of your customers. Businesses
>>> are taking advantage of Intel(R) vPro (TM) technology - will your software
>>> be a part of the solution? Download the Intel(R) Manageability Checker
>>> today! http://p.sf.net/sfu/intel-dev2devmar
>>> _______________________________________________
>>> Bacula-users mailing list
>>> Bacula-users AT lists.sourceforge DOT net
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>> 
>> 
>> 
>> 
> 


------------------------------------------------------------------------------
Enable your software for Intel(R) Active Management Technology to meet the
growing manageability and security demands of your customers. Businesses
are taking advantage of Intel(R) vPro (TM) technology - will your software 
be a part of the solution? Download the Intel(R) Manageability Checker 
today! http://p.sf.net/sfu/intel-dev2devmar
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

<Prev in Thread] Current Thread [Next in Thread>