Bacula-users

Re: [Bacula-users] About retention, and pruning.

2011-05-04 05:59:21
Subject: Re: [Bacula-users] About retention, and pruning.
From: Hugo Letemplier <hugo.let.35 AT gmail DOT com>
To: Graham Keeling <graham AT equiinet DOT com>
Date: Wed, 4 May 2011 11:56:06 +0200
2011/5/4 Graham Keeling <graham AT equiinet DOT com>:
> On Fri, Apr 29, 2011 at 11:11:24AM +0200, Hugo Letemplier wrote:
>> 2011/4/29 Jérôme Blion <jerome.blion AT free DOT fr>:
>> > On Thu, 28 Apr 2011 17:33:48 +0200, Hugo Letemplier
>> > <hugo.let.35 AT gmail DOT com>
>> > wrote:
>> >> After the job ran many times: I have the following volume <=> job
>> > matching
>> >> Vol name   Level      Time
>> >> Test1         Full        15:50
>> >> 324            Inc         16:00
>> >> 325            Inc         16:10
>> >> 326            Inc         16:20
>> >> 324            Inc         16:30
>> >> Test2         Full        16:40
>> >> 325            Inc         16:50
>> >> 326            Inc         17:00
>> >>
>> >> This is problematic because Vol324 is recycled instead of creating a new
>> >> one
>> >> I am not sure to understand the various retention periods : File, job,
>> >> volume
>> >> I think that I can increase the retention times but the problem will
>> >> always be the same.
>> >> ex : if I keep my incremental one hour then my first ones will always
>> >> be purged first
>> >> In a good strategy you purge the full sequence of incremental at the
>> >> same time because you need to recycle you volume and don't want to
>> >> keep a recent volume (incremental) without the previous ones.
>> >
>> > You would waste your tape/disk space.
>> >
>> >> To do that I imagine that I need to create one pool per day and reduce
>> >> progressively the retention periods. It doesn't makes sense !
>> >>  I turned the problem on all its sides but I cant find a good
>> >> solution. Maybe the other retention period are the solution but I
>> >> didn't succeeded ?
>> >> Thanks in advance
>> >
>> > That means that your upper backup levels should have greater retentions to
>> > be sure that at any time, you can use the full + diff + inc if needed.
>> > Keeping incremental without full backup can be useful to restore only
>> > specific files.
>> Yes, but this problem is the same between incremental backups:
>> Lots of people recommended me to use one pool per level:
>> It works for Full and differentials, but not for inc pool
>> Maybe one inc-pool per "incremental run of a scheduling cycle" should
>> be good ? But it 's not simple
>> I think that a new feature that add dependency between various job
>> levels for the next versions of bacula could be cool.
>> The idea is to allow pruning only for volume/jobs that aren't needed
>> by other ones whatever are the retention time.
>> As a consequence : you can prune a full only (((if the differential is
>> pruned) if the XXX incrementals are pruned) if the last incremental is
>> pruned )
>> So you you can say that the maximum retention time for a full is at
>> least equal to the retention time of the last inc + the delay between
>> the full and the this last inc so you have something like this :
>> full  : ========================>>>>>>>>
>> inc  :   =========>>>>>
>> inc  :     =========>>>>
>> inc  :       =========>>>
>> inc  :         =========>>
>> inc  :           =========>
>> inc  :             =========
>> diff  :               ================>
>> inc  :                 =========>>>>>
>> inc  :                   =========>>>>
>> inc  :                     =========>>>
>> inc  :                       =========>>
>> inc  :                         =========>
>> inc  :                           =========
>> diff  :                             ================>
>> inc  :                               =========>>>>>
>> inc  :                                 =========>>>>
>> inc  :                                   =========>>>
>> inc  :                                     =========>>
>> inc  :                                       =========>
>> inc  :                                         =========
>>
>> and not like that :
>> diff  : ==================>
>> inc  :   =======>
>> inc  :     =======>
>> inc  :       =======>
>>
>> What do you think about such a feature ?
>
> A while ago, I made a patch that does it. Nobody seemed to want it though.
> http://www.adsm.org/lists/html/Bacula-users/2011-01/msg00308.html
>
>
> ------------------------------------------------------------------------------
> WhatsUp Gold - Download Free Network Management Software
> The most intuitive, comprehensive, and cost-effective network
> management toolset available today.  Delivers lowest initial
> acquisition cost and overall TCO of any competing solution.
> http://p.sf.net/sfu/whatsupgold-sd
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
First I think your patch would be nice to have that kind of "security"
option to avoid breaking backup dependency.
I think it's strange that nobody else needed this.
For the moment I don't want to leave the official packages and compile
a new bacula.

I found some tip to do it as I want, but it is still possible to have mistakes.

1 volume / full
1 volume / diff
1 volume / incremental sequence
An incremental sequence means all the successive increments between
fulls and/or diffs
Then I used Volume use duration. I defined this directive to the
duration of my incremental sequence ex : one week.
After one week the volume is not used anymore.
I specify a volume retention period of three weeks so that the lasts
incremental job will upgrade the retention time  of this volume.

For the moment I do Differentials every weeks and I manage the full
manually ( still in a test/optimization phase ) I do incremental
backups daily

Thanks,

Hugo

------------------------------------------------------------------------------
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network 
management toolset available today.  Delivers lowest initial 
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users