Bacula-users

Re: [Bacula-users] premature recycling

2010-10-22 11:42:43
Subject: Re: [Bacula-users] premature recycling
From: Martin Simmons <martin AT lispworks DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Fri, 22 Oct 2010 16:40:01 +0100
>>>>> On Thu, 21 Oct 2010 18:08:06 -0400, Dan Langille said:
> 
> On 10/21/2010 12:10 PM, Martin Simmons wrote:
> >>>>>> On Thu, 21 Oct 2010 10:51:51 -0300, Kleber Leal said:
> >>
> >> 2010/10/21 Martin Simmons<martin AT lispworks DOT com>
> >>
> >>>>>>>> On Wed, 20 Oct 2010 15:34:42 -0400, Dan Langille said:
> >>>>
> >>>> I'm getting recycling before retention expires.  Or so I think.  It may 
> >>>> be
> >>>> possible that the Pools definitions are out of date, but before I run an
> >>>> update command, I wanted to see if anyone saw something out of order 
> >>>> here.
> >>>>
> >>>> This is the Volume before it was recycled:
> >>>>
> >>>> Pool: FullsFile
> >>>> | mediaid | volumename    | volstatus | enabled | volbytes      |
> >>> volfiles | volretention | recycle | slot | inchanger | mediatype |
> >>> lastwritten         |
> >>>> |     445 | FileAuto-0445 | Full      |       1 | 5,368,691,646 |
> >>>   1 |   94,608,000 |       1 |    0 |         0 | File      | 2010-04-16
> >>> 08:24:59 |
> >>>>
> >>>> That is 3 years retention. Last written April this year
> >>>>
> >>>> The Volume is then recycled during the job:
> >>>>
> >>>> 20-Oct 20:18 kraken-sd JobId 39232: User defined maximum volume capacity
> >>> 5,368,709,120 exceeded on device "MegaFile"
> >>> (/storage/compressed/bacula/volumes).
> >>>> 20-Oct 20:18 kraken-sd JobId 39232: End of medium on Volume
> >>> "FileAuto-0437" Bytes=5,368,688,857 Blocks=83,220 at 20-Oct-2010 15:18.
> >>>> 20-Oct 20:18 bacula-dir JobId 39232: There are no more Jobs associated
> >>> with Volume "FileAuto-0445". Marking it purged.
> >>>> 20-Oct 20:18 bacula-dir JobId 39232: All records pruned from Volume
> >>> "FileAuto-0445"; marking it "Purged"
> >>>> 20-Oct 20:18 bacula-dir JobId 39232: Recycled volume "FileAuto-0445"
> >>>> 20-Oct 20:18 kraken-sd JobId 39232: Recycled volume "FileAuto-0445" on
> >>> device "MegaFile" (/storage/compressed/bacula/volumes), all previous data
> >>> lost.
> >>>> 20-Oct 20:18 kraken-sd JobId 39232: New volume "FileAuto-0445" mounted on
> >>> device "MegaFile" (/storage/compressed/bacula/volumes) at 20-Oct-2010 
> >>> 15:18.
> >>>
> >>> I think you might be confused about the meaning of the Volume Retention
> >>> directive.  It controls automatic pruning, but it doesn't prevent 
> >>> recycling if
> >>> the volume has no jobs remaining in the catalog.  The latter can happen 
> >>> if the
> >>> Job Retention directive in the Client(s) caused pruning prior to this
> >>> backup.
> >>
> >> He is not confused. I think.
> >> The bacula should not to recycle a volume, even if there is not appendable
> >> volumes*.
> >> *The manual says: *"**The Volume Retention record defines the length of 
> >> time
> >> that Bacula will guarantee that the Volume is not reused** counting from 
> >> the
> >> time the last job stored on the Volume terminated". *Not mention to jobs on
> >> catalog.
> >>
> >> The files and job retention period comes to prevent catalog growing, not to
> >> recycle volumes, says the manual.
> >>
> >> *The Volume Retention record defines the length of time that Bacula will
> >> guarantee that the Volume is not reused counting from the time the last job
> >> stored on the Volume terminated.
> >
> > It seems that the manual contradicts itself about this directive, because
> > http://www.bacula.org/5.0.x-manuals/en/main/main/Configuring_Director.html#SECTION0018150000000000000000
> > in the same manual defines the Volume Retention directive as:
> >
> > "The Volume Retention directive defines the length of time that Bacula will
> > keep records associated with the Volume in the Catalog database after the 
> > End
> > time of each Job written to the Volume."
> >
> > I.e. it controls pruning, not recycling.
> 
> Which in turn affects recycling.
> 
> > If a Full or Used volume has no jobs in the catalog, then it can be marked 
> > as
> > Purged and can be recycled.  AFAICS, the code has worked like that for a 
> > long
> > time and the message
> >
> > "There are no more Jobs associated with Volume .... Marking it purged."
> >
> > is printed when it detects that, regardless of the Volume Retention period.
> 
> I suggest that jobs are being pruned before they should be pruned. 

Yes, it looks like that.


> Perhaps I've misunderstood it all these years. If the retention period 
> is 1 year (default), why are jobs being pruned before before that?  The 
> last written value is April.

The Job Retention directive in the Client resource will cause it (the default
is 180 days, so April would be about right).

It will report messages like this after a job:

28-Sep 02:10 xxx-dir JobId 30060: Begin pruning Jobs older than 6 months .
28-Sep 02:10 xxx-dir JobId 30060: Pruned 9 Jobs for client xxx-fd from catalog.

This prunes from all pools, not just the one that was used by the job.

__Martin

------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store 
http://p.sf.net/sfu/nokia-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users