Re: [Bacula-users] Help on retention period

2011-01-12 15:53:21
Subject: Re: [Bacula-users] Help on retention period
From: Blake Dunlap <ikiris AT gmail DOT com>
To: Graham Keeling <graham AT equiinet DOT com>
Date: Wed, 12 Jan 2011 14:50:36 -0600

On Wed, Jan 12, 2011 at 11:41, Graham Keeling <graham AT equiinet DOT com> wrote:
On Wed, Jan 12, 2011 at 10:54:13AM -0600, Mark wrote:
> On Wed, Jan 12, 2011 at 9:35 AM, Valerio Pachera <sirio81 AT gmail DOT com> wrote:
> >
> > SCOPE: We want the possibility of restore any file till 2 weeks ago.
> >
> ...
> > at sunday of the third week, the first full backup get overwritten.
> >
> > _ _ _ _ _ _ | _ _ _ _ _ _ |
> >
> > This means that, of the first week, I can only restore file present in
> > the incremental backup.
> > In other words I do not have a cicle of 2 weeks but 1.
> >
> >
> When your first week's full backup gets overwritten, what are those
> incremental backups "incremental to"?  What's you're describing sounds like
> what I expect fulls and incrementals to be.  When you overwrite the full,
> you've essentially orphaned the incrementals that were created based on that
> full backup.

Bacula doesn't prevent backups that other backups depend on from being

If there is no previous Full before the Incrementals, you cannot easily restore
the files in the Incrementals. You have to extract the files from the
individual volumes with the 'bextract' command.

It also doesn't have anything that indicates that a particular backup was
based on another particular backup. It calculates everything based on dates.

So, if an Incremental from the middle of a sequence got purged (somehow),
bacula won't notice and will happily restore from the latest Incremental.
It is good that you can restore something, but bad because the files you get
back may well not be the same as the ones that were on your client machine on
the day that the Incremental was made.

Anyway, this all means that you need to set your retention times very

If you set them so that they cover the periods that you're worried
about - like Valerio's example of wanting to restore from two weeks back...
F I I I I I I F I I I I I I I F might decide to set the retention of Fulls to 3 weeks.
But be careful! If, for some reason, a Full backup fails, time will march on
and you will end up having a Full that other backups depend on getting purged
(imagine the 2nd 'F' in the sequence above being missed out).

I actually wrote a patch that enforces that bacula only purges backups
that other backups don't depend on. But it makes some assumptions about your
setup. It assumes that you are using one job per volume and it assumes that
you are not using Job / File retentions (ie, you have set them very high) and
instead rely purely on Volume retentions.

So, if your retention is set to one week, the purge sequence will be like this
(with new backups being made on the right).

F I I I I I   F I I I I I I F I
F I I I I     F I I I I I I F I I
F I I I       F I I I I I I F I I I
F I I         F I I I I I I F I I I I
F I           F I I I I I I F I I I I I
F             F I I I I I I F I I I I I I
             F I I I I I I F I I I I I I F
             F I I I I I   F I I I I I I F I

This also suffers if a Full backup is missed, because you end up having to
keep more old backups.

I can upload this patch if it is interesting to people.

P.S. One last thing - try not to worry about what bacula does if your clock
somehow goes wrong and decides that it is 2037. Or 1970. Or last week. :)

This is supposedly better in the upcoming version at least in regards to pruning. If they have gotten the pruning to work properly on pools as well, it could potentially make a lot of lives easier. I personally never understood the logic behind the implicit assumptions that the restore and backup algorithms make in regards to job tree structure, but at least things are progressing. I'm just still hoping that they will some day include a more realistic concept for management of locations of data and the implications related to backups / restores.

Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand 
malware threats, the impact they can have on your business, and how you 
can protect your company and customers by using code signing.
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
<Prev in Thread] Current Thread [Next in Thread>