Bacula-users

Re: [Bacula-users] Volume retention periods

2011-02-22 13:12:00
Subject: Re: [Bacula-users] Volume retention periods
From: Bob Hetzel <beh AT case DOT edu>
To: bacula-users AT lists.sourceforge DOT net
Date: Tue, 22 Feb 2011 13:09:42 -0500

> Date: Mon, 21 Feb 2011 15:43:52 -0500
> From: Javier Gomez <gomez AT dynamicquest DOT com>
> Subject: [Bacula-users] Volume retention periods
> To: bacula-users AT lists.sourceforge DOT net
>
>      We have been using Bacula for a few years now and have enjoyed its
> features.  But I need some help to understand one key retention period
> concept.  Our fundamental issue is attempting to make sure we have good
> backups for a minimum amount of retention period.  Our issue is being
> able to set a File Retention, Job Retention, and/or Volume Retention for
> a period of time with using the least amount of space but ensuring that
> we have a solid 30 days of total backups.  If I had to label the
> properties I would think this would be called:
> MinBackupRetentionPeriod.  Which sounds easy, but let me describe an
> example:
>
> Assume we have the following:
>  >  400 servers being backed up
>  >  We want a minimum of 30 day of solid backups for retention.
>  >  We use File type storage for all of our backups
>  >  Since we use File storage we want to use the space the smartest way
>  >  Assume that if a Full fails to run that it is considered lost and we
> need to wait until the following scheduled Full backup (next weekend)
>
> Business Requirement:
> - We need to be able to pull a backup of any file for any day within the
> past 30 day period of time.
>
> Not so perfect world:
> - So in a perfect world this is very easy to accomplish.  But lets just
> say for purposes of discussion that we have the following backup schedule:
>
> Full Backups - Every Weekend
> Incremental Backups - Every Weekday
>
> Lets assume that we have a File Retention and Job retention set to 30
> days each.  Now lets also assume that we have two Weekends where the
> Full backups fail for one of the servers.  No big deal since the next
> weekend we can get a new Full.  So to put dates on this:
>
> Full Backup:  01/01/11   - Successful
> Full Backup:  01/08/11   - Failed
> Full Backup:  01/15/11   - Failed
> Full Backup:  01/22/11   - Successful
>
> Lets assume all of the Incremental backups were also Successful in
> between the fulls.  So now its 02/13/11.  The business user asks for a
> file that was on the server on 01/16/11.  Noticed that since our Job and
> File retention is set to 30 days and the 01/01/11 Full Backup has been
> purged, that we now can not recover that file since we are missing the
> Full Failed Backup from the 01/15/11 date.  We have the Incrementals but
> without the Full we can not have a consistent recovery.
>
> So the Question is:  Does Bacula have a way to make sure that it holds
> onto the last successful Backup in order to maintain integrity
> regardless of the Retention period.  So the ideal concept would be that
> Bacula would hold onto a Volume/Job/File based on its Retention period
> AND the retention date of its Dependent Backups.  So in this picture, it
> would be great if Bacula would hold onto the 01/01/11 Full Backup for
> the 30 retention period assigned to the 01/21/11 last dependent
> Incremental Backup.
>
> This may be possible already within Bacula, or maybe some other
> combination of flags would make it easier to implement that I have not
> seen yet.  The documentation seems to say that the Retention periods are
> based on a hard time frame and once that time frame has passed, then the
> files are free to be purged.  Any thoughts or advise...?  Thanks...

There may be a reason for you not to use this setting, but there is this 
setting

Rerun Failed Levels = yes

In your example, on the next backup after 1/8/11 you'd then get a full 
backup instead of an incremental backup.

Also note that if the file had changed on 1/15/11 or 1/16/11 you'd still 
get it backed up in an incremental.

Regarding your specific question about retention times, yes bacula is set 
up such that it's possible to purge a full backup w/o purging the 
incrementals that came after it.  If you aren't careful you can purge fulls 
before incrementals (but you may not even care about that... see below). 
The upside to that is it's far simpler to code and administer.  Keep in 
mind the following:  The most recent full, plus the incrementals following 
it are all you'll most likely need for disaster recovery.  I would want to 
go at least two cycles deep in my data retention policies in case a tape 
breaks.  Files that didn't change between the two backups wouldn't be 
different in either full backup anyway.  Files that change frequently would 
get picked up more than once.

The question of how long to hold my backups beyond that depends upon the 
answers to these two questions:
a) How many tapes do I have available or can I afford to purchase?
b) How much risk can I tolerate with respect to accidentally deleted files? 
  Am I likely to find out about an accidentally deleted file right away or 
perhaps only week, month, quarter, 6 months 12 months later.

Sadly, it's a fact of life that you're always going to get situations where 
you didn't save a backup for long enough.  Your backup strategy should be 
geared towards determining the acceptable risk level for your organization.



------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT data 
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

<Prev in Thread] Current Thread [Next in Thread>