Bacula-users

Re: [Bacula-users] [Bacula-devel] feature suggestion: convert old full backups to "reverse" incremental backups

2009-09-03 10:18:11
Subject: Re: [Bacula-users] [Bacula-devel] feature suggestion: convert old full backups to "reverse" incremental backups
From: Gavin McCullagh <gavin.mccullagh AT gcd DOT ie>
To: bacula-devel AT lists.sourceforge DOT net
Date: Thu, 3 Sep 2009 14:52:13 +0100
Hi Kern,

many thanks for your reply.

On Thu, 03 Sep 2009, Kern Sibbald wrote:

> This sounds to me like it would be resolved by our Base project (though 
> perhaps not as automatically as you would like).  See the "projects" file for 
> details.  This project is currently being implemented for the next major 
> release.

It does solve the particular example that I presented, but I'm afraid that
reflects my poor choice of example.  I'll use another more general one.

Apologies that this email is a little long.

We have a mail server using a maildir-type format (ie every email has a
unique file).  Some people like to keep email for years and to be honest,
we don't discourage that or want to.  As a result we find our mail spool
disk usage grows moreorless linearly in time.  So, we have new files
appearing in the backups all the time which often never go away.  We
currently store backups on-disk, using a method based on rsync and hard
links.  No file ever gets duplicated and at any time I can (in principal)
delete the backup job from any date (oldest, newest, ...) without affecting
the others.  We have backups going back several years.

         http://www.mikerubel.org/computers/rsync_snapshots/

We're moving all of our backups to bacula and this is the one backup system
that I'm finding hard to migrate.   Some duplication will be fine, but not
a constantly growing amount.  You could create a single full backup on day
1 and then create differentials each month indefinitely but that would
duplicate a lot of data.  You could create a single backup on day 1 and
store incrementals indefinitely forward but the restore time for recent
backup jobs would grow as time went on.  It seems important to always have
a recent full backup.

What I'd like to have is N full backups (say 1 per month for the past 6)
and to keep incremental jobs for the previous months which could bring me
back to months 7, 8, 9, etc.  Having seen the virtualfull backup, it's
clear that you can create full backup jobs out of other existing (full and
incremental) backup jobs.  What I propose is that each month, before a new
full volume gets created and the oldest full volume is recycled, a job
could be run to derive an incremental backup from the 6 month old volume to
the 5 month old one.  Then the 6-month old volume gets recycled, but I can
continue to get that month.  

This would mean that the restore time would be larger the further back you
need to go in time, but I'd say the likelihood of needing to consult them
decreases, so that makes some sense.  Duplication of files would be limited
to N (in this case 6) copies but you could get back much further.

I suppose an alternative approach would be to store:

 - one original full backup
 - incremental backups from day one for each subsequent month
 - full backups for the last 6 months

but this would make 7 months ago the longest restore time which is not
ideal and I think you'd effectively end up running the two backup strategys
in parallel.

I hope this makes sense.  I'm trying my best to be clear here but if it's
not yet I'm happy to try again :-)

Gavin


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users