Bacula-users

Re: [Bacula-users] [Bacula-devel] feature suggestion: convert old full backups to "reverse" incremental backups

2009-09-03 11:15:21
Subject: Re: [Bacula-users] [Bacula-devel] feature suggestion: convert old full backups to "reverse" incremental backups
From: Kern Sibbald <kern AT sibbald DOT com>
To: bacula-devel AT lists.sourceforge DOT net, Gavin McCullagh <gavin.mccullagh AT gcd DOT ie>
Date: Thu, 3 Sep 2009 17:11:12 +0200
Hello,

I suspect that all that is already possible with Bacula but may require a bit 
of scripting.  In short, you need support from a Bacula specialist, and we 
don't do that on this list (bacula-devel).

What you are asking for is a bit specialized, so I doubt that it is something 
that would be directly implemented in Bacula, unless we saw that it was 
something a lot of users want, which means it would need to be a very clearly 
defined Feature Request, voted on, then someone would need to volunteer to 
write it.  

As I say, the other solution is to ask on the bacula-users list or ask for 
professional help.

Thanks for using Bacula.

Best regards,

Kern

PS: Be careful with programs like BackupPC that create a lot of hard links.  
If you create enough of them, you will not be able to boot your system 
because fsck must keep track of all the hard linked files in memory, so if 
you have too many hard links and not enough memory, you may one day be stuck.
The same problem exists with restoring hard links with Bacula.  On a normal 
system that has a few hundred or even a hundred thousand, no problem, but if 
you start getting millions of them you may be in trouble.

On Thursday 03 September 2009 15:52:13 Gavin McCullagh wrote:
> Hi Kern,
>
> many thanks for your reply.
>
> On Thu, 03 Sep 2009, Kern Sibbald wrote:
> > This sounds to me like it would be resolved by our Base project (though
> > perhaps not as automatically as you would like).  See the "projects" file
> > for details.  This project is currently being implemented for the next
> > major release.
>
> It does solve the particular example that I presented, but I'm afraid that
> reflects my poor choice of example.  I'll use another more general one.
>
> Apologies that this email is a little long.
>
> We have a mail server using a maildir-type format (ie every email has a
> unique file).  Some people like to keep email for years and to be honest,
> we don't discourage that or want to.  As a result we find our mail spool
> disk usage grows moreorless linearly in time.  So, we have new files
> appearing in the backups all the time which often never go away.  We
> currently store backups on-disk, using a method based on rsync and hard
> links.  No file ever gets duplicated and at any time I can (in principal)
> delete the backup job from any date (oldest, newest, ...) without affecting
> the others.  We have backups going back several years.
>
>        http://www.mikerubel.org/computers/rsync_snapshots/
>
> We're moving all of our backups to bacula and this is the one backup system
> that I'm finding hard to migrate.   Some duplication will be fine, but not
> a constantly growing amount.  You could create a single full backup on day
> 1 and then create differentials each month indefinitely but that would
> duplicate a lot of data.  You could create a single backup on day 1 and
> store incrementals indefinitely forward but the restore time for recent
> backup jobs would grow as time went on.  It seems important to always have
> a recent full backup.
>
> What I'd like to have is N full backups (say 1 per month for the past 6)
> and to keep incremental jobs for the previous months which could bring me
> back to months 7, 8, 9, etc.  Having seen the virtualfull backup, it's
> clear that you can create full backup jobs out of other existing (full and
> incremental) backup jobs.  What I propose is that each month, before a new
> full volume gets created and the oldest full volume is recycled, a job
> could be run to derive an incremental backup from the 6 month old volume to
> the 5 month old one.  Then the 6-month old volume gets recycled, but I can
> continue to get that month.
>
> This would mean that the restore time would be larger the further back you
> need to go in time, but I'd say the likelihood of needing to consult them
> decreases, so that makes some sense.  Duplication of files would be limited
> to N (in this case 6) copies but you could get back much further.
>
> I suppose an alternative approach would be to store:
>
>  - one original full backup
>  - incremental backups from day one for each subsequent month
>  - full backups for the last 6 months
>
> but this would make 7 months ago the longest restore time which is not
> ideal and I think you'd effectively end up running the two backup strategys
> in parallel.
>
> I hope this makes sense.  I'm trying my best to be clear here but if it's
> not yet I'm happy to try again :-)
>
> Gavin
>
>
> ---------------------------------------------------------------------------
>--- Let Crystal Reports handle the reporting - Free Crystal Reports 2008
> 30-Day trial. Simplify your report design, integration and deployment - and
> focus on what you do best, core application coding. Discover what's new
> with Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Bacula-devel mailing list
> Bacula-devel AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-devel



------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users