BackupPC-users

Re: [BackupPC-users] Incremental Seems To Backup Whole System

2010-02-18 21:29:30
Subject: Re: [BackupPC-users] Incremental Seems To Backup Whole System
From: John Rouillard <rouilj-backuppc AT renesys DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Fri, 19 Feb 2010 02:27:43 +0000
On Thu, Feb 18, 2010 at 06:06:15PM -0700, Mike Bydalek wrote:
> On Thu, Feb 18, 2010 at 12:04 PM, John Rouillard
> <rouilj-backuppc AT renesys DOT com> wrote:
> > On Thu, Feb 18, 2010 at 07:51:13AM -0700, Mike Bydalek wrote:
> >> My question is, why did backups 13 and 14 backup all that data?  Same
> >> with 2 and 7 for that matter.
> >
> > What level are your incremental backups? if backup 2 was at level 1
> > and backup 7 was at level 1 (you use levels 1 2 3 4 5 6) and backup 13
> > is back at level 1 that's kind of what I would expect since level 1
> > backs up everything since since the last full.
> >
> > However 14 should be quite a bit less unelss it was also a level 1.
> >
> >> Below is my config.  I'm still messing with the IncrLevels and have a
> >> super short period just to get some increments and all that going.
> >  [...]
> >> $Conf{IncrLevels} = [
> >>   '1',
> >>   '2',
> >>   '3',
> >>   '4',
> >>   '5',
> >>   '6'
> >> ];
> >
> 
> After re-reading the documentation for {IncrLevels} again the
> configuration settings are starting to make sense.  The only question
> I have left is, does creating a new "full" backup *have* to do the
> entire full backup again?  Can't it just perform an increment and
> merge it to create a full?  The reason I ask is I'm planning on moving
> this server off-site so it'll go over a WAN.  Sending 250G over a 1M
> connection every week or two doesn't sound fun! 

If you are using rsync, it will only transfer new/incremental data.
If you are using tar I think it transfers everything. I assume you
aren't using smb across the internet 8-). Ftp will transfer
everything.
 
> Is this what $Conf{IncrFill} is supposed to handle?

No, that controls what the tree in the storage directory looks like.
Normally an incremental on disk (under the pc/hostname/number
directory) consists only of new files. The merging of multiple
incremental backups and fulls is done by the web interface so by
browsing backup 27 you can restore /etc/password that was backed up in
run 0.

If you want to sync an entire backup tree offsite (e.g. for disaster
recovery under another backuppc instance) and you sync an incremental
you will be missing most of the files. If you sync a filled
incremental however you actually have a merged copy (on disk) of all
the files on the system from the prior backups. A copy of the merged
incremental can be used to restore a system.

> What I want is to basically perform a backup every day and keep 30
> days of backups without doing another 'full' backup.  I don't really
> care how many 'full' backups I have as long as I can restore from 29
> days ago.  Would these settings do the trick for that?
> 
> $Conf{FullPeriod}  = 30;
> $Conf{IncrPeriod}  = 1;
> $Conf{IncrKeepCnt} = 30;
> $Conf{IncrLevels}  = [1, 2, 3, 4, 5, 6 ...... 30];
> $Conf{IncrFill} = 1;

They will, but to restore backuppc will have to merge the files from
30 incrementals. This could slow down the web interface or
BackupPC_zip/tar generation. If you do something like (untested):

> $Conf{FullPeriod}  = 8;
> $Conf{FullKeepCount} = 4;
> $Conf{IncrPeriod}  = 1;
> $Conf{IncrKeepCnt} = 28;
> $Conf{IncrLevels}  = [1, 2, 3, 4, 5, 6, 7];
> $Conf{IncrFill} = 0;

you should have a full every 8 days and you keep 4 of them so you get
32 days of coverage. You do incrementals between every full and keep
28 of them (which leads to 32 day coverage 28 incrementals + 4 fulls).
But to restore you only have 8 backups that have to be merged together
to create a valid restore.

Also an advantage to this is that you get fulls more often and these
fulls should take less time to run than the full done every 30
days. E.G. assume you change 2GB of data every 8 days. Each full will
need to sync only 2GB of data. In your original scheme the full will
have to move ~8GB of data (4x2). If you have a defined time window in
which a backup must complete, you need to schedule your fulls so that
they can complete within the window and having more frequent fulls
does this.

-- 
                                -- rouilj

John Rouillard       System Administrator
Renesys Corporation  603-244-9084 (cell)  603-643-9300 x 111

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/