BackupPC-users

Re: [BackupPC-users] Designing a BackupPC install over a WAN - minimising Full backups

2009-01-23 00:44:11
Subject: Re: [BackupPC-users] Designing a BackupPC install over a WAN - minimising Full backups
From: Peter Wright <pete AT flooble DOT net>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Fri, 23 Jan 2009 14:42:28 +0900
On 21/01 14:44:36, Holger Parplies wrote:
> Tino Schwarze wrote on 2009-01-21 10:48:50 +0100 [Re: [BackupPC-users] 
> Designing a BackupPC install over a WAN -?minimising Full backups]:
> > On Wed, Jan 21, 2009 at 02:08:31PM +0900, Peter Wright wrote:
> > > Adam Goryachev, Wed, Jan 21 2009, 14:01:19 +1100:
> > > > David Crisp, Wed, Jan 21 2009, 12:31:35 +1100:
> > > > > Idealy I would love the system to be able to sit in place
> > > > > and make regular incrementals of the data with the most
> > > > > minimal of network traffic requried to backup the data.
> 
> this is a frequently asked question. Minimal bandwidth usage with
> rsync is achieved by alternating full and incremental backups, or
> using full backups only.

I may be misunderstanding what you mean here.

By "full backup" do you mean "transfer the entire share (albeit
compressed) over the appropriate medium"? (generally network).

Or is it a different concept, more to do with how backuppc manages
data on the server?

> Using multi-level incremental backups can achieve the same bandwidth
> savings, but at increasing costs for building the backup view
> (meaning you can probably take it to level 3 or 4 but not level 365).
> 
> You will need to decide for yourself to which degree bandwidth costs
> outweigh other costs. If you have little change in your backup data,
> you will prefer doing more incrementals in between full backups,

What I'm thinking of is the (fairly common?) scenario of a fileserver
containing lots of Micosoft Office and/or PDF documents. Most of the
documents don't change at all once added. Some of them are modified
for a while (maybe a few weeks or at most a few months) and then left
unchanged. New stuff is added fairly steadily.

Given that usage model, where files more than a year old are almost
certainly *never* changed (and there's easily more than a decade's
worth of files stored), do you see how it'd be hard to justify the
bandwidth cost of doing a full backup (especially every *week* :-))?


Assuming my first understanding (my second paragraph above) is
correct, do you think it could be plausible to use the
incremental-backup stored data to build something that *looks* to
backuppc like a full backup?

> > > > An incremental backup will transfer changed data from the
> > > > previous full or incremental of a lower level.
> > > 
> > > Can backuppc be configured to transfer changed data from the
> > > previous incremental of the *same* level, or would this require
> > > a patch?
> 
> This would simply be incorrect.

Okay...

> If you backup relative to a level 1 backup, you get a level 2
> backup, whatever you call it. BackupPC presents an identical view to
> you through web interface and restore functionality, meaning you
> don't have to restore (level 0 + level 1 + level 2) - the level 2
> backup *appears like* a full backup.

Aha.

So we *could* take the backup data and construct something that
backuppc would view as a genuine full backup?

I suspect this is probably quite possible, but I'm not sure as to
whether I'm missing something that'd make it completely inappropriate
and/or impractical :). 

Any advice on this point would be much appreciated.

> Patching BackupPC to call (and store) your level 2 backup as a level
> 1 gains you something in terms of costs for constructing the view,
> but not in terms of exactness (which is very good for rsync in any
> case, though). There does not seem to be much point in doing that.

Wouldn't it mean that (given my example usage model described above)
keeping backup bandwidth costs more under control?

<thinking>

Would it be fair to say that the most common usage model for backuppc
is over a local network? Do many people tend to use backuppc over an
internet connection?

[ snip ]
> An incremental backup, in contrast, does not check file contents
> that appear unchanged, so errors are possible (if unlikely) - with
> increasing level you potentially get more errors.

Okay, fair point.

> Regards,
> Holger

Pete.
-- 
"There are two types of programming languages; the ones that people
bitch about and the ones that no one uses."  -- Bjarne Stroustrup

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/