BackupPC-users

Re: [BackupPC-users] Bug#497888: backuppc: please make use of the rsync algorithm, particularly in resuming interrupted backups

2008-09-11 17:40:16
Subject: Re: [BackupPC-users] Bug#497888: backuppc: please make use of the rsync algorithm, particularly in resuming interrupted backups
From: Holger Parplies <wbppc AT parplies DOT de>
To: Ludovic Drolez <ldrolez AT debian DOT org>
Date: Thu, 11 Sep 2008 23:39:00 +0200
Hi,

Ludovic Drolez wrote on 2008-09-11 16:55:18 +0200 [Re: [BackupPC-users] 
Bug#497888: backuppc: please make use of the rsync algorithm, particularly in 
resuming interrupted backups]:
> On Fri, Sep 05, 2008 at 05:09:19PM +1000, Tim Connors wrote:
> > It would be very nice if the backuppc client communicated as a regular
> > rsync client back to the rsync server, and didn't wipe the tree that
> > had already been partially trasferred.  So when 400MB of a 600MB file
> > has been successfully transferred, only the delta gets transmitted the
> > next time.  Yes, of course, this behaviour should be configurable so
> 
> Yes backuppc should work this way. You may have a configuration
> problem somewhere...

actually, I believe BackupPC *does* remove the in-progress file at the time
the backup failed. This is logical in that you don't have a [partial] backup
that reflects an incorrect state of a file (i.e. if it's there it's correct) -
a random file from the user's point of view, as I think should be pointed out.
There is no easy way to "see" with which file in-progress the backup failed
when you're navigating the file tree, so you can't easily account for
incorrect file contents. As for used disk space on the BackupPC server, the
partial file would be unlikely to match a pool file, but it would be removed
again once a larger partial completes, so keeping it would not overly
"pollute" the pool.

The other thing to note is that partial backups are only saved for full
backups. When an incremental backup is aborted, no partial is saved. This
makes sense, because an incremental backup is based on exactly one reference
backup - the full backup in the simple and usual case. Technically, you could
use a merged view of the reference backup and the partial backup as a starting
point, but this would - strictly speaking - make it a level N+1 incremental
instead of the requested level N. This may be splitting hairs for rsync, and
there may be far better reasons for not using partials on incrementals that I
am missing.


Tim Connors wrote in 497888 on bugs.debian.org:
> Furthermore, if it's not running the delta algorithm over an
> interrupted backup, I'm guessing that a regular incremental backup run
> involves the backuppc server transferring whole files corresponding to
> those files that had changed compared to a previous version, and then
> running the comparison against the pool to work out which files can be
> linked.

That is not the case. Though you are misinterpreting '--inplace' (which only
means command line rsync does not make in intermediate copy of the file(s)),
BackupPC does do exactly what you want: determine a sensible reference file
and transfer deltas on both full and incremental runs. Note that the
reference for a (level 1) incremental is the preceeding full backup, so you
may be re-transferring the same delta multiple times, but for your case of a
slowly growing mbox file, this is only a problem if you have a *long* time
between full backups. Bandwidth-wise full backups may be cheaper than
incremental backups, as has been explained on the mailing list many times.
In particular, if you currently have only one very old full backup with your
mbox file only a few KB in size, a level 1 incremental will transfer almost
all of the file, while a full would transfer only the delta since the last
incremental (!).

Regards,
Holger

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>