BackupPC-users

Re: [BackupPC-users] How to reuse already saved data until an interrupted incr backup

2009-07-06 12:42:05
Subject: Re: [BackupPC-users] How to reuse already saved data until an interrupted incr backup
From: Matthias Meyer <matthias.meyer AT gmx DOT li>
To: backuppc-users AT lists.sourceforge DOT net
Date: Mon, 06 Jul 2009 18:35:09 +0200
Holger Parplies wrote:

> Hi,
> 
> Matthias Meyer wrote on 2009-07-05 21:14:57 +0200 [Re: [BackupPC-users]
> How to reuse already saved data until an interrupted incr backup]:
>> > I use BackupPC 3.1 and rsyncd.
>> > If an incr Backup aborts, e.g. because of lost network connections, all
>> > of the data, which was copied into the /new directory, will be deleted.
>> > The next time an incr backup runs, all the work (compare and transfer)
>> > must be done again.
>> > 
>> > What is the reason for this behavior?
> 
> that is a good question, but it is difficult to answer without a close
> look at the code. Without looking at the code, I'd *guess* it might be the
> same reason as for not using the previous backup as reference for any
> backup. Strictly, completing a partial level N backup would give you a
> level N+1 backup, because the reference is (partly) a (partial) level N
> backup. Yes, I know, nobody except me takes that seriously.
> 
>> > Is there a posibility to save the already saved data and reuse it for
>> > the next backup in a similiar way as in interupted full backups?
> 
> No.
It's a pity
> 
>> [...]
>> In the next step I will try to move the /new directory to a /<number>
>> directory, update the /backups file and run BackupPC_link during the
>> DumpPostUserCmd.
>> 
>> Is there any reason to not do that?
> 
> This is not a good question. The answer is, obviously, "yes" - otherwise
> BackupPC would already be doing that by default (it's not as though that
> would be difficult for BackupPC - it would just require *not* handling the
> error). The question "why shouldn't I do that" is not much better. Isn't
> the fact that you shouldn't good enough? But if you insist on an answer
> ...
> 
> 1. You will have something that looks like a successful backup but isn't.
> You
>    just might be able to remember that now, but what happens in two weeks
>    time, when you need to restore files?
>    I don't exactly know what such a backup would look like, but there are
>    basically two possibilities:
>    a) The backup view contains all files processed (i.e. found to be
>    "same" or
>       transferred). All files beyond the point where it failed are
>       missing.
>    b) All files beyond the point where it failed are identical to their
>    state
>       in the reference backup.
>    I presume it's case a, but if it's case b, things get even more
>    awkward. You see what seems to be a complete backup, but some files are
>    current and some might be older versions than in preceeding
>    incrementals. Case a would at least show you an incomplete backup when
>    you are browsing it (presuming enough is missing for you to notice).

I presume it is case b) because BackupPC will fill an incremental backup
with the files from previous backups. So I will see an backup, seems to be
complete but isn't.

> 
> 2. What is run next? That depends on your setup. If it's an incremental of
> the
>    same (or lower) level, all changes are re-transferred, so you gain
>    nothing. You simply end up with a bogus backup in your history.
>    On the other hand, if it's a full or an incremental of *higher* level
>    (you have a level N backup in your 'backups' file, so BackupPC would do
>    a level N+1 if that is what you configured), the backup will need to
>    re-transfer *all files you missed on the partial* (at least in case a
>    above), meaning you probably lose (much) more than you gain, aside from
>    also having a bogus backup in your history.
> 
I have a level N backup configured. So the next incremental backup will
backup all changed files since last (the incomplete incremantal) backup.
I understand. With my strategy I will not backup those files which are
changed before but not included in the last, incomplete incremental backup.
> 
> Beside that, as Les said, it's either infrequent enough not to be
> important, or you should be fixing something rather than kludging around a
> real problem.
> 
> Regards,
> Holger
> 
> P.S.: Doing something because you don't understand why not to suggests you
>       have more relevant questions to ask than this one. Then again, maybe
>       you *are* asking part of that question. Maybe your current problem
>       is one aspect of an answer you understand? More clearly: maybe you
>       should not be using incremental levels because you'll end up with
>       failing incremental backups which will not be completed. But that's
>       just a guess from the parts of the puzzle you decided to give us.
>       It's certainly *not* a general recommendation against incremental
>       levels. There are simply cases where you should be doing frequent
>       full backups.
> 

My basic problem are notebooks which are often only a short time online.
If such a notebock have one new big file (around 500MB or above) it often
use the short online time to transfer this big file.
Because of disconnection the incremental backup will not finish and BackupPC
delete this file on the server side. Therefore the same play will take
place at the next day.

Thanks for spending your time for your answer. It is very helpful for me.
My "new" strategy:
If an incremental abort I will copy the last backup (called #X) into a new
backup (called #Y) and merge the partial incremental into it.
In the __TOPDIR__/pc/<host>/backups file I duplicate the last line and
increment the backup number (to #Y). I will also try to change the
text "full" or "incr" to "partialIncr" but I would believe that BackupPC
will be surprised about this string.
The advantage of the above: The next backup is looking for new files against
the timestamp from backup #X and against the files from backup #Y.
The disadvantage: It is possible that backup #Y contains files which are
deleted after backup #X on client side.

If the next backup (called #Z):
- is an incremental backup  and abort too, I will only merge the transferred
files into backup #Y. If it ends successful I have to merge the backup #Y
into backup #Z and delete backup #Y.
- is a full backup I would end up with an inconsistent incremental #Y.
Therefore I will delete backup #Y after the successful full backup #Z.

My goal, save as much transfered files as possible, should be reached.
The disadvantage to have "deleted" files within an backup would be bearable
for me.

What do you think about that?

br
Matthias
-- 
Don't Panic


------------------------------------------------------------------------------
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/