BackupPC-users

Re: [BackupPC-users] A question about partial backups and fatal errors

2016-03-20 10:10:15
Subject: Re: [BackupPC-users] A question about partial backups and fatal errors
From: David Raison <david AT tentwentyfour DOT lu>
To: backuppc-users AT lists.sourceforge DOT net
Date: Sun, 20 Mar 2016 14:50:53 +0100
HI

I apologize for re-opening an old thread.

Unfortunately we have still not managed to solve this issue or to even
fully understand it. We have determined that indeed the Windows client
in this example is sending a RST after a random duration, but we we're
not quite certain how to get to the source of this, i.e. how to
determine why it is doing so.


The other problem that remains and that is really bugging me is that
problem with partial backups, see below:

On 05/12/15 21:11, Les Mikesell wrote:

>>>>> Backuppc doesn't exactly 'resume' where a partial stopped. 
>>>>> The previous partial is used as the basis for the next full 
>>>>> where a complete comparison is done (partials quickly skip 
>>>>> files where the timestamp and length match).  So, it is
>>>>> still going to take the time to read those files even though
>>>>> only differences will be transmitted. At the end of the next 
>>>>> failing partial, the run with the most files will be kept.
>>> 
>>> I realize that, but the comparison doesn't seem to be happening 
>>> in this case, or is it?
>>> 
>>> Whereas I see a pattern like this for most other backups:
>>> 
>>> same pool same …
>>> 
>>> This one – the failing one – is always listing "pool" or
>>> "create" maybe, but not a single time do I see a "same".
> I'm not sure if the xfer always has the same ordering - maybe it is 
> just hitting different files first.  Are you seeing files logged that
> already exist in the partial copy?


We are back to where backuppc never marks any files as "same" and is
re-transferring all files at each run. Given that the backups always
abort after a certain time due to the Windows client sending that RST
signal, we will never be seeing this backup succeed.

I incremented PartialAgeMax to a much higher value than the default in
order to allow it more attempts at finishing the full backup before it
will no longer consider the partial as a base for the next full backup:

$Conf{PartialAgeMax} = '28';

However, it seems that even with this set, any subsequent full runs
would discard any partials that got less than the number of files in the
previous partial:

> Not saving this as a partial backup since it has fewer files than the
> prior one (got 7 and 7 files versus 304)

Since none of the full attempts would manage to get anywhere near the
304 files, we're in an endless loop of rsyncing the same few files over
and over again and not making any progress at all.

If rsync ignores partials older than PartialAgeMax, shouldn't the
decision whether to keep files or not also respect the same limit?

Here's the arguments to rsync in our current setup:


Sending args: --server --sender --numeric-ids --perms --owner --group -D
--links --hard-links --times --block-size=2048 --recursive
--checksum-seed=32761 --bwlimit=24000 --timeout=1800 --ignore-times . .


Notwithstanding that we need to solve the problem with the client
sending those RSTs, does anyone have any advice on how to get full runs
to not start from scratch on each run?

Best regards,
David

-- 
TenTwentyFour S.à r.l.
W: www.tentwentyfour.lu
T: +352 20 211 1024
F: +352 20 211 1023
9, avenue des Hauts-Fourneaux
L-4362 Esch/Alzette

Attachment: signature.asc
Description: OpenPGP digital signature

------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/