BackupPC-users

Re: [BackupPC-users] Unnecessary reads with rsync?

2016-08-04 19:13:30
Subject: Re: [BackupPC-users] Unnecessary reads with rsync?
From: Adam Goryachev <mailinglists AT websitemanagers.com DOT au>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Fri, 5 Aug 2016 09:12:53 +1000
On 05/08/16 06:31, martin f krafft wrote:
> also sprach martin f krafft <madduck AT madduck DOT net> [2016-08-04 22:16 
> +0200]:
>> Right now, I am staring at the lsof output of the rsync process on
>> a backup client, spawned by BackupPC. It's processing a 3.5G file
>> that has not been touched in 5 years and has been backed up numerous
>> times. According to strace, the entire file is being read, and it's
>> taking a toll:
> I also can't help but notice that the pool file is open on the
> server, and that the corresponding dump process does continuious
> reading (according to strace) on a socket, presumably linked to the
> SSH process connected with the client.
>
> Maybe reliance on file metadata isn't good enough for a backup.
> After all, a backup should care about file content, not metadata.
>
> But instead of (what seems to be) chunk-wise checksum transmission,
> why don't we (also) store the whole-file checksum on the server (can
> be computed in the same pass) and at least give people the option to
> risk reading every file once to compute this checksum, if it means
> being able to skip files without further ado or large data
> transfers?
A couple of possibilities:
a) you haven't enabled checksum caching (--checksum-seed=32761)
b) you haven't completed at least 2 full backups including this file
c) you haven't configured backuppc the way you want it 
(RsyncCsumCacheVerifyProb)
d) something else, provide more information and we might be able to 
comment further

PS, backuppc v4 does store full file checksums, but you probably still 
want to verify the block checksums of the file from time to time on the 
slim chance that the full file checksum matches but the content is 
different.

Regards,
Adam


-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

------------------------------------------------------------------------------
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>