Re: [BackupPC-users] md4 doesn't match even though VSS is used

Matthias Meyer wrote at about 00:21:25 +0200 on Wednesday, September 14, 2011:
 > Jeffrey J. Kosowsky wrote:
 > 
 > > Matthias Meyer wrote at about 15:29:09 +0200 on Sunday, September 11,
 > > 2011:
 > >  > Dear all,
 > >  > 
 > >  > I've a problem by backing up a large file (9GB) because the internet
 > >  > connection of the client interrupts every 24 hours.
 > >  > 
 > >  > BackupPC (V3.1.0) can rsync this file once with status:
 > >  > "md4 doesn't match: will retry in phase 1; file removed"
 > >  > With "lsof /var/lib/backuppc" I can see this "phase 1" transfer some
 > >  > minutes later.
 > >  > But the internet connection will interrupt shortly before this second
 > >  > transfer were finished :-(
 > >  > 
 > >  > I am sure that the source file (Windows client) is on a volume shadow
 > >  > copy and rsync is using this source because:
 > >  > - /etc/rsyncd.conf contains only one "path = /cygdrive/Z/"
 > >  > - ls /cygdrive/ shows only the drives C and D
 > >  > - ls /cygdrive/Z lists the same files as ls /cygdrive/C
 > >  > 
 > >  > So it should not possible that the source was changed.
 > >  > 
 > >  > Did /usr/share/backuppc/lib/BackupPC/Xfer/RsyncFileIO.pm compare the
 > >  > md4 diggest from the begin of a transfer with a recalculated md4
 > >  > diggest at the end of the transfer?
 > >  > 
 > >  > Somebody else have a similiar problem?
 > >  > 
 > >  > Is there any known solution to solving my problem?
 > >  > 
 > >  > What happens if I patch the RsyncFileIO.pm so that it will ignore the
 > >  > "md4 doesn't match"?
 > >  > 
 > >  > I know I should try it instead asking for it. But I'm not sure what the
 > >  > meaning of md4 is and hopefully someone can give me a hint.
 > >  > 
 > > 
 > > I would not ignore the md4 mismatch. md4sums are used on block sums
 > > and file sums both to ensure the transfer was errorless and also as
 > > part of rsync's delta matching algorithm that allows it to only
 > > transfer over changed blocks.
 > > 
 > > I'm not sure what is the cause of your problem. But I would first try
 > > naked rsync (without BackupPC) and with protocol manually set to
 > > protocol 28 so that md4 sums are used rather than md5sums. See what
 > > happens when you try to transfer the file...
 > > 
 > > 
 > I've got the file with native rsync --protocol=28 from the client without an 
 > error.
 > In the next step I backup this file with BackupPC onto one machine within my 
 > LAN and "cp -al" the directory into the last backup set from the original 
 > client. Thats work. The original client run the next backup without an 
 > error.
 > 
 > Maybee my previous error was that I copied the file without the attrib-file 
 > into the backup set of the original client.
 > 

Well, officially one should never manually add/change/delete
individual files in past backups.

As many people know, I wrote a program to all for manual deletions
(BackupPC_deleteFile.pl) but that involves all the trickiness of
changing attrib files and dealing with incremental inheritance. Adding
or moving files has the potential for similar issues.

Just manually copying/moving/deleting files in the pc-tree is a
*really* bad idea. 

That being said, I'm not exactly sure why such a change would cause
the error you report except that if you have checksum caching turned
on, the filesize mentioned in the attrib file is used to determine
where the file contents end and the md4 checksums are stored. I know
in my routines if the filesize is missing, I manually calculate the
size of the file but I'm not sure if that happens also in all instances
in BackupPC. There is also the possibility that the attrib file
contains a record for a similarly named file but with a different size
-- this could very well cause md4sum errors since the location of the
md4sums and their values themselves may be different...

------------------------------------------------------------------------------
BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
Learn about the latest advances in developing for the 
BlackBerry&reg; mobile platform with sessions, labs & more.
See new tools and technologies. Register for BlackBerry&reg; DevCon today!
http://p.sf.net/sfu/rim-devcon-copy1 
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/