Re: [Bacula-users] Verify differences: SHA1 sum doesn't match but it sho

Could be due to a transient error (transmission or wild/torn read at time of calculation).   I see this a lot with integrity checking of files here (50TiB of storage).

Only way to get around this now is to do a known-good sha1/md5 hash of data (2-3 reads of the file make sure that they all match and that the file is not corrupted) save that as a baseline and then when doing reads/compares if one fails do another re-read and see if the first one was in error and compare that with your baseline.     This is one reason why I'm switching to the new generation of sas drives that have ioecc checks on READS not just writes to help cut down on some of this.

Corruption does occur as well and is more probable with the higher the capacity of the drive.     Ideally you would have a drive that would do ioecc on reads, plus using T10 PI extensions (DIX/DIF) from drive to controller up to your file system layer.    It won't always prevent it by itself but would allow if you have a raid setup to do some self-healing when a drive reports a non transient (i.e. corrupted sector of data).

However the T10 PI extensions are only on sas/fc drives (520/528 byte blocks) and so far as I can tell only the new LSI hba's support a small subset of this (no hardware raid controllers I can find) and have not seen any support up to the OS/filesystem level.    SATA is not included at all as the T13 group opted not to include it in the spec.

On 2010-08-28 04:25, Tobias Brink wrote:

Hello!

I run Bacula 3.0.2 (dir and sd) from Debian backports and bacula-fd
5.0.2 (Debian testing) on the client and I have a problem with a disk to
catalog verify job.

I make differential backups every week which get consolidated into a
full backup with the VirtualFull feature once a month.  Now I ran a
differential backup as usual but followed it immediately by a
DiskToCatalog verify job.  The only processes running during that time
were absolutely necessary things like udev and mdadm and I was logged in
but did nothing apart from checking bconsole.

Apart from the expected differences (bash history etc.) the job reported
"SHA1 digest differs" for a number of different files.  The files ranged
from /usr/bin/ contents to older MP3s in my home directory.  The files
also range in their age, some of the files where backed up in the last
differential run while others are much older.

So I checked the md5sum of the /usr/bin/ files against the Debian
provided MD5 sums and they matched.  I then restored some of the
offending files from backup, but diff showed no differences.

So I'm at a loss here.  Is my catalog corrupted or is this a Bacula bug?
Is there a fix or workaround?

Thanks in advance for any tips,
Tobias

PS: Is there a way to convert from Bacula's sha1 (which look like
"QGHMaXrQMew6S5MaBRVcgJ5S4LQ") to the output of the of the sha1sum CLI
utility (which looks like "c274a48e3a65f6b1e9e7a0103c4a7b522c8f5cd7")?

------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users 
worldwide. Take advantage of special opportunities to increase revenue and 
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users 
worldwide. Take advantage of special opportunities to increase revenue and 
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Verify differences: SHA1 sum doesn't match but it should