Re: [Bacula-users] Verify differences: SHA1 sum doesn't match but it should
2010-08-28 07:47:59
Could be due to a transient error
(transmission or wild/torn read at time of calculation). I see
this a lot with integrity checking of files here (50TiB of
storage).
Only way to get around this now is to do a known-good sha1/md5
hash of data (2-3 reads of the file make sure that they all match
and that the file is not corrupted) save that as a baseline and
then when doing reads/compares if one fails do another re-read and
see if the first one was in error and compare that with your
baseline. This is one reason why I'm switching to the new
generation of sas drives that have ioecc checks on READS not just
writes to help cut down on some of this.
Corruption does occur as well and is more probable with the higher
the capacity of the drive. Ideally you would have a drive that
would do ioecc on reads, plus using T10 PI extensions (DIX/DIF)
from drive to controller up to your file system layer. It won't
always prevent it by itself but would allow if you have a raid
setup to do some self-healing when a drive reports a non transient
(i.e. corrupted sector of data).
However the T10 PI extensions are only on sas/fc drives (520/528
byte blocks) and so far as I can tell only the new LSI hba's
support a small subset of this (no hardware raid controllers I can
find) and have not seen any support up to the OS/filesystem
level. SATA is not included at all as the T13 group opted not
to include it in the spec.
On 2010-08-28 04:25, Tobias Brink wrote:
Hello!
I run Bacula 3.0.2 (dir and sd) from Debian backports and bacula-fd
5.0.2 (Debian testing) on the client and I have a problem with a disk to
catalog verify job.
I make differential backups every week which get consolidated into a
full backup with the VirtualFull feature once a month. Now I ran a
differential backup as usual but followed it immediately by a
DiskToCatalog verify job. The only processes running during that time
were absolutely necessary things like udev and mdadm and I was logged in
but did nothing apart from checking bconsole.
Apart from the expected differences (bash history etc.) the job reported
"SHA1 digest differs" for a number of different files. The files ranged
from /usr/bin/ contents to older MP3s in my home directory. The files
also range in their age, some of the files where backed up in the last
differential run while others are much older.
So I checked the md5sum of the /usr/bin/ files against the Debian
provided MD5 sums and they matched. I then restored some of the
offending files from backup, but diff showed no differences.
So I'm at a loss here. Is my catalog corrupted or is this a Bacula bug?
Is there a fix or workaround?
Thanks in advance for any tips,
Tobias
PS: Is there a way to convert from Bacula's sha1 (which look like
"QGHMaXrQMew6S5MaBRVcgJ5S4LQ") to the output of the of the sha1sum CLI
utility (which looks like "c274a48e3a65f6b1e9e7a0103c4a7b522c8f5cd7")?
------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users
worldwide. Take advantage of special opportunities to increase revenue and
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users
worldwide. Take advantage of special opportunities to increase revenue and
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d _______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
|
|