Holger Parplies wrote at about 17:59:43 +0100 on Sunday, December 2, 2012:
> Hi,
>
> Matthias Meyer wrote on 2012-12-02 01:57:07 +0100 [[BackupPC-users]
> BackupPC_verifyPool mismatchs found - what to do?]:
> > I've tried the BackupPC_verifyPool.pl from Holger Parplies.
> > Unfortunately some MD5 errors found and print out. e.g.:
> > [28878] 083212b41c2482783128c9212f1f8a26 ( 78) !=
> > 70f6bdb839eed7efdfe8f8b01f4dcbc7
> >
> > Did I have a problem?
>
> if there are no other indications, not necessarily. While this is not
> *supposed* to happen, there used to be a bug in BackupPC that caused
> top-level
> attrib files for backups with more than one share to be linked into the pool
> with an incorrect pool file name. The only adverse effect was that the (small
> amount of) space for the attrib file was not shared between backups. I'm not
> sure when this bug was fixed. Perhaps Jeffrey can provide you with more
> detail
> - he's the one that debugged the problem. The indicated file size (78 bytes)
> seems plausible for a top-level attrib file, so this may well be what you are
> seeing.
Here is a link to the thread explaining the original problem and
outlining several possible solutions.
http://adsm.org/lists/html/BackupPC-users/2009-12/msg00259.html
I think that Craig ultimately corrected it in the current release but
I am not sure.
Note that I have also diagnosed problems when switching between x86
and ARM architectures due to bugs in the checkusmming algorithms
though I would guess this is unlikely to be your issue. In addition to
being an unlikely use case, it also happens to make all the file
checksums wrong.
>
> > What should/can I do?
>
> Investigate whether this is the case. If not, look at the files in question.
> See below.
>
> > How to find out which file it is?
>
> The above file is cpool/0/8/3/083212b41c2482783128c9212f1f8a26 (or pool/...
> if
> you used the -u switch). Which file(s) in the pc/ tree this links to is more
> difficult to determine. First of all, for the problem I mentioned above, my
> understanding is that the link count of the pool file should be 2 (one pool
> link, one attrib file link; after that the pool file will never be re-used,
> because it will never match, as it has a name not matching its contents). If
> the link count is *not* 2, this seems to be an indication that the contents
> changed on disk when they in no case should have, which is Not Good(tm).
>
> >From the pool file ('ls -i cpool/0/8/3/083212b41c2482783128c9212f1f8a26')
> >you
> can determine the inode and search for that in the pc/ tree. Assuming this is
> a top-level attrib file, you can speed things up greatly by not traversing
> the
> backups. Depending on the number of hosts and backups, you might get away
> with
> something as simple (though not necessarily as fast as you might expect) as
> 'ls -i1 pc/*/*/attrib | sort > /tmp/top-level-attrib-inums'. Search the
> output
> file for the inode numbers you determined.
>
> If it's not top-level attrib files, you're looking at something like
> 'find pc/ -inum ... -ls', which will take *long*. You'll probably want to at
> least look for all inodes in one traversal, which is probably easier to code
> in Perl than type into one find invocation ;-).
>
> In any case, you can look at the contents using just the pool file name and
> BackupPC_zcat. That might give you enough information to be able to locate
> the
> file.
> For attrib files, BackupPC_zcat produces output that is not very human
> readable, though it *does* contain the file names (meaning share names, in
> the
> case of a top-level attrib file), so it might be good enough. I'm not sure
> whether BackupPC_attribPrint will work with the pool file name, but you could
> try that as well.
>
> Hope that helps.
>
> Regards,
> Holger
Thanks Holger for giving the above pointers.
While I am not familiar with Holger's routine, I have written a
similar routine that verifies and also *fixes* broken md5sum file
names. It does so by re-naming +/- moving the pool file as required.
Note that the program is more complicated than one might expect since
it needs to deal with "chains" of pool files with the same partial
md5sum but different suffixes, including knowing where/when to add into
the chain, including linking rather than copying if the target already
exists in the chain or alternatively adding a new suffix to the chain if the
content is distinct or
if MAXLINKS exceeded. Also, if the moved file is part of a chain, then
the original chain needs to be renumbered to fill the hole following
the renaming.
Here is the code in case you are interested...
Note: it requires my jlib library of routines which should be
available on the wikki
########################################################################
#!/usr/bin/perl
#============================================================= -*-perl-*-
#
# BackupPC_fixPoolMdsums: Rename/move pool files if mdsum path name invalid
#
# DESCRIPTION
# See 'usage' for more detailed description of what it does
#
# AUTHOR
# Jeff Kosowsky
#
# COPYRIGHT
# Copyright (C) 2011 Jeff Kosowsky
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
#
#========================================================================
#
# Version 0.2, released January 2011
#
#========================================================================
use strict;
use warnings;
use lib "/usr/share/BackupPC/lib";
use BackupPC::Lib;
use BackupPC::jLib 0.4.0; # Requires version >= 0.4.0
use File::Glob ':glob';
use Getopt::Long qw(:config no_ignore_case bundling);
#Variables
my $bpc = BackupPC::Lib->new or die("BackupPC::Lib->new failed\n");
my $md5 = Digest::MD5->new;
my $MAXLINKS = $bpc->{Conf}{HardLinkMax};
#Option variables:
my $Force;
my $warndups;
my $outfile;
my $TopDir = my $TopDir_def = $bpc->{TopDir};
my $verbose=0;
#$dryrun=0; #Global variable defined in jLib.pm (do not use 'my')
$dryrun=1; #Global variable defined in jLib.pm (do not use 'my')
usage() unless(
GetOptions(
"dryrun|d!" => \$dryrun,
"Force" => \$Force, #Override stuff...
"outfile|o=s" => \$outfile,
"topdir|t=s" => \$TopDir, #Location of TopDir
"verbose|v+" => \$verbose, #Verbosity (repeats allowed)
"warndups|w" => \$warndups, #Warn if dup created in new
chain
)
&& defined $outfile
);
#############################################################################
if($TopDir ne $TopDir_def) {
#NOTE: if we are not using the TopDir in the config file, then we
# need to manually override the settings of BackupPC::Lib->new
# which *doesn't* allow you to set TopDir (even though it seems so
# from the function definition, it gets overwritten later when the
# config file is read)
$TopDir =~ s|//*|/|g; #Remove any lurking double slashes
$TopDir =~ s|/*$||g; #Remove trailing slash
$bpc->{TopDir} = $TopDir;
$bpc->{Conf}{TopDir} = $TopDir;
$bpc->{storage}->setPaths({TopDir => $TopDir});
$bpc->{PoolDir} = "$bpc->{TopDir}/pool";
$bpc->{CPoolDir} = "$bpc->{TopDir}/cpool";
}
%Conf = $bpc->Conf(); #Global variable defined in jLib.pm (do not use 'my')
#############################################################################
my $compress = $Conf{CompressLevel};
my $pool = $compress > 0 ? "cpool" : "pool";
my $compare = $compress > 0 ? \&zcompare2 : \&jcompare;
my $file2md5 = $compress > 0 ? \&zFile2MD5 : \&File2MD5;
my ($OUT);
die "ERROR: '$outfile' already exists!\n" if -e $outfile;
open($OUT, '>', "$outfile") or
die "ERROR: Can't open '$outfile' for writing!($!)\n";
chdir $TopDir;
if(!$Force &&
(my @partialbackups = glob("pc/*/NewFileList{,.[0-9]*}"))) {
die("ERROR: Pool conflicts will occur if NewFileList present (--Force
overrides):\n " .
join('\n ', @partialbackups) . "\n");
}
system("$bpc->{InstallDir}/bin/BackupPC_serverMesg status jobs >/dev/null
2>&1");
unless(($? >>8) == 1) {
die "Dangerous to run when BackupPC is running!!!\n"
if $TopDir eq $TopDir_def;
warn "WARNING: May be dangerous to run when BackupPC is running!!!\n";
#Warn but don't die if *appear* to be in different TopDir
}
my $totalfiles = 0;
my $md5errors = 0;
my $fixed = 0;
my $badpoolentry = 0;
my $chaindups = 0;
my $norename = 0;
my $renumbererr = 0;
scan_pool($pool);
#Note: Total = total pool files scanned
# NotFixed = NoRename (i.e. rename fails - shouldn't happen...)
# ChainDups (only calcualted if --warndups set)
# RenumberErrs = Errors renumbering old chain after move (shouldn't
happen...)
printf("Total=%d Md5PathErrors=%d [Fixed=%d, NotFixed=%d]%s\n",
$totalfiles, $md5errors, $fixed, ($md5errors-$fixed),
($dryrun ? " DRY-RUN" : ""));
printf("BadPoolEntries=%d RenumberErrs=%d\n", $badpoolentry, $renumbererr);
printf("ChainDups=%d\n", $chaindups) if $warndups;
print "\n";
exit;
#######################################################################
#Run through the pool looking for misnamed md5sum paths
sub scan_pool
{
my ($fpool) = @_;
my ($dh, @fstat);
return unless glob("$fpool/[0-9a-f]"); #No entries in pool
my @hexlist = ('0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a',
'b', 'c', 'd', 'e', 'f');
my ($idir, $jdir, $kdir);
foreach my $i (@hexlist) {
print STDERR "\n**$fpool/$i: " if $verbose >=2;
$idir = $fpool . "/" . $i . "/";
foreach my $j (@hexlist) {
print STDERR "$j " if $verbose >=2;
$jdir = $idir . $j . "/";
foreach my $k (@hexlist) {
$kdir = $jdir . $k . "/";
unless(opendir($dh, $kdir)) {
warn "Can't open pool directory:
$kdir\n" if $verbose>=4;
next;
}
#Sort directory entries so that chains are
ordered lowest to
#highest - This preserves sequential order between source and
#target chains PLUS ensures that we fill holes corretly and
#most efficiently
my @entries = sort {poolname2number($a) cmp
poolname2number($b)}
(readdir ($dh));
close($dh);
warn "POOLDIR: $kdir (" . ($#entries-1) ."
files)\n"
if $verbose >=3;
my $chainholes = 0;
my $chainstart;
my $lastdigest='';
foreach (@entries) {
next if /^\.\.?$/; # skip dot files
(. and ..)
$totalfiles++;
my $origfile = ${kdir} . $_;
unless(m|^([0-9a-f]+)(_[0-9]+)?|) {
warn "ERROR: '$origfile' is not
a valid pool entry\n";
$badpoolentry++;
next;
}
my $origdigest = $1;
my $newdigest = $file2md5->($bpc, $md5,
$origfile, -1, $compress);
if($newdigest eq "-1") {
$badpoolentry++;
warn "ERROR: Can't calculate
md5sum name for: $origfile\n";
next;
}
if($newdigest ne $origdigest) { #MD5sum
Path is WRONG
$md5errors++;
if($origdigest ne $lastdigest)
{ #New chain
#So go back and
renumber last chain to remove holes
if($chainholes > 0) {
renumber_pool_chain($chainstart, $chainholes)>0
or
$renumbererr++;
}
$lastdigest=$origdigest; #Reset to new chain base
$chainholes = 0;
$chainstart =
$origfile; #lowest element of chain
#since we are sorting
directory in chain order
}
if(rename_entry($origfile,
$newdigest)>0) {$chainholes++}
}
}
#Check in case any holes unfixed at end of
directory scan
if($chainholes > 0) {
renumber_pool_chain($chainstart,
$chainholes) >0
or $renumbererr++;
}
}
}
}
print STDERR "\n" if $verbose >=2;
}
#Rename/move pool chain entry $source to first open position
#in $digest chain if permitted. Renumber source chain as
#needed after the move
sub rename_entry
{
my ($source, $digest) = @_;
my $dups = '';
my @dups = ();
my $poolpath = $bpc->MD52Path($digest,$compress);
my $poolbase_ = $poolpath . "_";
for(my $i=0; -f $poolpath; $i++) { # Iterate through pool chain with
same md5sum to find
# first free entry
if($warndups && (stat(_))[3] < $MAXLINKS &&
! $compare->($source,$poolpath)) { #Matches existing pool
entry
push(@dups,$i);
}
$poolpath = $poolbase_ . $i;
}
$poolpath =~ m|^$TopDir/?((.*)/.*)|;
my $target = $1; #Path relative to TopDir
my $dir = $2; #Relative to TopDir
# print "$source $target [$md5errors/$totalfiles]$dups\n";
if(@dups) {
$dups = " CHAINDUPS(" . join(',', @dups) . ")";
warn "WARN: $dups: $source->$target\n" if $verbose >=1;
$chaindups++;
}
if((!-d $dir && !jmkpath($dir, 0, 0750)) || #Directory creation error
!jrename($source,$poolpath)) { #Rename error
warn "ERROR: Can't rename: $source->$target\n" if $verbose >=1;
print $OUT "$source $target NO_RENAME$dups\n";
$norename++;
return -1;
}
#Fixed without errors
print $OUT "$source $target FIXED$dups\n";
$fixed++;
return 1;
}
sub usage
{
print STDERR <<EOF;
usage: $0 [options] --outfile|-o <outfile>
Options:
--dryrun|-d Dry-run
Negate with: --nodryrun
--Force Force - MAY BE DANGEROUS!
--topdir|-t Location of TopDir. [Default = $TopDir_def]
Note you may want to change from default for example
if you are working on a shadow copy.
--verbose|-v Verbose (repeat for more verbosity)
--warndups|-w Warn if renamed pool entry is a pool duplicate
DESCRIPTION:
Find and fix md5sum pool name errors in pool and cpool
DETAILS:
Recurses through pool and cpool trees to test if the md5sum name of each
pool file is correct relative to the file data. If not, the program attempts
to rename (i.e. move) it to its proper md5sum name.
If there already are pool files with the new name, then move it to
the end of the target chain. After removing, renumber the source
chain (if needed) to fill in holes left by the move. Note that the relative
ordering of each chain is preserved.
If the contents of the file match the contents of any of the files in the
target chain, note the duplicate suffix numbers.
If the --warndups|-w flag is set then check to see if the renamed pool
entry duplicates an existing pool entry (with <MAX LINKS). This may be
slow if you have a lot of chains since to check for duplicates you need
to compare files
Note: it is not generally an error to have two pool entries in the same
chain with the same data (in fact, it occurs intentionally when you exceed
MAX LINKS), it just may waste some space. My routine BackupPC_fixLinks.pl
can correct just such duplicates later if that is an issue.
In any case, if all your misnumbering was consistent you won\'t have this
situation anyway.
<outfile> records all the changes made plus appends a status code:
FIXED = pool file moved/renamed and original chain renumbered if needed.
NO_RENAME = Signals error in renaming/moving the pool file. The mdsum name
was thus not corrected.
DUPS(n1,n2,n3) = Signals duplicates in the target chain and lists the
suffixes (-1 = no suffix)
The following errors are tabulated but shouldn\'t occur:
BadPoolEntries = Files in pool with name not of form [0-9a-f]+(_[0-9]+)?
or that can\'t be read to compute the MD5 path digest
RenumberErrors = Failures to renumber original chain to fill hole
after renaming
EOF
exit(1)
}
------------------------------------------------------------------------------
Keep yourself connected to Go Parallel:
DESIGN Expert tips on starting your parallel project right.
http://goparallel.sourceforge.net/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
