Hi,

I wrote on 2008-10-07 01:11:39 +0200 [Re: [BackupPC-users] suffering from the 
backups have stopped syndrome/rsyncd connection stalled]:
> [...]
> A while back someone traced something like this down to a corrupt attrib
> file. I've been trying to find the time to write a script that does some
> sanity checking on attrib files but haven't gotten around to doing it so
> far.

ok, version 0.0001, quick hack mode, no warranties, seems to basically appear
to sometimes work for me [BackupPC 3.0.0] and so on :-). Uncommented, batteries
not included. Run as backuppc user, uses BackupPC::Lib, adjust "use lib" path
where not Debian/Ubuntu. Does *not* use BackupPC::Attrib for the sake of having
a second opinion on the validity of attrib files - I don't know yet if that's
good or bad :). Future versions could include a command line switch to select
between BackupPC::Attrib and quick'n'dirty unpack. Requires Perl 5.006
according to 'perldoc perlpacktut' and checks this.

Command line switches:

-d <directory> ... Directory to start from. Default: /var/lib/backuppc/pc.
                   Traverses the substructure and checks all attrib files.
                   Reports "suspicious" findings, see below.
-v ............... Verbose mode, show names and lengths of processed attrib
                   files. Not recommended except for fun (too much output -
                   you're bound to miss any "suspicious" files that might be
                   reported).
-p ............... Progress indication mode - output the name of each attrib
                   file parsed. Uses \r - if your terminal is wide enough,
                   each name will overwrite the preceeding, so you see what's
                   going on but still see real output (uses \n).
-u <uidlist> ..... UIDs to ignore. If you have files in your backups belonging
                   to uids that don't map to user names, list the uids here,
                   separated by commas.
-X <permlist> .... List of bogus permissions to ignore. My local favorites:
                   0720, 04770 and 0464. Oh well ...
                   Again, separate with commas. eval()led (well, each element
                   separately, so you can't do too fancy things), which might
                   ultimately be a security problem in a 1.x version :), but
                   allows octal modes (or hex, if someone uses that).
-D ............... Debug output. List users and groups (the ids, that is) that
                   are mappable to names (or whitelisted).
-m <max> ......... Maximum attrib file size to consider valid. Default: 100MB,
                   I hope you don't really have *that* large directories ;-).
                   The default should probably be smaller, but I wanted to
                   avoid OOM situations (I read an entire attrib file into
                   memory before parsing it) rather than catch invalid attrib
                   files. I hope those will be detected by their entries ;-).
-M <max> ......... Maximum "file size / 4GB" value to consider valid. Defaults
                   to 0. If you have files in your backups *larger than 4GB*,
                   you will need to use "-M 1" or "-M 2" or something. We're
                   probably looking for invalid values in this field so don't
                   use too high values here.


What is reported as a suspicious file?
- Files of a size exceeding 4GB (or 4GB * (N + 1) if you use "-M N")
- Files that do not belong to a valid uid (or one whitelisted with -u)
  or gid (hmm, no option for that, sorry, but easy enough to add :)
- Files with an unreasonable permission field (I tried to enumerate the
  permission values that seem to make sense, but I'm bound to have missed
  some or included some you find unreasonable; they also need reordering :)
- Files with an invalid type (not one of the values listed in
  BackupPC::Attrib)

Feel free to add or suggest other things worth checking (file name length?).


Basic usage suggestion:
If you have problems with your host 'client', whose most recent [successful]
backup number is 123 (and your TopDir is /var/lib/backuppc), try something like

BackupPC_attribCheck -d /var/lib/backuppc/pc/client/123

and add a -u and/or -X option if you get output that is indicative of broken
users rather than broken attrib files ;-).

Output format:

[type] mode uid gid size/4GB size%4GB 0xTimestamp Filename

so it's the second and third field that will tell you what -u and -X options
to add and the 5th that indicates an extremely large file size.

Hmm, I'll probably need to include the *path* to the file in version 0.0002 :).

Please tell me if you actually find something suspicious. The part of my pool
that I checked seems to be fine, but then it should be, shouldn't it? :)

Good luck.

Regards,
Holger

BackupPC_attribCheck
Description: Text document

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/