On Fri, Aug 20, 2004 at 07:08:47AM +0800, Scott Frisbie wrote:
> Ran the revised shell and am attaching it to the mail along with the other
> files.
Thanks. Unfortunately, all it does is convince me that some
*really weird* stuff is going on :-(
Running the headers through "od -c" was to prove to us that what
you'd shown in your email is what was really there, as opposed to
having been further corrupted by cut-and-paste, CTRL-M's causing
parts of lines to be overwritten on the screen, etc.
I am now convinced that what you showed was for real. Too bad; I
was really hoping that there were control characters in either
amanda.conf or disklist that were causing this. There aren't.
Here are those headers, turned back into plain text, but with
non-printable characters turned into escape sequences:
AMANDA: TAPESTART DATE 20040810 TAPE BootCamp207\n\f\n
AMANDA: FILE 20040810 192.168.3.224 /home/shared/perllib lev 1 comp"N
program /bin/gtar\nTo restore,"position tape at start of file and run:\n\tdd
if=<tape>"bs=32k skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 nis /var/lib/dhcp lev 0 comp N program /bin/gtar\nTo
restore, position tape at start\xA0of file and run:\n\tdd if=<tape> bs=32k
skip=1 | /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 intrepid /usr/shares/marketing lev 1 comp"N program
/bin/gtar\nTo restore,"position tape at start of file and run:\n\tdd
if=<tape>"bs=32k skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 cvs /etc lev 0 comp"N program /bin/gtar\nTo
restore,"position tape at start of file and run:\n\tdd if=<tape>"bs=32k
skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 intrepid /home/mailhub/scottf/recover_test lev 0
comp"N program /bin/gtar\nTo restore, position tape at start of file and
run:\n\tdd if=<tape>"bs=32k skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 192.168.3.224 /home/odin/reports/scripts lev 1 comp"N
program /bin/gtar\nTo restore,"position tape at start of file and run:\n\tdd
if=<tape>"bs=32k skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 nis /var/yp lev"0 comp N program /bin/gtar\nTo
restore, position\xA0tape at start\xA0of file and run:\n\tdd if=<tape> bs=32k
skip=1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 192.168.3.224 /home/odin/stats lev 1 comp"N program
/bin/gtar\nTo restore,"position tape at start of file and run:\n\tdd
if=<tape>"bs=32k skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 nis /var/named lev 0 comp"N program /bin/gtar\nTo
restore,"position tape at start of file and run:\n\tdd if=<tape>"bs=32k
skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 intrepid /usr/shares/support lev 1 comp"N program
/bin/gtar\nTo restore, position tape at start of file and run:\n\tdd if=<tape>
bs=32k skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 192.168.3.224 /etc lev 0 comp"N program /bin/gtar\nTo
restore, position tape at start of file and run:\n\tdd if=<tape> bs=32k
skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 nis /etc lev 0 comp"N program /bin/gtar\nTo restore,
position tape at start of file and run:\n\tdd if=<tape>"bs=32k skip\xBD1"|
/bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 intrepid /usr/shares/IT_Dept lev 0 comp"N program
/bin/gtar\nTo restore, position tape at start of file and run:\n\tdd if=<tape>
bs=32k skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 intrepid /home/amanda lev 1 comp N program
/bin/gtar\nTo restore, position\xA0tape at start\xA0of file and run:\n\tdd
if=<tape> bs=32k skip=1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 intrepid /home/archive lev 0 comp"N program
/bin/gtar\nTo restore, position tape at start of file and run:\n\tdd if=<tape>
bs=32k skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 intrepid /home/snips lev 0 comp"N program
/bin/gtar\nTo restore, position tape at start of file and run:\n\tdd if=<tape>
bs=32k skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 intrepid /home/os-iso lev"1 comp N program
/bin/gtar\nTo restore, position\xA0tape at start\xA0of file and run:\n\tdd
if=<tape> bs=32k skip=1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 intrepid /home/sus lev 0 comp"N program /bin/gtar\nTo
restore,"position tape at start of file and run:\n\tdd if=<tape>"bs=32k
skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 intrepid /etc lev"0 comp N program /bin/gtar\nTo
restore, position\xA0tape at start of file and run:\n\tdd if=<tape> bs=32k
skip=1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 intrepid /usr/shares/integration lev 2 comp"N program
/bin/gtar\nTo restore, position tape at start of file and run:\n\tdd if=<tape>
bs=32k skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 192.168.3.224 /var/lib/mysql lev 1 comp"N program
/bin/gtar\nTo restore,"position tape at start of file and run:\n\tdd
if=<tape>"bs=32k skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 cvs /usr/CVS lev 0 comp N program /bin/gtar\nTo
restore, position tape at start of file and run:\n\tdd if=<tape> bs=32k
skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: FILE 20040810 192.168.3.224 /usr/archive lev 0 comp N program
/bin/gtar\nTo restore, position tape at start of file and run:\n\tdd if=<tape>
bs=32k skip\xBD1"| /bin/gtar -f... -\n\f\n
AMANDA: TAPEEND\xA0DATE 20040810\n\f\n
Here's the same thing, with the *expected* variations (host,
path, dumplevel) ironed out, and the results run through
"sort | uniq -c":
2 AMANDA: FILE 20040810 HOST PATH lev # comp N program /bin/gtar\nTo
restore, position tape at start of file and run:\n\tdd if=<tape> bs=32k
skip\xBD1"| /bin/gtar -f... -\n\f\n
1 AMANDA: FILE 20040810 HOST PATH lev # comp N program /bin/gtar\nTo
restore, position tape at start\xA0of file and run:\n\tdd if=<tape> bs=32k
skip=1 | /bin/gtar -f... -\n\f\n
1 AMANDA: FILE 20040810 HOST PATH lev # comp N program /bin/gtar\nTo
restore, position\xA0tape at start\xA0of file and run:\n\tdd if=<tape> bs=32k
skip=1"| /bin/gtar -f... -\n\f\n
6 AMANDA: FILE 20040810 HOST PATH lev # comp"N program /bin/gtar\nTo
restore, position tape at start of file and run:\n\tdd if=<tape> bs=32k
skip\xBD1"| /bin/gtar -f... -\n\f\n
2 AMANDA: FILE 20040810 HOST PATH lev # comp"N program /bin/gtar\nTo
restore, position tape at start of file and run:\n\tdd if=<tape>"bs=32k
skip\xBD1"| /bin/gtar -f... -\n\f\n
8 AMANDA: FILE 20040810 HOST PATH lev # comp"N program /bin/gtar\nTo
restore,"position tape at start of file and run:\n\tdd if=<tape>"bs=32k
skip\xBD1"| /bin/gtar -f... -\n\f\n
1 AMANDA: FILE 20040810 HOST PATH lev"# comp N program /bin/gtar\nTo
restore, position\xA0tape at start of file and run:\n\tdd if=<tape> bs=32k
skip=1"| /bin/gtar -f... -\n\f\n
2 AMANDA: FILE 20040810 HOST PATH lev"# comp N program /bin/gtar\nTo
restore, position\xA0tape at start\xA0of file and run:\n\tdd if=<tape> bs=32k
skip=1"| /bin/gtar -f... -\n\f\n
1 AMANDA: TAPEEND\xA0DATE 20040810\n\f\n
1 AMANDA: TAPESTART DATE 20040810 TAPE BootCamp207\n\f\n
Things to notice:
1. Some of the spaces have been turned into \xA0
(which is ' ' | 0x80)
2. Some of the "="s have been turned into \xBD
(which is '=' | 0x80; hmmm...)
3. Some of the spaces have been turned into '"'
(which is *not* ' ' | 0x80)
4. (3) can happen anywhere from 0-4 times in a given header, but
at fairly predictable places
5. No characters other than ' ' and '=' have been corrupted
I don't have a clue what all this means, I'm afraid. It still
doesn't smell like hardware data corruption -- it's not
completely regular, but it is semi-regular. Still, to rule that
out, it'd be interesting (to put it mildly) to see whether
similar corruption is occurring in the actual backup data as well
as in the headers. Try this:
1. Run a test backup *without* a tape in the drive; it'll run in
degraded mode, leaving the backups in the holding disk
2. Copy the backups out of the holding disk to a temporary
location
3. Run amflush
4. DD the backups off of the tape -- do *not* try to use
amrestore or amrecover; we want the bits exactly as they're
on tape, except that we use "bs=32k skip=1" to discard the
same headers we were examining above:
dd bs=32k skip=1 <$TAPEDEV >file
5. Similarly, peel the headers off of the holding-disk-file
copies. That looks similar to step 4:
dd bs=32k skip=1 <file1 >file2
6. Compare the corresponding files from steps 4 and 5. (Most
diff's other than GNU diff can't handle binary files, so use
cmp instead). (Note that for huge backups -- by default,
ones greater than 2 GB -- the holding-disk data will be split
into several files, which you'll have to reassemble before
doing the comparison.)
Of course we expect the files from steps 4 and 5 to be identical;
if they're not, you've almost certainly got hardware problems.
If the files *are* identical -- especially on sizeable backups --
but the headers show the same sort of weirdness as before, I
won't know what to make of that :-(
--
| | /\
|-_|/ > Eric Siegerman, Toronto, Ont. erics AT telepres DOT com
| | /
It must be said that they would have sounded better if the singer
wouldn't throw his fellow band members to the ground and toss the
drum kit around during songs.
- Patrick Lenneau
|
