Amanda-Users

invalid compressed data--crc error and other corruption on disk files

2005-02-18 06:55:07
Subject: invalid compressed data--crc error and other corruption on disk files
From: Thomas Charles Robinson <robinstc AT ocean.com DOT au>
To: amanda mailing list <amanda-users AT amanda DOT org>
Date: Fri, 18 Feb 2005 11:36:46 +0000
Hi,

Currently I have several configurations backing up to disk (just not
using a tape in the slot leaves dump-files on the holding disk) but when
I check the dump files for consistency, about a third of them are
corrupted.

I am doing the dump directly to disk now and purposely not flushing them
so I can verifying each dump file manually (I was unsure whether our
tape unit was the cause of the corruption. Unfortunately I still get
corruption).

The manual verification is done using dd, gzip and tar:

dd if=$FILE bs=32k skip=1 | /usr/bin/gzip -dc 2>>$LOGFILE | /bin/tar tf - 
2>>$LOGFILE > /dev/null

BTW I am using rpm versions of amanda, gzip and tar on a Red Hat 9
system:

amanda-2.4.4p1-1
gzip-1.3.3-9
tar-1.13.25-11

All the backups (except for one configuration which I have not bothered
to test) are done on the backup host (i.e. locally) from an ext3
file-system which is rsync-ed to another host. The rsync does not run
during the backup.

Lastly, I am currently using an nfs share for the holding disk but this
was NOT being used previously and I was still getting the corruption
mentioned.

My experience with amanda has been nothing but good in the past but now
I find I need some help on this as I don't know whether gzip, tar or
amanda is at fault. Any help is appreciated.

Thanks in advance,

Tom

The errors are:
--- a file --
gzip: stdin: invalid compressed data--crc error

--- another file ---
/bin/tar: Skipping to next header
/bin/tar: Archive contains obsolescent base-64 headers

gzip: stdin: invalid compressed data--crc error

gzip: stdin: invalid compressed data--length error
/bin/tar: Error exit delayed from previous errors

--- yet another file ---
/bin/tar: Skipping to next header
/bin/tar: Archive contains obsolescent base-64 headers
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Archive contains `v where numeric off_t value expected
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Archive contains `\224\321\377u\241\317' where numeric off_t value 
expected
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Archive contains `v where numeric off_t value expected
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Archive contains `v where numeric off_t value expected
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Archive contains `n\226\006\n-9\377' where numeric off_t value 
expected
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Archive contains `r\n\032\356\212\377' where numeric off_t value 
expected
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header
/bin/tar: Skipping to next header

gzip: stdin: invalid compressed data--crc error

gzip: stdin: invalid compressed data--length error
/bin/tar: Error exit delayed from previous errors
--

--script-- amdumpfileverify 
#!/bin/bash

FILES=$(ls -Sr)

echo testing $FILES

for FILE in $FILES
do
        /var/lib/amanda/bin/amdumpfiletest $FILE
done

exit 0
--script-- amdumpfiletest
#!/bin/bash

LOGFILE=$(basename $0).log

function usage()
{
        echo "usage: $(basename $0) <file>"
}

# start

# check the arguments and call correct function
if [ $# -ne 1 ]; then
        usage
        #exit 0
else
        FILE=$1
        echo TESTING $FILE >> $LOGFILE
        dd if=$FILE bs=32k skip=1 | /usr/bin/gzip -dc 2>>$LOGFILE | /bin/tar tf 
- 2>>$LOGFILE > /dev/null
        echo DONE >> $LOGFILE
fi

exit 0
-------------
__--__-- A programmer from a very large computer company went to a
software conference and then returned to report to his manager, saying:
"What sort of programmers work for other companies? They behaved badly
and were unconcerned with appearances. Their hair was long and unkempt
and their clothes were wrinkled and old. They crashed out hospitality
suites and they made rude noises during my presentation." The manager
said: "I should have never sent you to the conference. Those programmers
live beyond the physical world. They consider life absurd, an accidental
coincidence. They come and go without knowing limitations. Without a
care, they live only for their programs. Why should they bother with
social conventions?" "They are alive within the Tao." -- Geoffrey James,
"The Tao of Programming"

Attachment: signature.asc
Description: This is a digitally signed message part