Amanda-Users

Re: amrecover failure, corrupted gzip file?

2003-03-28 14:04:57
Subject: Re: amrecover failure, corrupted gzip file?
From: Mike Simpson <msimpson AT abyssus.doit.wisc DOT edu>
To: amanda-users AT amanda DOT org
Date: Fri, 28 Mar 2003 11:46:11 -0600
Hi --

> Any tips or tricks or other thoughts?  Is this the Linux dump/restore 
> problem I've seen talked about on the mailing list?  I don't 
> understand how the gzip file could be corrupted by a problem internal 
> to the dump/restore cycle.

Answering my own question after a week of testing ... I think I've 
discovered a bug in Amanda 2.4.4.  This is what I've deciphered:

(1) Restores of backup sets that compressed to < 1 gb worked fine.
    Backup sets that, when compressed, were > 1 GB blew up every time
    with gzip corruption error messages.  This was consistent across
    OS's (Solaris 8, RedHat 7.x), filesystem types (ufs, vxfs, 
    ext2/3), and backup modes (DUMP, GNUTAR).

(2) The gzip corruption message always occured at the same spot, i.e.

        gzip: stdin: invalid compressed data--format violated
        Error 32 (Broken pipe) offset 1073741824+131072, wrote 0

    which is 1024^3 bytes + 128k.  I note that in my Amanda 
    configuration, I had "chunksize" defined to "1 gbyte" and 
    "blocksize" set to "128 kbytes" (the chunksize was just for
    convenience, the blocksize seems to maximize my write 
    performance).

(3) I used "dd" to retrieve one of the compressed images that was 
    failing.  At the 1 gb mark in the file, the more-or-less random
    bytes of the compressed stream were interrupted by exactly 32k of
    zeroed bytes.  I note that 32k is Amanda's default blocksize.

(4) For last night's backups, I set "chunksize" to an arbitrarily
    high number, to prevent chunking, which works fine in my setup
    because I use one very large ext3 partition for all of my Amanda
    holding disk, which nullifies concerns about filesystem size and
    max file size.  The restores I've done this morning have all 
    worked fine, including the ones that had previously shown the
    corruption.

I'm not enough of a C coder to come up with a real patch to fix this. 
I'm hoping the above gives enough clues to let someone who _is_ a 
real C coder do so.

If this should be posted to the amanda-hackers list, please feel free 
to do so, or let me know and I'll do it.  Also, if any other 
information would be helpful, just ask.

Thanks,

-mgs




<Prev in Thread] Current Thread [Next in Thread>