Hi Mike,
Thanks for your good description of the problem.
You found a bug in a the way the taper read a file from holding disk
if blocksize > 32k.
There is two posible workaround (untested).
1. Set your chunksize to '32k + n * blocksize' where n is an integer.
2. Set file-pad to false.
Setting chunksize to an arbitrary big value will not work if a
dump must be written to two different holding disk, you should set
it acording to workaround 1.
Jean-Louis
On Fri, Mar 28, 2003 at 11:46:11AM -0600, Mike Simpson wrote:
> Hi --
>
> > Any tips or tricks or other thoughts? Is this the Linux dump/restore
> > problem I've seen talked about on the mailing list? I don't
> > understand how the gzip file could be corrupted by a problem internal
> > to the dump/restore cycle.
>
> Answering my own question after a week of testing ... I think I've
> discovered a bug in Amanda 2.4.4. This is what I've deciphered:
>
> (1) Restores of backup sets that compressed to < 1 gb worked fine.
> Backup sets that, when compressed, were > 1 GB blew up every time
> with gzip corruption error messages. This was consistent across
> OS's (Solaris 8, RedHat 7.x), filesystem types (ufs, vxfs,
> ext2/3), and backup modes (DUMP, GNUTAR).
>
> (2) The gzip corruption message always occured at the same spot, i.e.
>
> gzip: stdin: invalid compressed data--format violated
> Error 32 (Broken pipe) offset 1073741824+131072, wrote 0
>
> which is 1024^3 bytes + 128k. I note that in my Amanda
> configuration, I had "chunksize" defined to "1 gbyte" and
> "blocksize" set to "128 kbytes" (the chunksize was just for
> convenience, the blocksize seems to maximize my write
> performance).
>
> (3) I used "dd" to retrieve one of the compressed images that was
> failing. At the 1 gb mark in the file, the more-or-less random
> bytes of the compressed stream were interrupted by exactly 32k of
> zeroed bytes. I note that 32k is Amanda's default blocksize.
>
> (4) For last night's backups, I set "chunksize" to an arbitrarily
> high number, to prevent chunking, which works fine in my setup
> because I use one very large ext3 partition for all of my Amanda
> holding disk, which nullifies concerns about filesystem size and
> max file size. The restores I've done this morning have all
> worked fine, including the ones that had previously shown the
> corruption.
>
> I'm not enough of a C coder to come up with a real patch to fix this.
> I'm hoping the above gives enough clues to let someone who _is_ a
> real C coder do so.
>
> If this should be posted to the amanda-hackers list, please feel free
> to do so, or let me know and I'll do it. Also, if any other
> information would be helpful, just ask.
>
> Thanks,
>
> -mgs
>
>
--
Jean-Louis Martineau email: martineau AT IRO.UMontreal DOT CA
Departement IRO, Universite de Montreal
C.P. 6128, Succ. CENTRE-VILLE Tel: (514) 343-6111 ext. 3529
Montreal, Canada, H3C 3J7 Fax: (514) 343-5834
|
