On Saturday 10 December 2005 16:13, Ian Turner wrote:
>Gene,
>
>On Saturday 10 December 2005 02:33 pm, Gene Heskett wrote:
>> The patch itself may be great, Kevin, BUT Bzip2 isn't anywhere near
>> as dependable as gzip.
>
>That is because bzip2, by default, uses a much larger block size than
> gzip. Gzip's default block size is a mere 32k, whereas bzip2 uses a
> default 900K block. That's one of the reasons bzip2 can deliver such
> better compression. Even so, it's wrong to say that bzip2 silently
> ignores errors in compressed files. If a CRC check fails, you get
> this message:
>
>bzip2: Data integrity error when decompressing.
>        Input file = foo, output file = bar
>
>It is possible that the compressed file(s) have become corrupted.
>You can use the -tvv option to test integrity of such files.
>
>You can use the `bzip2recover' program to attempt to recover
>data from undamaged sections of corrupted files.
>
>If you find a case where bzip2 silently fails to report an archive
> error, tell us how to reproduce it. Maybe the problem can be fixed.
>
>Cheers,
>
>--Ian

This is how its happened to me, several times.  Only I didn't make 10 
copies of the tree, but I have been known to blow it away and unpack 
it the third time before it would build correctly, twice AIR.  Blowing 
away the first copy when it wouldn't build and the second unpack works 
just has been quite a bit more common.

Go grab a linux kernel src in bz2 format from kernel.org, make yourself 
about 10 directories to play in, and do a "tar xjf /path/to/.bz2" file 
in each of them, and then compare trees.  Somewhere in that 10 unpacks 
there is a decent chance there will be a hiccup, and it will not 
report an error of any kind when it happens.  If that fails by working 
correctly, blow them away and go get a different kernel file, that one 
may not have triggered the buglet.  With enough tries, you will get 
the exact scenario I've seen quite a few times.  Another good error 
detector is to import a known good .config, and make but don't install 
the bzimage & modules.  Some will make, some will error out because 
somethings past its use date or something.  I don't have that sort of 
trouble with gzipped srcs.

And because this is after all amanda, the supposedly bulletproof backup 
program, I consider the use of bzip2 with amanda as a huge hole 
letting errors get into your backups.  A little more tape, or an extra 
day in your dumpcycle makes up for the diff in compression efficiency 
any day.

The bzip2recover program you refer to has not been able to recover a 
useable output for me, ever.

-- 
Cheers, Gene
People having trouble with vz bouncing email to me should use this
address: <gene.heskett AT verizononline DOT net> which bypasses vz's
stupid bounce rules.  I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.