Amanda-Users

Re: Multi-Gb dumps using tar + software compression (gzip)?

2004-10-20 07:21:25
Subject: Re: Multi-Gb dumps using tar + software compression (gzip)?
From: Toralf Lund <toralf AT procaptura DOT com>
To: Amanda Mailing List <amanda-users AT amanda DOT org>
Date: Wed, 20 Oct 2004 13:18:45 +0200
Gene Heskett wrote:

On Tuesday 19 October 2004 11:10, Paul Bijnens wrote:
Michael Schaller wrote:
I found out that this was a problem of my tar.
I backed up with GNUTAR and "compress server fast".
AMRESTORE restored the file but TAR (on the server!) gave some
horrible messages like yours.
I transferred the file to the original machine ("client") and all
worked fine.
I guess this is a problem of different tar versions ...
That's strange and freightening!  Tar is supposed to be a portable
format!  Especially gnutar  -- there are indeed differences with
normal OS-supplied tar formats, but only to overcome limits in
filesize, path name length etc.; but the same version of gnutar on
different architectures should be able to read each others files.

I'm not 100% sure what happens if you compile tar on an architecture
without largefile support on and try to restore a file exceeding
such a limit.

Are you sure you used the correct version of tar. I've called mine
"gtar" to avoid confusion with the OS-supplied tar (actually, amanda
even uses "amgtar", which is a link to the correct version, or a
wrapper that does some pre/post processing if needed on e.g.
database DLE's).

We probably should point out to the new bees here, that tar-1.13 is indeed broken. In other words, if your "tar --version" doesn't report that its at least 1.13-19, it may not, and probably is not, compatible with anything but itself. (and I'm not sure that 1.13 could even recover its own output!)

I hate to be boreing and repetitive, but there are those here *now* who did not go thru that period of hair removal that 1.13 caused.
Yep.

But how about gzip? Any known issues there? I think I've ruled out problems with one particular gzip version since I've tried server as well as client compression, where the client has a different gzip version from the server (and I've tried using both for recovery, too), but if a range of releases has a problem...

Other possible error sources that I think I have eliminated:

  1. tar version issues - since gzip complains even if I just uncopress
     and send the data to /dev/null, or use the -t option.
  2. Network transfer issues. I get errors even with server
     compression, and I'm assuming gzip would produce consistent output
     even if input data were garbled  due to network problems.
  3. Problems with a specific amanda version. I've tried 2.4.4p1 and
     2.4.4p3. Results are the same.
  4. Problems with a special disk. I've tested more than one, as target
     for "file" dumps as well as holding disk.

- Toralf