Amanda-Users

Re: how to dd if=... a chunked dump blob?

2005-12-06 11:14:39
Subject: Re: how to dd if=... a chunked dump blob?
From: Jon LaBadie <jon AT jgcomp DOT com>
To: amanda-users AT amanda DOT org
Date: Tue, 6 Dec 2005 10:58:26 -0500
On Tue, Dec 06, 2005 at 03:14:44PM +0000, Will Partain wrote:
> I asked:
> 
> >>>   dd if=foo.verilab.com._.1 bs=32k skip=1 | tar tfv - | sort +2nr | head
> >>> 
> >>>What if, instead, my dump blob is "chunked", as in this case (1GB chunks):
> 
> Alexander Jolk suggested (one of two possibilities):
> 
> >> Well, you either do some shell magic:
> >> for i in foo.verilab.com._.1*; do dd if=$i bs=32k skip=1; done | tar tvf 
> >> ...
> 
> Gerhard den Hollander followed on with:
> 
> > or simply
> >
> > cat foo.verilab.com._.1* | dd bs=32k skip=1 | tar tfv - | sort +2nr | head
> 
> Alexander's solution definitely works.  Gerhard's solution may do
> something sensible, but I would classify it as "wrong" -- sorry :-(
> 
> If each chunk has an Amanda header on it (as it appears it does), then
> you need something like Alexander's solution to strip them off.
> 
> With Gerhard's solution, I get ...
> 
>    % cat *.0* | dd bs=32k skip=1 | tar tfv - | sort +2nr
>    tar: Skipping to next header
>    tar: Skipping to next header
>    ... etc...
> 
> ... which means 'tar' is having to do some guesswork, which I don't
> like.  The results I'm seeing on my sample run suggest it is *not*
> "skipping" to the right place; i.e. the results are wrong.  The Jolk
> way is the best way!

AJ's is definitely a correct approach.

An "unchunked" amanda tar dump consists of a long tar file 
"--------------------"
with a 32KB header "||".  So it looks like "||-----------------------------".

When it gets "chunked" an extra header is added to each chunk so it now looks 
like
"||-----", ||-----", "||-----".

You could manually remove each header making tmp files of "-----", "-----" and
"-----", but each would not be a valid, complete, tar file by itself.  Some of
your archived files would be split by the end of one chunk and the beginning
of the next chunk.  Thus the "skipping to next header (tar header) messages.

But you could 'cat' the tmp files together into another big tmp file that would
be the entire original tar dump.  That could be fed to tar.

AJ's nice solution automated the manual steps and eliminated the need for
the temp files by piping directly to tar.

-- 
Jon H. LaBadie                  jon AT jgcomp DOT com
 JG Computing
 4455 Province Line Road        (609) 252-0159
 Princeton, NJ  08540-4322      (609) 683-7220 (fax)

<Prev in Thread] Current Thread [Next in Thread>