Amanda-Users

Re: dumps spanning tapes

2003-05-07 06:47:17
Subject: Re: dumps spanning tapes
From: Gene Heskett <gene.heskett AT verizon DOT net>
To: DK Smith <dks AT MediaWeb DOT com>, amanda-users AT amanda DOT org
Date: Wed, 7 May 2003 06:43:53 -0400
On Wed May 7 2003 04:34, DK Smith wrote:
>>The next level of this (which I thought was very interesting)
>> would be to allow AMANDA to write chunks of backups as they
>> become available, without waiting for an entire DLE to be ready.
>> This would allow the tape to start writing almost immediately,
>> an advantage for sites with lots of large DLEs.

At the present time, there is no marker placed on the tape (that I 
know about) which seperates the chunks.  This is a  carryover from 
the days when most filesystems had a filesize limit quite a bit 
smaller than the drives themselves.  As the filesystems themselves 
get upgraded, I'd assume that might even go away by amanda-3.0 or 
so. :-)  Or it might develop into a marking system that did allow 
it to track the chunks by putting a filemark that included the 
chunk number, and an md5 sum of the previous chunk.  Such a 
mechanism might then be used to allow a DLE to span more than one 
tape by not accepting that chunk as a valid chunk if the tape hits 
EOT and the md5 sum isn't there, *and* you allowed it to use more 
than 1 tape in the runtapes setting.  For this to work at recovery 
time, the chunksize would need to be recorded in the tapes name 
header and recoverable, else the whole scheme would blow up if you 
changed the chunk size trying to get better tape filling.  This 
would be added as an option to amrestore or amrecover by having it 
re-write that variable in the amanda.conf file from the tape 
headers info.  And, with a 32k block being used for a tape label 
holder, it doesn't seem like much of a stretch to record the active 
config settings for the tape right there, thats plenty of room.  If 
it included a count of active DLE entries that would be even more 
useful.

You all can ignore me, I'm (hint) just thinking out loud. :-)

>Sort of near-topic...
>
>Regarding a DLE producing a very large dumpfile... What is the
>experience of those that have added a * in order to file expand
> (or is it regex?) the directory names in /home directory? Would
> this have the effect of making the amdump run shorter?

This is essentially how it works anyway when using tar, because tar, 
given the options it is given, recurses thru the filesystem from 
the base directory you give in the DLE entry.

>>There was a resolution to the restorability issue, I think with a
>> small package of tools at the beginning of each tape, but I'm
>> not sure.
>
>would this not also have the side effect of more specific
> selection possibilities, and hence shorter time to restoration?
>
>( Not suggesting that this will solve anyone's problems, I am
>soliciting experiences of those that did this and were also paying
>close attention :)

Well, atm I am appending to the end of the tape, a copy of the 
indices generated, and the configs as they are in effect at the 
time of the backup's completion.  As this info isn't available 
until the backup run itself is done, there is no handy way to put 
it at the head of the tape.  But if the drive is searchable, the 
recovery of those last two files shouldn't be that big a problem, 
just rewind it, do an fsf to the last filemark amanda wrote, and 
grab those first in the event of needing to do a bare metal 
recovery.

I've purposely reduced the size of the tapetype a bit to make sure 
amanda usually leaves space for the last write my script does, 
which last night was 1940 records for the completed indice tree and 
one record for the config tree according to the email tar sent me.

-- 
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz  512M
99.26% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attornies please note, additions to this message
by Gene Heskett are:
Copyright 2003 by Maurice Eugene Heskett, all rights reserved.


<Prev in Thread] Current Thread [Next in Thread>