Amanda-Users

Re: Large filesystems...

2003-05-19 04:40:06
Subject: Re: Large filesystems...
From: Gene Heskett <gene.heskett AT verizon DOT net>
To: amanda-users AT amanda DOT org
Date: Mon, 19 May 2003 04:38:19 -0400
On Monday 19 May 2003 03:40, Richard Russell wrote:
><snip>

And I've put this back on the list only, all those CC's were getting 
confusing.

>> >I thought Amanda got dump images from clients in parallel, and
>> > put them on the holding disk, and _then_ wrote these out to
>> > the
>>
>> tape,  one
>>
>> >at a time...
>>
>> It does exactly that, unless there is no holding disk at all. 
>> But in doing so, then tar has lost the ability to track whats
>> been sent to the tape, and therefore has no idea of how many
>> bytes have been sent to the media.  Ditto for dump.
>
>Ah, of course... bit what about the actual program it uses to
> write to tapes? Is it dd, something else, or does it do its own
> writing?

I think that depends on your definition of amanda.  There is an amdd 
utility available, and this may be heavily borrowed from dd, but if 
its typical of this type of util, its been stripped of anything 
amanda doesn't normally need, or that might be "platform" 
dependant.

I've also never, ever seen it in a 'ps -ea' output while amanda is 
running. For whatever that might be worth...

I've played with ammt a few times, only to find its not as capable 
as mt.  It can do what amanda needs, but its not terribly aware of 
compression related commands and IIRC it can't even do a 'tell'.

I suspect those utils are there as much to help in a recovery as 
they are in actual use by amanda.  Rewinds.label reads and such are 
generally done with ioctl messages right in the running code as 
opposed to having the running code call a shell with one of those 
utils as argument.

><snip>
>
>> Better yet, when you have it setup and running, simply box up
>> the last "dumpcycle" tapes and take them to secure storage on
>> say Monday's, bringing in the ones taken offsite the week
>> before. With 3 dumpcycles worth of tapes in the pool, you could
>> leave 2 sets of them offsite, a highly desirable condition,
>> particularly if there are 2 different offsite storage locations
>> available.
>
>True... True...
>
>> This simplify's amanda's record-keeping immensely, and doesn't
>> require two seperate configurations, nor for you to recall
>> exactly what was where in which config.  That can and will be
>> (Murphy is alive and well, thank you, I ran into him just
>> yesterday morning), a major PITA, only discovered when its too
>> late.
>
>True...
>
>> Hopefully, unless the accountant is really trying to cover his
>> tracks, all 3 locations won't burn simultainiously.  OTOH, the
>> accountant should NOT have accurate, uptodate knowledge of the
>> offsite location(s) ever.  Thats just plain good CMA sense.  A
>> little paranoia is a Good Thing(tm). :)  FWIW, I've seen several
>> examples of that failure in my 68 years.
>
>Mmmm... true...
>
>Thing is, my users have this annoying habit of wanting something
>restored from three weeks ago... Which complicates issues further,
> as I need to have onsite backups for about a month, but would
> like to have offsites less than a week old...

Thats where you extract that bottle of Lynchburg Lemonaide, for the 
trip home to get them :-)

In setting this up, you have the opportunity to set in stone, what 
it can, and cannot do as far as management is concerned, and that 
changing things after the fact is somewhat like reinventing wheels, 
to be avoided.  If they want archival backups of say, more than a 
months duration of availablity, then you must make them understand 
that there will be an ongoing expense line for the tapes taken out 
of the service pool and placed in long term, non-reusable storage.

Because taking the tape out of amanda's tapelist with an amrmtape 
command also wipes the indices and such, it comes to mind that you 
may want to do something I've been doing here for several months 
when I discovered that the indices (record = yes) weren't uptodate, 
and sometimes even missing, I presume from file locks preventing 
access since they are in one of my LDE's.

So I have a seperate wrapper script that runs amanda, and when 
amanda is done, calls a second script does a simple tar of that 
data, and appends it to the end of the tape, so that I can always 
do an fsf to the last file on the tape, and recover the indices 
(and also in my case the config dir tree since that might also 
change as I adjust the disklist, etc) as they existed at the time 
that tape was recorded.

Pretty simple minded stuff, but they seem to work just fine in the 
tests so far.  A bit of a pita to extract that from the end of the 
tape, but a genuine butt-saver since you are ready for a full 
almost bare metal recovery to that epochs state with that data in 
hand.  I could email you a copy, but its not "purty" enough for 
wide public consumption.  I can figure out just enough bash to be 
dangerous :(

I also have purposely restricted amanda's view of the tapesize by 
about 250 megs to help prevent EOT's while writing that potentially 
valuable data.  Saving a spot for it so to speak, and so far thats 
worked just fine. The data is actually about 55 megs for this 
particular system. YMMV of course.  It also means I have two 
copies, in seperate directories, of all that data, and it also does 
the housekeeping on a per tape basis, when a tape is reused, its 
old data files are rm'd and new ones generated.  One of the reasons 
its not "purty" and would need fine tuning for the next persons 
useage. :)

-- 
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz  512M
99.26% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attornies please note, additions to this message
by Gene Heskett are:
Copyright 2003 by Maurice Eugene Heskett, all rights reserved.


<Prev in Thread] Current Thread [Next in Thread>