Amanda-Users

Re: chg-manual, span

2003-05-18 18:58:28
Subject: Re: chg-manual, span
From: Gene Heskett <gene.heskett AT verizon DOT net>
To: tjungeblut AT t-online DOT de (Thorsten Jungeblut)
Date: Sun, 18 May 2003 14:56:09 -0400
On Sunday 18 May 2003 05:53 pm, Thorsten Jungeblut wrote:
>On Thu, 15 May 2003, Gene Heskett wrote:
>> On Thu May 15 2003 18:46, Thorsten Jungeblut wrote:
>> >Hi!
>> >
>> >I've problems to span one filesystem over multiple tapes.
>> >For testing I got data of 3 GB to backup to 2GB tapes.
>> >
>> >My config is:
>> >
>> >org "std"               # your organization name for reports
>> >dumpcycle 7 days        # the number of days in the normal dump
>> > cycle tapecycle 10 tapes      # the number of tapes in
>> > rotation bumpsize 20 MB          # minimum savings (threshold)
>> > to bump level 1 -> 2 bumpdays     1          # minimum days at
>> > each level bumpmult     4          # threshold = bumpsize *
>> > (level-1)**bumpmult runtapes     5         # explained in
>> > WHATS.NEW
>>
>> You only have 10 tapes?, yet the dumpcycle is 7 days, and
>> runtapes is 5?
>
>Just for testing... I didn't think about typecycle etc., yet.
>I just want to make sure everything fits on the tapes, so i made
> runtapes big enough.

I'd suspect that you are confusing amanda, and that may lead to 
erronious results.  OTOH, I've not tried to fib to amanda on that 
scale myself.

>> That implies the use of at least 35 tapes, and preferably about
>> 70 in the tapecycle pool.  One really should have 2 full
>> generations of backups on hand.  For some reason, the most
>> recent might be fubar, and its nice to be able to back up to
>> another cycle.  I use DDS2's also, and get them on ebay at quite
>> reasonable prices in the vicinity of $20 a box of 10.  Plus the
>> usual exhorbitant shipping of course.
>>
>> >tpchanger "chg-manual" # the tape-changer glue script, see
>> > TAPE.CHANGERS #changerdev "/dev/null"
>> >changerfile "/etc/amanda/std/changer"
>> >tapedev "/dev/nst0"     # Linux @ tuck, important: norewinding
>> >
>> >holdingdisk hd1 {
>> >        directory "/disk1/amanda"
>> >        comment "main holding disk"
>> >        use 10000 Mb
>> >        chunksize 1 GB
>> >        }
>>
>> How much _could_ you give it? 10 gigs may be a bit tight.
>>
>> >[...]
>> >
>> >define dumptype standard {
>> >    comment "standard"
>> >    no-compress
>> >    index yes
>> >    priority medium
>> >}
>> >
>> >disklist:
>> >
>> >localhost        /dev/hda5       standard
>>
>> Ouch.  For numerous reasons one should not use localhost, but
>> use instead the FQDN of the machine.  Localhost will bite you, a
>> bit higher than the ankles.  Probably at recovery time, which is
>> a bad time to find that out according to people who've run into
>> it.
>>
>> >It is my first try with amanda, so please correct me, too, if I
>> > make some trivial mistakes. :)
>> >
>> >To start complete new run, I do an
>> >
>> >amrmtape std STD00 (STD01, ... Tapes i used in tests before)
>> >amcleanup std
>> >rm -r /var/lib/amanda/std/*
>> >rm -r /disk1/amanda/*
>> >
>> >(is this the correct way to restart? did i miss something?)
>> >
>> >amdump std
>> >
>> >(which should now do an level 0 dump, right?)
>> >
>> >amstatus std
>> >
>> >says, that amanda is dumping the data (everything) to the
>> > holding disk and is starting to write it to the tape.
>> >
>> >then it requires to insert the 2nd tape
>> >then it requires to insert the 3rd tape (? 3 GB should easily
>> > fit on 2 2GB tapes?)
>>
>> It would, IF amanda could span a *single* disklist entry across
>> more than one tape.  It cannot.  You didn't mention which dumper
>> you were using, dump being limited to whole filesystems, but tar
>> can allow one to split the disklist entries up into subdirs
>> which will fit.  Just make sure the tar is at least 1.13-19, and
>> 1.13-25 should be better.  Just plain old 1.13 is broken at
>> recovery time. Generally speaking, most of this group recommends
>> the use of tar, not dump or some cousin of it.
>
>define dumptype standard {
>    comment "standard"
>    no-compress
>    index yes
>    priority medium
>}
>
>Should i just add 'program "GNUTAR"' to my dumptype?

No, add it to your initial configuration arguments.  The location of 
gnutar found by the configuration utility is hard coded into amanda 
at compile time.  See the sample configuration script I use, and 
which was reposted here just a few days back.

>little:/disk1/amanda/20030518# tar --version
>tar (GNU tar) 1.13.25
>Copyright (C) 2001 Free Software Foundation, Inc.
>This program comes with NO WARRANTY, to the extent permitted by
> law. You may redistribute it under the terms of the GNU General
> Public License; see the file named COPYING for details.
>Written by John Gilmore and Jay Fenlason.

Thats great.

>btw: i tried to speed up my test by making my tapes smaller by
> setting length 100 mbytes in my tapetype, but it seems, that
> amanda doesn't bother about it?

It should, but with so many other things going on, I have NDI what 
the planner will wind up doing.  If you are under the impression 
that amanda will ask for a new tape when its done the tapesize in 
mb set it the tapetype, then no, thats not how it works.  What it 
does is give the planner an idea of how much will fit on a tape so 
that it can attampt to work out a schedule.  But taper itself 
ignore this value and cheerfully write data to the tape till it 
hits an EOT from the drive, at which point it will go on to the 
next tape, and restart the dump that encountered the EOT error from 
byte # 0 on the next tape.

You may have a bit of a miss-understanding about amanda.  Amanda is 
not, and wasn't designed as, a one shot backup to be run whenever 
you have the time.  Its designed to be run from a non-root crontab 
entry on a fixed (more or less) schedule.  This gives amanda the 
ability to spread out the fulls done while maintaining them within 
dumpcycle days, given runtapes per run, runspercycle number of runs 
within those dumpcycle days, and tapecycle tapes.

Run in this manner, amanda will work out a schedule, and it might 
take amanda 3 or more dumpcycles to fine tune itself before the 
schedule becomes even remotely predictable, particularly when 
amanda is constrained to use just one tape per nightly run and 
dividing the total to backup into (runspercycle*tapesize) results 
in a tape thats consistently filled to 90% or thereabouts.  Just 
last night, after thinking it had settled into a rut, it had to 
delay a 1.3 gig DLE because it wouldn't fit.  So it appears I'll 
have to further break those 4 entries, which are the debian 30r2 
disks, 2 at a time, into single DLE's per iso, giving amanda a 
finer grained size to play with in adjusting the schedules.

Even after its up and running, expect to pay attention to the mailed 
reports and take whatever corrective action might be indicated, as 
I am going to do yet this evening.  But, once it is up and running, 
thats about all you have to do, which isn't much compared to the 
logistics of trying to do it with some one-shot backup proggy.

>> >amdump exits after 0kb of the 3rd tape, log show an error about
>> > to many tape retries.
>> >
>> >->amflush std
>> >->requires a 4th tape.
>> >
>> >tape ist ejected, changer.debug complains about failing to
>> > write to /dev/tty (I think that should be the request for the
>> > next tape, which failed, because amflush puts itself to the
>> > background)
>> >
>> >Am I doing anything totally wrong?
>> >
>> >Tnx for your help!!

Amanda might have been given a retry counter at some point, doing  a 
bailout if its hopeless rather than burning up all the tapes you 
have without finding one big enough, but thats a SWAG on my part, 
best answered by JRJ, Jean-Louis or one of the other developers.

-- 
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz  512M
99.26% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attornies please note, additions to this message
by Gene Heskett are:
Copyright 2003 by Maurice Eugene Heskett, all rights reserved.


<Prev in Thread] Current Thread [Next in Thread>