Amanda-Users

Re: ???

2002-09-10 00:19:31
Subject: Re: ???
From: Gene Heskett <gene_heskett AT iolinc DOT net>
To: Galen Johnson <gjohnson AT trantor DOT org>, amanda-users AT amanda DOT org
Date: Tue, 10 Sep 2002 00:03:26 -0400
On Monday 09 September 2002 21:49, Galen Johnson wrote:
>
>Ok...I'm pretty sure I'm gonna have to change the number of
> runtapes to at least 2 since the current system being backed up
> is almost through a tape (using whatever software they used
> previously nad I'm guessing hardware compression). 

Gaaccckkk!  Suffice to say that to amanda, hardware compression 
strikes a pretty crippling blow because amanda counts bytes *sent 
to the tape* and with hardware compression on, there are many 
current archival methods that do their own crunching so well that 
the rll encoder in the tape drive is overpowered and the data will 
actually grow some.  Software typically gets me, for those 
partitions I can compress, output thats about in the middle 30% of 
the the input file size.  I don't compress all partitions because 
some aren't compressable, and you can tell those from the email 
report, the compression ratio will be over 100%.  Thats not good.

With hardware disabled, then amanda counts the bytes sent to the 
tape after whatever compression is done, and knows exactly how much 
the tape can handle.


 Jon La Badie
> pointed out that since I've 3 magazines that my possession that
> tapecycle would really be 21.
>
>>>runtapes 1  #I'm thinking of upping this to 2 or 3
>>
>>You can, but probably won't need to if the tapes are properly
>> sized, the proper sizing being that a tape must be capable of
>> holding any one disklist entry with room enough for a few
>> incrementals too.
>
>see above...
>
>>>tpchanger "chg-scsi"    # not sure about this yet
>>>tapedev "/dev/nst0"
>>
>>This if you're going to use chg-scsi, will turn into the config
>>number you use out of a multi-config bearing chg-scsi.conf file.
>
>Actually need to change this to either chg-mtx or chg-zd-mtx...
>
>>>rawtapedev "/dev/null"
>>
>>comment this one
>
>Roger, wilco.
>
>>>changerdev "/dev/sg0"
>>
>>This is whatever your dmesg file idents it as.
>
>it is...I've been playing with the changer with mtx to get the
> feel for the eccentricities (and it has some).
>
>>>tapetype EXB-EZ17
>>>
>>>where EXB-EZ17 is defined by:
>>>
>>>define tapetype EXB-EZ17 {
>>>  comment "just produced by tapetype program"
>>>  length 18600 mbytes
>>>  filemark 623 kbytes
>>>  speed 2137 kps
>>>}
>>>
>>>
>>>The current plan is to set up the backups to run 1 full per week
>>>with incrementals (yes, I know amanda does her own thing). 
>>> There are currently three 7-tape containers available which I
>>> plan to use to allow off-site store for 1 container in
>>> rotation.
>>
>>amanda should know each tape as a unique identifier, so the
>>recomendations above then become at least 3x plus spares, not 2x.
>>Label them all sequentially.  Put a cleaning tape in the last
>> slot of each magazine, and if you're carefull, only using one
>> tape a night, you can stay with a set day of the week to change
>> the magazines, just be sure after changing the magazine that you
>> do an "amtape /config/ reset" so she knows shes back to slot 1.
>>
>>As you build the disklist, do it incrementally so that she won't
>> try and do more than one tapes worth of level 0's on any one
>> nightly run.
>
>not exactly sure how...yet.

Well, you can make it up in one shot once you have the syntax down, 
but then I'd start at the top of the list with only enough 
uncommented to 2/3rds fill a tape, and uncomment about that much 
for the next run each time till you get to the bottom of the list.  
That way, no one backup will be all level 0's and it will have a 
head start on adjusting its scheduling to even out the average size 
of each nights run.

I have 37 entries in my disklist, covering a 46 gig drive which also 
has an rsync image of my gateway machine in one directory.  I'm 
using a DDS2 changer, 4 gig tapes (nominally if you want to believe 
the marketing weenies) and my average nights actual on tape data 
size is about 2gigs.  The input size is closer to 6 gigs before the 
"compress server best" compression is applied.

>>>Any input would be greatly appreciated.  This is running Amanda
>>>2.4.3b3 on a Slackware 8.1 OS using tar (not dump).  I
>>> configured the build  to use /home/amanda/bin/tar (which is
>>> currently just a copy of tar but could easily become a
>>> wrapper).  I followed Gene's advice and made myself a build
>>> script (since I found myself building and rebuilding as I came
>>> across new material). It's as follows:
>>>
>>>#!/bin/sh
>>># Since I'm always forgetting to build it as the amanda user
>>>if [ `whoami` != 'amanda' ]; then
>>>       echo
>>>       echo "WARNING!!!!"
>>>       echo "AMANDA needs to be configured and built by the
>>>amanda user," echo "but must be installed by root."
>>>       echo
>>>       exit 1
>>>fi
>>
I "borrowed" the above for my own script, neat little safety 
feature, thanks. :-)

>>Love it, can I steal it?
>
>Help yourself...I basically stole your script... :-D...Jon La
> Badie also had mention of some things he needs to do on his
> system...I may just mod my script and throw it up for grabs.
>
>>>make clean
>>>rm -f config.status config.cache
>>>./configure --prefix=/home/amanda \
>>>           --libdir=/usr/local/lib/amanda \
>>>           --mandir=/usr/local/man \
>>>           --with-user=amanda \
>>>           --with-group=disk \
>>>           --with-config=daily \
>>>           --with-configdir=/home/amanda/config \
>>>           --with-gnutar=/home/amanda/bin/tar \
>>>           --with-gnutar-listdir=/home/amanda/var \
>>>           --with-changer-device=/dev/sg0 \
>>>           --with-tape-device=/dev/nst0 \
>>>           --with-amandahosts \
>>>           --with-gnu-ld \
>>>           --with-db=db \
>>>           --with-debugging=/tmp/amanda-dbg
>>>
>>># Create directories that aren't created by make
>>>if [ ! -d /home/amanda/var ]; then
>>>       mkdir /home/amanda/var
>>>fi
>>>
>>>if [ ! -d /home/amanda/config ]; then
>>>       mkdir /home/amanda/config
>>>fi
>>>
>>>
>>>=G=
>>
>>Looks good with the above caveats.  Welcome to the amanda world.
>
>thx...I'm still skirting the atmosphere and turning red on
> planetfall. With everyone's help thus far...I should bring it
> down nice and easy...of course I still have the turbulence of
> samba ahead.

Humm, bit of weather advisory, samba doesn't handle dates all that 
well because its winderz SMB workalike, and doesn't have all the 
dateing a *nix filesystem does because winderz doesn't, the end 
results will be lots of messages in your email from amanda that the 
"file changed while we read it", even if its a linux filesystem on 
the other end of the network cable!

You'll know better, but amanda won't, so she will back it up when 
its been done yesterday already thinking the file has changed 
everytime she runs by it.  So in that regard, its a space on the 
tape waster.  

Now if we could just figure out how to make rsync do this wholesale 
to a local holding disk mirror, and then backup the mirror like I 
do for my gateway machine, now that would be nice.  And rsync 
doesn't have this winderz "game leg" re filedates.   But, I just 
ran du against that holding dir, and its just short of a tapefull 
at 3,841,420kb, but a lot of it is source code, often compressible 
to less than 15% of its real size. :-)

That said, I'm not running the absolute latest samba either, 
something about a library version clash so I'm still running 2.2.3 
on both machines shared to each other.  Maybe thats been fixed, but 
as thats not part of the M$ Server Message Block specs samba 
emulates, I have doubts thats been changed.

-- 
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz  512M
99.14% setiathome rank, not too shabby for a WV hillbilly

<Prev in Thread] Current Thread [Next in Thread>