Amanda-Users

Re: Newbie and backup scheme... comments pls

2002-11-29 01:18:57
Subject: Re: Newbie and backup scheme... comments pls
From: Gene Heskett <gene_heskett AT iolinc DOT net>
To: "Joffer" <joffer AT online DOT no>, "Amanda Users List" <amanda-users AT amanda DOT org>
Date: Thu, 28 Nov 2002 23:51:11 -0500
On Thursday 28 November 2002 19:22, Joffer wrote:
>I just got 7 tapes and an old scsi tape streamer from a friend. I
> will be installing Amanda and was thinking I would use the
> following backup scheme.
>
>The 7 tapes are "devided" into:
>2 Tapes OFF-Site
>4 tapes = 4 weeks , mon-sat = incred, sun = full dump
>1 - backup tape just in case
>
>The backup-scheme would be like this.
>
>First I run a full backup using the 1st of 2 OFF-SIte tapes. I
> think I should create a second config for manuel archiving,
> wouldn't that be the best solution?
>
>Then, I'd let the backupcycle begin:
>1st of 4 Cycle tapes: Midnight each Monday to saturday -
>changed/new files only Midnight Sunday - Full backup -
>Eject backup - insert 2nd of 4 Cycle tapes
>
>2nd of 4 Cycle tapes:   same schedule as 1st tape of course.
>
>3... same as 2nd etc
>4.... etc
>1..... etc
>2...... etc
>
>and when my schedule or my instincts tell me it's time for a new
> full off-site backup I do a manual full backup on 2nd of 2
> OFF-Site tapes while my normal cycle goes as normal (if I
> remember to insert the cycle tape again after I've done a manual
> one :)
>
>Since I've never used a tape streamer before and not very much
> into the backup-scheme, I'm asking for comments on my
> backup-scheme. I'de also very much like to get input on how to
> achieve this.. can't say I understand the runspercycle and
> typecycles etc just yet..

Hoo boy, here we go again, somebody wants to mold amanda to their 
way of thinking. :-)

Note the smiley please as you read on.

I think your last paragraph is maybe the most accurate.
You're going to have 4 problems I'd suspect.

1. That drive isn't big enough to backup anything much more complex 
than a floppy based firewall.  You've got 90 meter tapes which will 
hold 1 gig (uncompressed) on a really really good day.  How big did 
you say your hard drive was?

2. That drive, if it has any real use on it, probably has a 
headwheel thats on its last legs, with an accompanying high error 
rate.  I hate to toss water on your fire, but those drives go for 
less than a 5 dollar bill on ebay when they do sell, and I suspect 
many of them cost the seller the sales fee and still go unsold.  
With proper use of the cleaning tape, it may go for 2500-3000 
running hours, and that can pile up in a couple of years if its 
been doing anything like a daily backup of what must have been a 
pretty small system.  ISTR its data rate isn't more than 200kb a 
second, pretty slow in todays world.

3. You want to bend amanda to run things your way, and thats quite 
difficult for a newbie.  Amanda's philosophy is to take what you 
give her, and do the maximum she can with it.  That means amanda 
will want to be run every night, but its amanda who will decide the 
scheduleing, and amanda attempts to spread the backups out and 
adjust things on a day to day basis that will in time result in 
each nightly backup using about the same average amount of tape.

To that end you have 4 variables to control the outsides of the 
envelope amanda can use.

1st is the tapetype defined size of the tape, in your case maybe 950 
megs uncompressed. Turning any hardware compression the drive has 
off is the general recommendation because with it on, amanda can't 
get a clear view of how big the drive is and you have to fudge that 
value, sometimes a lot depending on the compressability of the data 
you backup, an unknown.  Besides, a software compressor can usually 
do a better job, sometimes by amazing ratios.  Amanda will develop 
a history of how each disklist entry compresses, and can after a 
few runs, have a very good idea of how much raw data she can put 
thru the compressor (usually gzip) and still make it fit the tape.

2nd is the 'dumpcycle' in amanda.conf.   This defines to amanda how 
many realtime days she has to fit all the stuff you want backed up 
into the schedule.

3rd is the 'runspercycle' in amanda.conf.  This tells amanda how 
many runs she actually has in the time defined by dumpcycle to do 
her job.

4th is the 'tapecycle' defined in amanda.conf.  This tells amanda 
how many tapes she has to use, and indirectly how quick she can 
re-use a given tape.

Now, to give you an idea, and I'm just a 2 machine home network guy 
who doesn't backup the second machine directly, but maintains on 
this machine, a set of rsync'd 'important' directorys that are 
backed up.

Thats a total of about 150 gigs worth of drives, none of which is 
even remotely close to half full after my last re-arrangement.

My drive is a changer, a 4 tape changer that I leave a cleaning tape 
in the bottom of the magazine, so it has 3 tapes available, which 
are DDS2's at nominally 3.85 gigs per tape uncompressed.  I 
practically stole it on ebay about a year ago, 135 USD, brand new.

I'm currently waiting for the schedule to settle a bit, having 
reduced dumpcycle and runspercycle from 7 days to 5 days over the 
last week.  And I've added a few more tapes to the rotation so 
there are now 28 tapes in the 'tapecycle'.  It shelved a full in 
favor of an incremental last night because the tape would have hit 
EOT and I haven't given it permission to 'runtapes' more than 1.

So in that 5 nights running, I have the ability to fully backup 
5x3.9 or about 17.5 gigs in that 5 days.

With your tape stock, .95 gig x 7 tapes=6.65 gigs if all were used 
in the dumpcycle.  However, this is not a very good arrangment 
because with every tape change, you will be overwriting the last 
good fulls of something or other.  This is why I have what to you 
looks like way too many tapes, but if the 5,5,28 schedule actually 
works, I will have 5.3 full level 0 copies of everything here.  If 
not, then I go back to 6 and 6, giving me 4.4 copies of everything 
on the shelf, actually a smallish solid oak pigeonhole rack I made 
in the shop.  We have lots of oak trees here in WV, USA. :-)

You need at least 2 full copies so that if one fails, you haven't 
destroyed your only good, but slightly older backup of that 
partition.  This is what backing up is supposed to do, give you 
nearly bulletproof recovery.

Also, be aware that amanda cannot, and possibly never will, have the 
ability to append to an already written tape, again the idea is 
bulletproof data security.  No one knows who might eject a tape 
during the day and reinsert it, thereby rewinding it and destroying 
the location knowledge of where it is on the tape.  When that 
happens, then the next nights incremental goes on the headend of 
the tape and everything beyond it on the tape from previous writes 
is then beyond recovery.  Amanda rewinds the tape before she 
starts, and reads the tape ident label you put on the tape, and if 
its not due in the tapecycle, then amanda won't use it.

However, if you have holding disk space enough, you can leave the 
drive empty for several days and then 'amflush' whats there to one 
tape provided it will fit.  When using a holding disk area, be sure 
and leave it out of the disklist, or exclude it in amanda.conf, no 
use backing up what you've already backed up.

Amanda can stand guard over your data like a vicious dog, but she 
would much rather she was allowed to do it her way.  If thats not 
to your liking, then you should probably investigate other 
software, it would be easier for you in the long view.  I also 
don't think the other software and methods will be as dependable 
but that one mans opinion.

However, picking up a newer, bigger drive, and its tapes from ebay 
will no doubt ease the road ahead.  Tape changers are also nice, 
but not a requirement.  120 meter DDS2 tapes are, when available, 
very reasonable, I don't think I've ever paid over 4 USD per tape 
when buying them in sealed tenpacks, brand new.  I've NDI how that 
translates to Norwegian money though, sorry.

Good luck!

-- 
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz  512M
99.19% setiathome rank, not too shabby for a WV hillbilly

<Prev in Thread] Current Thread [Next in Thread>