Amanda-Users

Re: migrating to amanda questions 3, 4 & 5

2006-02-13 17:52:58
Subject: Re: migrating to amanda questions 3, 4 & 5
From: Paul Bijnens <paul.bijnens AT xplanation DOT com>
To: Salvatore Enrico Indiogine <hindiogine AT gmail DOT com>
Date: Mon, 13 Feb 2006 23:48:30 +0100
Salvatore Enrico Indiogine schreef:
Question 3:

From reading the documentation it seems that I have to backup a
complete filesystem and can not simply define a b/u directory like a
/home/user1

Is this correct?


Amanda delegates the backup itself to a program that you specify in the
dumptype of a "DiskList Entry" (DLE for short).
If you use "DUMP" as the dump program, than that is correct.  That is
a limitation of the dump program.
But if you use "GNUTAR", then subdirectories are very well possible.
No problem at all.


Question 4:

The tapedevice is:

define tapetype SDLT320 {
     comment "HP Super DLTtape I, data cartridge, C7980A"
     length 139776 mbytes
     filemark 0 kbytes
     speed 13980 kps
 }

I obtained the specifics from the Amanda wiki.  When I define the b/u
job in the dump type definitions section of amanda.conf do I need to
specify any compression?   I do not think that the tape drive does
any.  At least with BRU you do not need to worry about that setting.

If you do not know or care the implications, then you do not need to
worry about that setting  :-)  So what are the implications?
The algorithm used in many tapedrives is a variation on the LZW algorithm. When you feed random data, or already compressed data
(data with randomness eliminated), then that algorithm has the property
of EXPANDING the data with about 15-20%.
In Amanda you have the option to specify software compression.  Feeding
that bytestream to a tape with hardware compression also enabled,
will result in a loss of tape capacity of about 15-20%.
AFAIK, only the LTO drives use a compression algorithm that does not
expand already compressed data.  (Gzip also avoids this problem.)
The conclusion here is that for most drives (all except LTO?) you
better not do both hardware + software compression.

The advantage of doing software compression is:
- amanda can better predict the tape usage, especially when using
the taperalgorithm "largestfit": read also:
http://wiki.zmanda.com/index.php/Filling_a_tape_to_100%25
(Predicting the real tapecapacity is important to not loose too much
data in the last incomplete, useless chunk on tape: capacity +
time wasted)
- your holdingdisk can hold more client data, resulting in more parallelism possible.
- doing software compression on the client saves bandwith.

The disadvantage of software compression is the load of the cpu's
(that's why many people do "compress client fast":  load is spread
of many cpu's, and fast is only a few percent less than "best", but
does not load the cpu too much).  If the CPU of the hosts cannot
cope with the additional load of the gzip process, than you can
use hardware compression.

The disadvantage of hardware compression is that you can only guess
the amount of bytes that a tape will hold.  What is the
average compression of lots of .txt files combined with lots of
.jpg and .zip files?

The above tapetype entry was one with hardware compression ENABLED, as
mentioned in the lines above it on the wiki.
And indeed, a tapecartridge that is promised to hold 160 Gbyte seems
to loose 20 Gbyte capacity.

The tapecapacity from that entry there is actually useless:
- when you will use software compression, you better disable the
  hardware compression, and you get around 20 Gbyte more on such a tape.
- when you will use hardware compression, add to that 160 gbyte a
  wild guess as how much data will compress.


Question 5:

I have also read that amanda is not able to append to a tape.  At the
moment the incrementals and differentials write to the same tape for
the same b/u job.   If I can not append to the tape do I need to
change them after each incremental?   I think I am not understanding
this correctly.

One run = one (set of) tapes.  Amanda opens the tape device once, writes
all the images and filemarks, and closes the device once.  Using that
sequence, there is no possibility that other programs interrupt the
sequence and accidentelly rewind the tape (just "mt -f /dev/st0 status" is enough, or even "amcheck daily"), or some error like a scsi bus reset
happens, with an implicit rewind too, without Amanda noticing that.
And informing you.
If it wouldn't,  then you could end up with only the last few backup
images on the tape, because the first ones were erased.  And you
wouldn't know unless you tried to restore from it.

When appending to a tape, there is the possibility that, between
the time that Amanda positions to the last image (that already is not
really trivial!), and opening the device for writing, a tape rewind
happens, and in that case Amanda would happily erase ALL of the tape,
and put your tiny incremental backup on that tape, that would be
worthless, because it is missing the level 0 that it is based on (that
was erased by accident).

Even a vtape (virtual tape emulated on disk) has that problem: a scsi
bus reset does not rewind the vtape, but you still can mess with "amcheck" or "ammt", or you are running amrecover to restore a file
from the level 0, while the backup is still running and you forgot that
it is using the same vtape. (But on vtapes you care less about the not-used space at the end of a vtape anyway.)

Until someone can create a patch that gives enough confidence that such
a sequence cannot happen, many experienced sysadmins will stay with one
run = one (set of) tapes.

When you have indeed grossly oversized the tape drive, and there is no
budget anymore to buy more cartridges because all the money went into
the expensive tape drive, then there also a plan B.
Buy a large holdingdisk instead. Set the default "reserve 100" parameter
in amanda.conf to something less, like 30, and set the parameter
"autoflush true" as well.
Do not insert a tape for a few days and let Amanda collect the backups
on the holdingdisk.
When you believe the holdingdisk will fill a tape, insert a tape,
and during the next run, Amanda will autoflush the collected images,
together with the backup of that night to tape.


--
Paul Bijnens, Xplanation                            Tel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM    Fax  +32 16 397.512
http://www.xplanation.com/          email:  Paul.Bijnens AT xplanation DOT com
***********************************************************************
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* kill -9 1,  Alt-F4,  Ctrl-Alt-Del,  AltGr-NumLock,  Stop-A,  ...    *
* ...  "Are you sure?"  ...   YES   ...   Phew ...   I'm out          *
***********************************************************************

<Prev in Thread] Current Thread [Next in Thread>