Amanda-Users

Re: Amanda Compression

2004-09-23 05:09:31
Subject: Re: Amanda Compression
From: Paul Bijnens <paul.bijnens AT xplanation DOT com>
To: kshriyan AT redhat DOT com
Date: Thu, 23 Sep 2004 11:03:50 +0200
Kaushal Shriyan wrote:


I have a 40GB/80GB HP VS80 DLT Tape Drive. Now amanda is only able to
recognize 40 GB only how do i make it enable to utilize the 80GB space
on the tape. Is there any way out ???

40 Gbyte is the native capacity.  80 Gbyte is what the marketing
department would like you to believe when you use hardware compression.

The nature of compression makes that some files compress very well,
other almost nothing, and, depending on the algoritm used, some may
even expand.  Marketing likes you to believe that "on average" you
can compress to 50% resulting in twice as much raw data on the tape.

That means it depends on _your_ data how much actually will fit
on the tape.  If you have an almost empty oracle database, then you may
compress even to 80% or 90% (meaning you can put a few 100 GByte on such
a tape).   If on the other hand your disk is full of jpgs, gifs, mpegs
mp3's, tar.gz-files etc, then most(*) tape drives with hardware
compression enabled will actually expand your data (usually to about
120%), resulting in a tapecapacity around 36 Gbyte, instead of the
expected 40 Gbyte.

The conclusion of this is that when using hardware compression,
you don't know how much will actually fit on a tape.
Amanda does not like this:  she can't plan very well.

If you use software compression however, amanda pipes the data through
gzip before writing to tape.  This has two advantages:  gzip does not
suffer from the "expand already compressed data" syndrome, and amanda
has an exact knowledge how much bytes to write on the tape.
The added accuracy greatly improves the planning capacities of amanda.
Amanda really likes to schedule using accurate numbers.

The report shows you how both amounts: before and after compression.
From my daily backup:

|STATISTICS:
|                          Total       Full      Daily
|                        --------   --------   --------
|...
|Output Size (meg)       30104.2    16097.0    14007.2
|Original Size (meg)     73802.4    32964.9    40837.5
|Avg Compressed Size (%)    40.8       48.8       34.3

Using software compression resulted in 73 Gbyte data compressing
to 30 Gbyte.  (I use AIT-1 tapes, 35 Gbyte native, actually
amtapetype reports 33400 Mbyte: difference between marketing and
reality).

Don't use software compression + hardware compression at the same
time.  The result is a reduced tapecapacity, as explained above,
already compressed data will expand on tape.
That's why "amtapetype" warns you about hardware compression.


(*) Really modern tapedrives -- actually, I know only one: the LTO-drives -- have an algorithm that expands data almost never.
Using such drives, you can mix hardware + software compression.
Do software compression to feed amanda with better statistics,
and do hardware compression only on those DLE's where you want
to spare CPU-cycles.


--
Paul Bijnens, Xplanation                            Tel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM    Fax  +32 16 397.512
http://www.xplanation.com/          email:  Paul.Bijnens AT xplanation DOT com
***********************************************************************
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* kill -9 1,  Alt-F4,  Ctrl-Alt-Del,  AltGr-NumLock,  Stop-A,  ...    *
* ...  "Are you sure?"  ...   YES   ...   Phew ...   I'm out          *
***********************************************************************



<Prev in Thread] Current Thread [Next in Thread>