Neil Marjoram wrote:

I have just completed a tapetype on my Storedge L9 with DLT 8000 again, and I'm a little confused.

The DLT 8000 drive can take 80GB compressed, the results return 40GB.

What you state here is correct.  The *bare* capacity is about 40 GB.

A tapedrive has a compression algorithm built into hardware.  Enabling
this feature *could* augment the capacity of a tape, depending on the
nature of the data.  The bare capacity is still 40 GB, but the hope is
that the result of applying this algoritm to the data creates less bits
to write to tape.  Pure text files, or database files etc probably
compress very well.  However mp3 files, jpgs, zips etc don't compress
well at all.  Actually, blindly applying a compression algorithm to
such data actually expands the data, resulting in *less* tape capacity.

Disklist entries in amanda specify if amanda has to do software
compression or not.  Using hardware compression on the already
compressed archives results in expanding data.
The gzip algorithm that amanda uses avoids expanding already compressed
data too.  (And sometimes performs better than the manufacturers wild
guess of 50% compression, resulting in twice the bare capacity...)

AFAIK there is only one type of drive whose hardware compression
algorithm is smart enough to NOT expand such data, the famous LTO
drives.  All the others, including DLT-drives suffer from expanding
already compress data.   That's why you better do not use both hardware
compression and software compression for all drives except LTO's.

As a side note, amanda's planner, and taper algorithm work better if
they accuratly know what capacity a tape has.  Hardware compression
hides this information to amanda.  That's why we recommend to use
software compression and disable hardware compression, if you have the
CPU resources.

Note that tape manufacturers use SI-prefixes (K=1000, M=1000000,etc)
as opposed to the KiB = 1024 bytes, Gi=1024^3 bytes.
See:   http://physics.nist.gov/cuu/Units/prefixes.html

That means that "40 GB" in manufacturer specifications is about 37.25 gb
or 38146 mb in amanda units (case doesn't matter either to amanda:
the 'b' is bytes, not bits, and 'm' means mega, not milli).


And take also into account that tapes have "about" the stated capacity
and there are a lot of factors that influence this.  E.g. a dusty
tapehead, or non-optimal temperature introduce soft write errors that
actually result in lower capacity (a block is rewritten a few times,
one after another, each time taking a bit amount of tape).
Also, when the tape cannot keep streaming because the input data
is draining, the drive has to stop the tape, rewind a little, restart
again.  Besides the fact that makes your tapedrive excessivly slow,
is

Amstatus shows that the amount of data stored on each tape is always less than 40G, the latest output from the status is :

tape 1        :  66  27906885k  25125884k ( 68.13%) AG3600
tape 2        :   1   9085868k   9327611k ( 22.18%) AG3602
tape 3        :   0         0k         0k (  0.00%) AG3598

These are not the numbers where amanda ran into EndOfTape, but
the capacity of the completely succeeded images only.
While writing file 67 on the first tape, amanda ran into EOT,
and was done all over again on tape 2.
The netto capacity of tape 1 is indeed 29906885k.  And
there happens to be a part the next archive, but that part is
not usable to restore from.
That's not completely unexpected, because 27906885k + 9085868k =
36992753k.  And -- although I can't prove this from the output
you gave -- you have software compression enabled, that is too much
for a DLT tape, with also hardware compression enabled.  (remember
soft+hard expands the data!).

While writing file 2 on tape 2, it ran into EOT too.  That's probably
a large file, and that's now being redone on tape 3.

In the final report amanda sends, you can find out where amanda
exactly hit into EOT in the "Notes" section.  Something like:

Meaning, after having written 31866432 kilobytes and 14 filemarks,
taper ran into EOT.

My question is two part, a. how do I get amanda to backup 80GB to the

You can only get about 40 GB data to a tape.
But you may choose to compress data before feeding it to the tape.

DLT, and secondly why is it that the second tape in a three part backup never holds as much data as the first tape?

This is a complete other problem.  Probably.

If the tapecapacity numbers are correct (see below for two remarks in
your tapetype definition), amanda can optimize the used tapecapacity
by specifying "taperalgo largestfit" in your amanda.conf.

Also note, there is a subtile interaction between number of dumpers,
dumporder and taperalgo, as explained in:

http://marc.theaimsgroup.com/?l=amanda-users&m=106277015010167&w=2

define tapetype DLT-L9 {
comment "DLT 8000 - Storedge L9"
length 40000 mbytes
filemark 22 kbytes
speed 1770 kbytes

Two remarks here.

I think length is a bit optimistic, noted the
explanation above.  Amanda's algorithm work better if given accurate
information.  That's why "amtapetype" was invented.

The speed seems to be a little low for a DLT8000 drive.
What does the spec indicate?  I would expect 3 or 6 mbytes/sec!
Are you sure the tape keeps streaming?
Was that number the result of "amtapetype"?

time; see the man page of amtapetype)

Run:  amtapetype -e 40g -f /dev/....

That should take about 4-5 hours on a not badly tuned machine.

And disable hardware compression!  Amtapetype will warn you about this,
if needed.

I am using the chg-zd-mtx script to handle the autoloader, Fedora Core 2 as my OS and Amanda version 2.4.4p2-3.

--
Paul Bijnens, Xplanation                            Tel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM    Fax  +32 16 397.512
http://www.xplanation.com/          email:  Paul.Bijnens AT xplanation DOT com
***********************************************************************
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* kill -9 1,  Alt-F4,  Ctrl-Alt-Del,  AltGr-NumLock,  Stop-A,  ...    *
* ...  "Are you sure?"  ...   YES   ...   Phew ...   I'm out          *
***********************************************************************