Amanda-Users

Re: little details that don't seem to be happening

2003-08-10 01:37:43
Subject: Re: little details that don't seem to be happening
From: Gene Heskett <gene.heskett AT verizon DOT net>
To: Craig White <craigwhite AT azapple DOT com>, amanda-users AT amanda DOT org
Date: Sun, 10 Aug 2003 01:30:47 -0400
On Sunday 10 August 2003 00:20, Craig White wrote:
>the gaps in my knowledge seem to close slowly sometimes...
>
>Thanks Jon for the fill-in on exclude list - you were right - I was
>confused by the description in the sample amanda.conf and the word
>pattern isn't supposed to be included - nor the ` or ' - amazing the
>stupid things that I do and the way I can take things way too
> literally some times.
>
>Anyway...
>
>1 - during backup - first tape has filled - apparently ejected (I am
>doing this remotely) but no email from 'chg-manual' which from
> reading the code within, should be coming but hasn't. I do get
> summaries after completion of backup process - is there something I
> need to do to make the request for tape swap email?
>
>2 - I am using a Sony SDT-9000 - which by the way, if anyone is
>interested - results from tapetype...
>define tapetype unknown-tapetype {
>    comment "just produced by tapetype program"
>    length 9675 mbytes
>    filemark 88 kbytes
>    speed 762 kps
>}

>From the above figures its apparent that A: you're not using the 
latest amtapetype, and B: hardware compression is on.

>anyway...I figure it's better to use the hardware compression and
> forget the software compression - Agreed?

Not in this chiefs camp.  Several opinions will no doubt be posted, 
but the final say-so is yours.

Useing hardware compression does this list of things:

1, causes the data from urandom to expand because its not 
compressable.  You see that effect above.  This same effect will 
attack you when backing up that directory full of mp3 or ogg format 
music, and that directory full of tar.gz and tar.bz2 stuff, it will 
quite likely expand instad of compress.

2, it hides the true capacity of the tape from amanda because amanda 
counts bytes sent to the drive and has no idea the drive is modifying 
what it sends by running it thru a compressor.  If the compressor is 
off, then amanda knows exactly what she has sent down the pipe to the 
media, and can do a much better job of adjusting things to prevent 
hitting EOT while cheerfully filling that tape up to 97% or better.  
Amanda counts bytes after any compression she may have commanded to 
be done.

3, the hardware compressor, while fast, isn't capable of more than 
about 2.6/1 compression.  For some files, like text files, gzip can 
do that at 9/1 or better.  The tradeoff is cpu horsepower used.

Now, if I've convinced you to turn the hardware smuncher off, be aware 
that a DDS tape, once written to with the compressor on, is a bit of 
a problem child to get it to turn off because even though you have 
reset the dipswitch to off mode, the tape recognition cycle when you 
put the tape into the drive, will find that flag on the tape and turn 
it back on for you.  Very thoughtfull of the drive, NOT!

What this means is that to get rid of that flag on such a tape, one 
must do something like this:

mt -f /dev/nst0 rewind
# now, save the tapes label
dd if=/dev/st0 of=scratch bs=32k
# turn off the compression
mt -f /dev/st0 defcompression off (or -1 for some mt's)
mt -f /dev/st0 compression off
# now put the label back using non-rewinding device
dd if=scratch of=/dev/nst0 bs=32k
# and flush the drives buffers to force the flag update
dd if=/dev/zero of=/dev/st0 bs=32k count=130 (or more)
# now read the label out to stdout to show you its still ok
# but please note that all the other data on the tape will be gone
dd if=/dev/st0

>Do I adjust the numbers of the 'length?'
>Does the length of 9675 seem small for a DDS-3 (125 meter /
> supposedly 12/24 Gigabyte or does formatting / filemarks / labels
> /etc cut 20%?

No, hardware compression did that. Re-run amtapetype after doing the 
sequence above to your test tape.  See the man page and give it the 
correct estimated size as an argument.  That will speed it up some.
Then do it to all tapes that are to be reused as you cycle them thru 
the drive until you've treated all your tapes to the no-compression
as shown above.  Or you could make a script out of it and do it before 
fireing off amanda.  But be aware that the above requires root 
access, whereas amanda will get her own, so its an "su amanda -c 
'amdump /configname/'" in your root script.

>Thanks,
>
>Craig
>
>sysinfo follows if desired (RH 7.2) / kernel 2.4.20-19
>
>amadmin Archive version
>build: VERSION="Amanda-2.4.2p2"
>       BUILT_DATE="Fri Jul 13 18:48:49 EDT 2001"
>       BUILT_MACH="Linux stripples.devel.redhat.com
> 2.4.5-10enterprise #1 SMP Wed Jun 27 14:01:18 EDT 2001 i686
> unknown"
>       CC="gcc"
>paths: bindir="/usr/bin" sbindir="/usr/sbin"
>       libexecdir="/usr/lib/amanda" mandir="/usr/share/man"
>       AMANDA_TMPDIR="/tmp/amanda" AMANDA_DBGDIR="/tmp/amanda"
>       CONFIG_DIR="/etc/amanda" DEV_PREFIX="/dev/"
>       RDEV_PREFIX="/dev/r" DUMP="/sbin/dump"
>       RESTORE="/sbin/restore" SAMBA_CLIENT="/usr/bin/smbclient"
>       GNUTAR="/bin/tar" COMPRESS_PATH="/usr/bin/gzip"
>       UNCOMPRESS_PATH="/usr/bin/gzip" MAILER="/usr/bin/Mail"
>       listed_incr_dir="/var/lib/amanda/gnutar-lists"
>defs:  DEFAULT_SERVER="localhost" DEFAULT_CONFIG="DailySet1"
>       DEFAULT_TAPE_SERVER="localhost"
>       DEFAULT_TAPE_DEVICE="/dev/null" HAVE_MMAP HAVE_SYSVSHM
>       LOCKING=POSIX_FCNTL SETPGRP_VOID DEBUG_CODE
>       AMANDA_DEBUG_DAYS=4 BSD_SECURITY USE_AMANDAHOSTS
>       CLIENT_LOGIN="amanda" FORCE_USERID HAVE_GZIP
>       COMPRESS_SUFFIX=".gz" COMPRESS_FAST_OPT="--fast"
>       COMPRESS_BEST_OPT="--best" UNCOMPRESS_OPT="-dc"

-- 
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz  512M
99.27% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attornies please note, additions to this message
by Gene Heskett are:
Copyright 2003 by Maurice Eugene Heskett, all rights reserved.