Amanda-Users

Re: spanning tapes

2005-10-31 08:30:45
Subject: Re: spanning tapes
From: Gene Heskett <gene.heskett AT verizon DOT net>
To: amanda-users AT amanda DOT org
Date: Mon, 31 Oct 2005 09:18:14 -0400
On Monday 31 October 2005 07:13, Jeff Allison wrote:
>Does anyone know how to switch off the tape spanning functionallity in
>2.4.5 I've recently updated from 2.4.2 to 2.4.5 and now my backups
never
>complete I have runtapes set to 3 and use the change-manual script.

I don't think the tape spanning patches have been applied to 2.4.5,
only to 2.5.0 IIRC.  Are you sure you don't have a disklist entry whose
level 0 size exceeds the tapes capacity?

>I've attached my disklist and amanda.conf
>
>this is the end of the log file
>
>SUCCESS taper dalston //bw/backup 20051030 0 [sec 1363.228 kb 1343892
>kps 985.8 {wr: writers 41998 rdwait 1.292 wrwait 1345.685 filemark
>11.265}]
>INFO taper tape homes_03 kb 11287584 fm 14 writing file: No space left
>on device
>START taper datestamp 20051030 label homes_04 tape 1
>INFO taper retrying dalston:/home/samba.0 on new tape: [writing file:
No
>space left on device]
>FAIL dumper dalston /home/samba 20051030 0 ["data write: Connection
>reset by peer"]
>  sendbackup: start [dalston:/home/samba level 0]
>  sendbackup: info BACKUP=/usr/local/bin/amtar
>  sendbackup: info RECOVER_CMD=/bin/gzip -dc |/usr/local/bin/amtar
-f...
>-
>  sendbackup: info COMPRESS_SUFFIX=.gz
>  sendbackup: info end
>FAIL driver dalston /home/samba 20051030 0 [dump to tape failed]
>INFO taper tape homes_04 kb 0 fm 0 [OK]
>FINISH driver date 20051030 time 51998.006
>
>basically what is happening is that the script is prompting for a new
>tape, and once it gets one it writes nothing to it and exits.
>
>Any Ideas ???
>
>-=-=-=-=Disklist-=-=-=-=-=
>
>dalston //bw/backup debs -1 eth0
>dalston /home/jeff comp-user-tar -1 local
>dalston /home/jim comp-user-tar -1 local
>dalston /home/debs comp-user-tar -1 local
>dalston /home/chris comp-user-tar -1 local
>dalston /home/tony comp-user-tar -1 local
>dalston /home/stuart comp-user-tar -1 local
>dalston /home/mark comp-user-tar -1 local
>dalston /home/bw comp-user-tar -1 local
>dalston /home/betterware comp-user-tar -1 local
>dalston /home/samba comp-user-tar -1 local
>dalston / root-tar -1 local
>leyton /home/jeff comp-user-tar -1 eth0
>#leyton /mnt/data/My\ Music comp-user-tar -1 eth0
>#leyton /mnt/mp3s comp-user-tar -1 eth0
>
>-=-=-=-=Amanda.conf-=-=-=-=
>
>
>
>#
># amanda.conf - sample Amanda configuration file.  This started off
life
>as
>#               the actual config file in use at CS.UMD.EDU.
>#
># If your configuration is called, say, "csd", then this file normally
>goes
># in /etc/amanda/csd/amanda.conf.
>#
>
>org "homes"                             # your organization name for reports
>mailto "root"                           # space separated list of operators at 
>your site
>dumpuser "amanda"               # the user to run dumps under
>
>inparallel 4                            # maximum dumpers that will run in 
>parallel
>
>dumporder "Ssss"        # specify the priority order of each dumper
>                        #   s -> smallest size
>                        #   S -> biggest size
>                        #   t -> smallest time
>                        #   T -> biggest time
>                        #   b -> smallest bandwitdh
>                        #   B -> biggest bandwitdh
>                        # try "BTBTBTBTBTBT" if you are not holding
>                        # disk constrained
>
>taperalgo first         # The algorithm used to choose which dump image
>to send
>                        # to the taper.
>
>                        # Possible values: [first|firstfit|largest|
>largestfit|smallest|last]
>                        # Default: first.
>
>                        # first         First in - first out.
>                        # firstfit      The first dump image that will
>fit on the current tape.
>                        # largest       The largest dump image.
>                        # largestfit    The largest dump image that
will
>fit on the current tape.
>                        # smallest      The smallest dump image.
>                        # last          Last in - first out.
>
>displayunit "m"         # Possible values: "k|m|g|t"
>                        # Default: k.
>                        # The unit used to print many numbers.
>                        # k=kilo, m=mega, g=giga, t=tera
>
>netusage 1000 Kbps              # maximum net bandwidth for Amanda, in KB per 
>sec
>
>dumpcycle 2 weeks               # the number of days in the normal dump cycle
>runspercycle 2 weeks    # the number of amdump runs in dumpcycle days

I wasn't aware that one could specify runspercycle in weeks!  So I'd
have used 14 days, although both are very long periods when it comes to
doing a recovery.  Too long IMO.

>tapecycle
17 tapes        # the number of tapes in rotation
>                                                # 4 weeks (dumpcycle) times 5 
> tapes per week (just
>                                                # the weekdays) plus a few to 
> handle errors that
>                                                # need amflush and so we do 
> not overwrite the full
>                                                # backups performed at the 
> beginning of the previous
>                                                # cycle
>### ### ###
># WARNING: don't use `inf' for tapecycle, it's broken!
>### ### ###
>
>bumpsize 20 Mb                          # minimum savings (threshold) to bump 
>level 1 -> 2
>bumppercent 20          # minimum savings (threshold) to bump level 1
->
>2
>bumpdays 1                              # minimum days at each level
>bumpmult 4                              # threshold = bumpsize * 
>bumpmult^(level-1)
>
>etimeout 300                            # number of seconds per filesystem for 
>estimates.
>dtimeout 1800           # number of idle seconds before a dump is
>aborted.
>ctimeout 30             # maximum number of seconds that amcheck waits
>                        # for each client host
>
>tapebufs 20
># A positive integer telling taper how many 32k buffers to allocate.
># WARNING! If this is set too high, taper will not be able to allocate
># the memory and will die.  The default is 20 (640k).

I have a gig of ram, so I'm useing 80 for tapebufs

># Specify tape device and/or tape changer.  If you don't have a tape
># changer, and you don't want to use more than one tape per run of
># amdump, just comment out the definition of tpchanger.
>
># Some tape changers require tapedev to be defined; others will use
># their own tape device selection mechanism.  Some use a separate tape
># changer device (changerdev), others will simply ignore this
># parameter.  Some rely on a configuration file (changerfile) to
># obtain more information about tape devices, number of slots, etc;
># others just need to store some data in files, whose names will start
># with changerfile.  For more information about individual tape
># changers, read docs/TAPE.CHANGERS.
>
># At most one changerfile entry must be defined; select the most
># appropriate one for your configuration.  If you select man-changer,
># keep the first one; if you decide not to use a tape changer, you may
># comment them all out.
>
>runtapes 3                              # number of tapes to be used in a 
>single run of amdump
>tapedev "/dev/nst0"
>tpchanger "chg-manual"          # the tape-changer glue script
>changerfile "/usr/local/var/lib/amanda/homes/changer"
>changerdev "/dev/null"

I don't believe this last should be specified, one device only IIRC.

>tapetype C1537A                 # what kind of tape it is (see tapetypes below)
>labelstr "^homes_[0-9][0-9]*$"          # label constraint regex: all tapes
>must match
>
># Specify holding disks.  These are used as a temporary staging area
for
># dumps before they are written to tape and are recommended for most
>sites.
># The advantages include: tape drive is more likely to operate in
>streaming
># mode (which reduces tape and drive wear, reduces total dump time);
>multiple
># dumps can be done in parallel (which can dramatically reduce total
>dump time.
># The main disadvantage is that dumps on the holding disk need to be
>flushed
># (with amflush) to tape after an operating system crash or a tape
>failure.
># If no holding disks are specified then all dumps will be written
>directly
># to tape.  If a dump is too big to fit on the holding disk than it
will
>be
># written directly to tape.  If more than one holding disk is specified
>then
># they will all be used round-robin.
>
>holdingdisk hd1 {
>    comment "main holding disk"
>    directory "/mnt/tmp/amanda"                 # where the holding disk is
>    use 7 Gb                            # how much space can we use on it

This is a bit small for a nominally 12GB tape, if you have the space,
use something like '-1 Gb' or '-500 Mb' so it will use all existing
space except what you've specified it should leave as I show above.  I
like to see at least 2x the tapes size and currently have about 25Gb to
spare for a holding disk.

>                                                # a negative value mean:
>                                                #        use all space except 
> that value
>#    chunksize 2 Gb             # size of chunk if you want big dump to be
>                                                # dumped on multiple files on 
> holding disks
>                                                #  N Kb/Mb/Gb split disks in 
> chunks of size N
>                                                #  0          split disks in 
> INT_MAX/1024 Kb chunks
>                                                # -N Kb/Mb/Gb dont split, dump 
> larger
>                                                #             filesystems 
> directly to tape
>                                                #             (example: -2 Gb)
>    }
>#holdingdisk hd2 {
>#    directory "/dumps2/amanda"
>#    use 1000 Mb
>#    }
>#holdingdisk hd3 {
>#    directory "/mnt/disk4"
>#    use 1000 Mb
>#    }
>
>
># If amanda cannot find a tape on which to store backups, it will run
># as many backups as it can to the holding disks.  In order to save
># space for unattended backups, by default, amanda will only perform
># incremental backups in this case, i.e., it will reserve 100% of the
># holding disk space for the so-called degraded mode backups.
># However, if you specify a different value for the `reserve'
># parameter, amanda will not degrade backups if they will fit in the
># non-reserved portion of the holding disk.
>
># reserve 30 # percent
>
>
># This means save at least 30% of the holding disk space for degraded
># mode backups.
>
># Amanda needs a few Mb of diskspace for the log and debug files,
># as well as a database.  This stuff can grow large, so the conf
>directory
># isn't usually appropriate.  Some sites use /usr/local/var and
>some /usr/adm.
># Create an amanda directory under there.  You need a separate infofile
>and
># logdir for each configuration, so create subdirectories for each conf
>and
># put the files there.  Specify the locations below.
>
>infofile "/usr/local/var/lib/amanda/homes/curinfo"              # database
filename
>logdir   "/usr/local/var/lib/amanda/homes"                              # log 
>directory
>indexdir "/usr/local/var/lib/amanda/homes/index"                # index 
>directory
>#tapelist "/var/lib/amanda/debs/tapelist"               # list of used tapes
># tapelist is stored, by default, in the directory that contains
>amanda.conf
>
>
># tapetypes
>
># Define the type of tape you use here, and use it in "tapetype"
># above.  Some typical types of tapes are included here.  The tapetype
># tells amanda how many MB will fit on the tape, how big the filemarks
># are, and how fast the tape device is.
>
># A filemark is the amount of wasted space every time a tape section
># ends.  If you run `make tapetype' in tape-src, you'll get a program
># that generates tapetype entries, but it is slow as hell, use it only
># if you really must and, if you do, make sure you post the data to
># the amanda mailing list, so that others can use what you found out
># by searching the archives.
>
># For completeness Amanda should calculate the inter-record gaps too,
># but it doesn't.  For EXABYTE and DAT tapes this is ok.  Anyone using
># 9 tracks for amanda and need IRG calculations?  Drop me a note if
># so.
>
># If you want amanda to print postscript paper tape labels
># add a line after the comment in the tapetype of the form
>#    lbl-templ "/path/to/postscript/template/label.ps"
>
># if you want the label to go to a printer other than the default
># for your system, you can also add a line above for a different
># printer. (i usually add that line after the dumpuser specification)
>
># dumpuser "operator"     # the user to run dumps under
># printer "mypostscript"  # printer to print paper label on
>
># here is an example of my definition for an EXB-8500
>
># define tapetype EXB-8500 {
># ...
>#     lbl-templ "/usr/local/amanda/config/lbl.exabyte.ps"
># }
>
>
>define tapetype C1537A {
>    comment "HP C1537A Streamer - DDS-3 Tape"
>#    lbl-templ "/usr/local/var/lib/amanda/HP-DAT.ps"
>    length 11694 mbytes
>    filemark 78 kbytes
>    speed 830 kps
>}
>
>
># dumptypes
>#
># These are referred to by the disklist file.  The dumptype specifies
># certain parameters for dumping including:
>#   auth                - authentication scheme to use between server and 
>client.
>#                                 Valid values are "bsd" and "krb4".  Default: 
>[auth bsd]
>#   comment             - just a comment string
>#   comprate            - set default compression rate.  Should be followed by
>one or
>#                                 two numbers, optionally separated by a 
>comma.  The 1st is
>#                                 the full compression rate; the 2nd is the 
>incremental rate.
>#                                 If the second is omitted, it is assumed 
>equal to the first.
>#                                 The numbers represent the amount of the 
>original file the
>#                                 compressed file is expected to take up.
>#                                 Default: [comprate 0.50, 0.50]
>#   compress            - specify compression of the backed up data.  Valid
>values are:
>#                                 "none"        - don't compress the dump 
>output.
>#                                 "client best" - compress on the client using 
>the best (and
>#                                                                 probably 
>slowest) algorithm.
>#                                 "client fast" - compress on the client using 
>fast algorithm.
>#                                 "server best" - compress on the tape host 
>using the best (and
>#                                                                 probably 
>slowest) algorithm.
>#                                 "server fast" - compress on the tape host 
>using a fast
>#                                                                 algorithm.  
>This may be useful when a fast
>#                                                                 tape host is 
>backing up slow clients.
>#                                 Default: [compress client fast]
>#   dumpcycle           - set the number of days in the dump cycle, ie, set how
>often a
>#                                 full dump should be performed.  Default: 
>from DUMPCYCLE above
>#   exclude             - specify files and directories to be excluded from the
>dump.
>#                                 Useful with gnutar only; silently ignored by 
>dump and samba.
>#                                 Valid values are:
>#                                 "pattern"       - a shell glob pattern 
>defining which files
>#                                                                   to exclude.
>#                                                                   gnutar 
>gets --exclude="pattern"
>#                                 list "filename" - a file (on the client!) 
>containing patterns
>#                                                                   re's (1 
>per line) defining which files to
>#                                                                   exclude.
>#                                                                   gnutar 
>gets --exclude-from="filename"
>#                                 Note that the `full pathname' of a file 
>within its
>#                                 filesystem starts with `./', because of the 
>way amanda runs
>#                                 gnutar: `tar -C $mountpoint -cf - 
>--lots-of-options .' (note
>#                                 the final dot!)  Thus, if you're backing up 
>`/usr' with a
>#                                 diskfile entry like ``host /usr 
>gnutar-root', but you don't
>#                                 want to backup /usr/tmp, your exclude list 
>should contain
>#                                 the pattern `./tmp', as this is relative to 
>the `/usr' above.
>#                                 Please refer to the man-page of gnutar for 
>more information.
>#                                 Default: include all files
>#   holdingdisk                 - should the holding disk be used for this 
>dump.
>Useful for
>#                                 dumping the holding disk itself.  Default: 
>[holdingdisk yes]
>#   ignore              - do not back this filesystem up.  Useful for sharing a
>single
>#                                 disklist in several configurations.
>#   index               - keep an index of the files backed up.  Default: 
>[index
no]
>#   kencrypt            - encrypt the data stream between the client and 
>server.
>#                                 Default: [kencrypt no]
>#   maxdumps            - max number of concurrent dumps to run on the client.
>#                                 Default: [maxdumps 1]
>#   priority            - priority level of the dump.  Valid levels are "low",
>"medium"
>#                                 or "high".  These are really only used when 
>Amanda has no
>#                                 tape to write to because of some error.  In 
>that "degraded
>#                                 mode", as many incrementals as will fit on 
>the holding disk
>#                                 are done, higher priority first, to insure 
>the important
>#                                 disks are at least dumped.  Default: 
>[priority medium]
>#   program             - specify the dump system to use.  Valid values are
"DUMP"
>and
>#                                 "GNUTAR".  Default: [program "DUMP"].
>#   record              - record the dump in /etc/dumpdates.  Default: [record
yes]
>#   skip-full           - skip the disk when a level 0 is due, to allow full
>backups
>#                                 outside Amanda, eg when the machine is in 
>single-user mode.
>#   skip-incr           - skip the disk when the level 0 is NOT due.  This is
>used in
>#                                 archive configurations, where only full 
>dumps are done and
>#                                 the tapes saved.
>#   starttime           - delay the start of the dump?  Default: no delay
>#   strategy            - set the dump strategy.  Valid strategies are
currently:
>#                                 "standard" - the standard one.
>#                                 "nofull"   - do level 1 dumps every time.  
>This can be used,
>#                                                      for example, for small 
>root filesystems that
>#                                                      only change slightly 
>relative to a site-wide
>#                                                      prototype.  Amanda then 
>backs up just the
>#                                                      changes.
>#                                 "noinc"    - do level 0 dumps every time.
>#                                                      Unfortunately, this is 
>not currently
>#                                                      implemented.  Use 
>`dumpcycle 0'
>#                                                      instead.
>#                                 "skip"     - skip all dumps.  Useful for 
>sharing a single
>#                                                      disklist in several 
>configurations.
>#                                 Default: [strategy standard]
>#
># Note that you may specify previously defined dumptypes as a shorthand
>way
># of defining parameters.
>
>define dumptype global {
>    comment "Global definitions"
>    # This is quite useful for setting global parameters, so you don't
>have
>    # to type them everywhere.  All dumptype definitions in this sample
>file
>    # do include these definitions, either directly or indirectly.
>    # There's nothing special about the name `global'; if you create
any
>    # dumptype that does not contain the word `global' or the name of
>any
>    # other dumptype that contains it, these definitions won't apply.
>    # Note that these definitions may be overridden in other
>    # dumptypes, if the redefinitions appear *after* the `global'
>    # dumptype name.
>    # You may want to use this for globally enabling or disabling
>    # indexing, recording, etc.  Some examples:
>    index yes
>    # record no
>}
>
>define dumptype always-full {
>    global
>    comment "Full dump of this filesystem always"
>    compress none
>    priority high
>    dumpcycle 0
>}
>
>define dumptype root-tar {
>    global
>    program "GNUTAR"
>    comment "root partitions dumped with tar"
>    compress none
>    index
>    exclude list "/usr/local/lib/amanda/exclude.gtar"
>    priority low
>}
>
>define dumptype user-tar {
>    root-tar
>    comment "user partitions dumped with tar"
>    priority medium
>}
>
>define dumptype high-tar {
>    root-tar
>    comment "partitions dumped with tar"
>    priority high
>}
>
>define dumptype comp-root-tar {
>    root-tar
>    comment "Root partitions with compression"
>    compress client fast
>}
>
>define dumptype comp-user-tar {
>    user-tar
>    compress client fast
>}
>
>define dumptype holding-disk {
>    global
>    comment "The master-host holding disk itself"
>    holdingdisk no # do not use the holding disk
>    priority medium
>}
>
>define dumptype comp-user {
>    global
>    comment "Non-root partitions on reasonably fast machines"
>    compress client fast
>    priority medium
>}
>
>define dumptype nocomp-user {
>    comp-user
>    comment "Non-root partitions on slow machines"
>    compress none
>}
>
>define dumptype comp-root {
>    global
>    comment "Root partitions with compression"
>    compress client fast
>    priority low
>}
>
>define dumptype nocomp-root {
>    comp-root
>    comment "Root partitions without compression"
>    compress none
>}
>
>define dumptype comp-high {
>    global
>    comment "very important partitions on fast machines"
>    compress client best
>    priority high
>}
>
>define dumptype nocomp-high {
>    comp-high
>    comment "very important partitions on slow machines"
>    compress none
>}
>
>define dumptype nocomp-test {
>    global
>    comment "test dump without compression, no /etc/dumpdates
recording"
>    compress none
>    record no
>    priority medium
>}
>
>define dumptype comp-test {
>    nocomp-test
>    comment "test dump with compression, no /etc/dumpdates recording"
>    compress client fast
>}
>
>define dumptype debs {
>    global
>    program "GNUTAR"
>    index
>    comment "debs machine"
>    compress server fast
>}
>
># network interfaces
>#
># These are referred to by the disklist file.  They define the
>attributes
># of the network interface that the remote machine is accessed through.
># Notes: - netusage above defines the attributes that are used when the
>#          disklist entry doesn't specify otherwise.
>#        - the values below are only samples.
>#        - specifying an interface does not force the traffic to pass
>#          through that interface.  Your OS routing tables do that.
>This
>#          is just a mechanism to stop Amanda trashing your network.
># Attributes are:
>#               use                             - bandwidth above which amanda 
>won't start
>#                                                 backups using this 
>interface.  Note that if
>#                                                 a single backup will take 
>more than that,
>#                                                 amanda won't try to make it 
>run slower!
>
>define interface local {
>    comment "a local disk"
>    use 100000 kbps
>}
>
>define interface eth0 {
>    comment "100 Mbps ethernet"
>    use 100000 kbps
>}
>
># You may include other amanda configuration files, so you can share
># dumptypes, tapetypes and interface definitions among several
># configurations.
>
>#includefile "/usr/local/amanda.conf.main"
>
>-=-=-=-=-version-=-=-=-=-
>[amanda@dalston homes]$ amadmin homes version
>build: VERSION="Amanda-2.4.5"
>       BUILT_DATE="Tue Oct 25 21:06:05 EST 2005"
>       BUILT_MACH="Linux dalston.blackshaw.dyn.dhs.org 2.4.20-20.7 #1
>Mon Aug 18 14:42:08 EDT 2003 i586 unknown"
>       CC="gcc"
>       CONFIGURE_COMMAND="'./configure'
>'--with-gnutar=/usr/local/bin/amtar' '--with-user=amanda'
>'--with-group=disk'"
>paths: bindir="/usr/local/bin" sbindir="/usr/local/sbin"
>       libexecdir="/usr/local/libexec" mandir="/usr/local/man"
>       AMANDA_TMPDIR="/tmp/amanda" AMANDA_DBGDIR="/tmp/amanda"
>       CONFIG_DIR="/usr/local/etc/amanda" DEV_PREFIX="/dev/"
>       RDEV_PREFIX="/dev/" DUMP="/sbin/dump"
>       RESTORE="/sbin/restore" VDUMP=UNDEF VRESTORE=UNDEF
>       XFSDUMP=UNDEF XFSRESTORE=UNDEF VXDUMP=UNDEF VXRESTORE=UNDEF
>       SAMBA_CLIENT="/usr/bin/smbclient"
>       GNUTAR="/usr/local/bin/amtar" COMPRESS_PATH="/bin/gzip"
>       UNCOMPRESS_PATH="/bin/gzip" LPRCMD="/usr/bin/lpr"
>       MAILER="/usr/bin/Mail"
>       listed_incr_dir="/usr/local/var/amanda/gnutar-lists"
>defs:  DEFAULT_SERVER="dalston.blackshaw.dyn.dhs.org"
>       DEFAULT_CONFIG="DailySet1"
>       DEFAULT_TAPE_SERVER="dalston.blackshaw.dyn.dhs.org"
>       DEFAULT_TAPE_DEVICE="/dev/null" HAVE_MMAP HAVE_SYSVSHM
>       LOCKING=POSIX_FCNTL SETPGRP_VOID DEBUG_CODE
>       AMANDA_DEBUG_DAYS=4 BSD_SECURITY USE_AMANDAHOSTS
>       CLIENT_LOGIN="amanda" FORCE_USERID HAVE_GZIP
>       COMPRESS_SUFFIX=".gz" COMPRESS_FAST_OPT="--fast"
>       COMPRESS_BEST_OPT="--best" UNCOMPRESS_OPT="-dc"
>
>
>=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>Jeff Allison
>jeff.allison AT allygray.2y DOT net
>
>ICQ 8142658
>Messenger jeff_allison AT tokata.com DOT au
>Yahoo Jeff Allison
>Mobile +44 410 502 702

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.35% setiathome rank, not too shabby for a WV hillbilly
Free OpenDocument reader/writer/converter download:
http://www.openoffice.org
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.



<Prev in Thread] Current Thread [Next in Thread>