Amanda-Users

Re: backup just on holding disks makes many level 0

2006-03-15 07:07:56
Subject: Re: backup just on holding disks makes many level 0
From: Thomas Widhalm <widhalmt AT unix.sbg.ac DOT at>
To: sgw AT amanda DOT org
Date: Wed, 15 Mar 2006 13:03:51 +0100
On Tue, 2006-03-14 at 22:49 +0100, Stefan G. Weichinger wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Thomas Widhalm schrieb:
> 
> >> Have you tried resetting some parameters like bumpsize, bumpmult,
> >> and bumpdays?  They may have an affect on your situation.
> >>
> > This could really be it. bumpsize was set 100Mb. Maybe, because this
> > setup was used to backup on a tape just once in a week. Unfortunately I
> > had to take over the whole machine without having time to look through
> > the configuration too much. 
> 
> If that didn't solve your issues you maybe share your config with us.
> This would help to get the picture.

Here are the 2 parts of the config:

first file:
#
# amanda.conf - sample Amanda configuration file.  This started off life
as
#               the actual config file in use at CS.UMD.EDU.
#
# If your configuration is called, say, "csd", then this file normally
goes
# in /etc/amanda/csd/amanda.conf.
#

mailto "root"           # space separated list of operators at your site
dumpuser "amanda"       # the user to run dumps under

netusage  100000 Kbps   # maximum net bandwidth for Amanda, in KB per
sec

                        # 4 weeks (dumpcycle) times 5 tapes per week
(just
                        # the weekdays) plus a few to handle errors that
                        # need amflush and so we do not overwrite the
full
                        # backups performed at the beginning of the
previous
                        # cycle
maxdumps 1

### ### ###
# WARNING: don't use `inf' for tapecycle, it's broken!
### ### ###

bumpsize 10 Mb          # minimum savings (threshold) to bump level 1 ->
2
bumpdays 1              # minimum days at each level
bumpmult 4              # threshold = bumpsize * bumpmult^(level-1)

etimeout 1800           # number of seconds per filesystem for
estimates.
#etimeout -600          # total number of seconds for estimates.
# a positive number will be multiplied by the number of filesystems on
# each host; a negative number will be taken as an absolute total time-
out.
# The default is 5 minutes per filesystem.
                        

# Specify tape device and/or tape changer.  If you don't have a tape
# changer, and you don't want to use more than one tape per run of
# amdump, just comment out the definition of tpchanger.

# Some tape changers require tapedev to be defined; others will use
# their own tape device selection mechanism.  Some use a separate tape
# changer device (changerdev), others will simply ignore this
# parameter.  Some rely on a configuration file (changerfile) to
# obtain more information about tape devices, number of slots, etc;
# others just need to store some data in files, whose names will start
# with changerfile.  For more information about individual tape
# changers, read docs/TAPE.CHANGERS.

# At most one changerfile entry must be defined; select the most
# appropriate one for your configuration.  If you select man-changer,
# keep the first one; if you decide not to use a tape changer, you may
# comment them all out.

runtapes 1              # number of tapes to be used in a single run of
amdump
#tpchanger "chg-manual" # the tape-changer glue script
#tapedev "file:/data/amanda/"   # the no-rewind tape device to be used
#tapedev "no-such-device"       # the no-rewind tape device to be used
#tapedev "/dev/null"    # the no-rewind tape device to be used
#rawtapedev "/dev/null" # the raw device to be used (ftape only)
#rawtapedev "no-such-device"    # the raw device to be used (ftape only)
#changerfile "/var/lib/amanda/DailySet1/changer"
#changerfile "/var/lib/amanda/DailySet1/changer-status"
#changerfile "/etc/amanda/DailySet1/changer.conf"
#changerdev "/dev/null"
#changerdev "no-such-device"

#tapetype HP-DAT                # what kind of tape it is (see tapetypes
below)
#labelstr "^DailySet1[0-9][0-9]*$"      # label constraint regex: all
tapes must match

#tapetype CISOS-notape
#labelstr "^ONYX[0-9][0-9]*$"   # label constraint regex: all tapes must
match

# Specify holding disks.  These are used as a temporary staging area for
# dumps before they are written to tape and are recommended for most
sites.
# The advantages include: tape drive is more likely to operate in
streaming
# mode (which reduces tape and drive wear, reduces total dump time);
multiple
# dumps can be done in parallel (which can dramatically reduce total
dump time.
# The main disadvantage is that dumps on the holding disk need to be
flushed
# (with amflush) to tape after an operating system crash or a tape
failure.
# If no holding disks are specified then all dumps will be written
directly
# to tape.  If a dump is too big to fit on the holding disk than it will
be
# written directly to tape.  If more than one holding disk is specified
then
# they will all be used round-robin.

#holdingdisk hd2 {
#    directory "/dumps2/amanda"
#    use 1000 Mb
#    }
#holdingdisk hd3 {
#    directory "/mnt/disk4"
#    use 1000 Mb
#    }


# If amanda cannot find a tape on which to store backups, it will run
# as many backups as it can to the holding disks.  In order to save
# space for unattended backups, by default, amanda will only perform
# incremental backups in this case, i.e., it will reserve 100% of the
# holding disk space for the so-called degraded mode backups.
# However, if you specify a different value for the `reserve'
# parameter, amanda will not degrade backups if they will fit in the
# non-reserved portion of the holding disk.

# reserve 30 # percent
reserve 0

# This means save at least 30% of the holding disk space for degraded
# mode backups.  

# Amanda needs a few Mb of diskspace for the log and debug files,
# as well as a database.  This stuff can grow large, so the conf
directory
# isn't usually appropriate.  Some sites use /usr/local/var and
some /usr/adm.
# Create an amanda directory under there.  You need a separate infofile
and
# logdir for each configuration, so create subdirectories for each conf
and
# put the files there.  Specify the locations below.

#tapelist "/var/lib/amanda/DailySet1/tapelist"  # list of used tapes
# tapelist is stored, by default, in the directory that contains
amanda.conf


# tapetypes

# Define the type of tape you use here, and use it in "tapetype"
# above.  Some typical types of tapes are included here.  The tapetype
# tells amanda how many MB will fit on the tape, how big the filemarks
# are, and how fast the tape device is.

# A filemark is the amount of wasted space every time a tape section
# ends.  If you run `make tapetype' in tape-src, you'll get a program
# that generates tapetype entries, but it is slow as hell, use it only
# if you really must and, if you do, make sure you post the data to
# the amanda mailing list, so that others can use what you found out
# by searching the archives.

# For completeness Amanda should calculate the inter-record gaps too,
# but it doesn't.  For EXABYTE and DAT tapes this is ok.  Anyone using
# 9 tracks for amanda and need IRG calculations?  Drop me a note if
# so.

# If you want amanda to print postscript paper tape labels
# add a line after the comment in the tapetype of the form
#    lbl-templ "/path/to/postscript/template/label.ps"

# if you want the label to go to a printer other than the default
# for your system, you can also add a line above for a different
# printer. (i usually add that line after the dumpuser specification)

# dumpuser "operator"     # the user to run dumps under
# printer "mypostscript"  # printer to print paper label on

# here is an example of my definition for an EXB-8500

# define tapetype EXB-8500 {
# ...
#     lbl-templ "/usr/local/amanda/config/lbl.exabyte.ps"
# }

define tapetype Quantum-DLT7000 {
  comment "Quantum DLT7000 (hardware compression on: Xmn)"
  length 29206 mbytes
  filemark 30 kbytes
  speed 1670 kps
}

define tapetype DEC-TZ89 {
  comment "DEC TZ89 (hardware compression on: Xmn)"
  length 29414 mbytes
  filemark 24 kbytes
  speed 1634 kps
}

define tapetype CISOS-notape {
    comment "ONYX dummy tape"
    length 1000 gbytes
    filemark 0 kbytes         # don't know a better value
    speed 6000 kbytes            # dito
}

define tapetype QIC-60 {
    comment "Archive Viper"
    length 60 mbytes
    filemark 100 kbytes         # don't know a better value
    speed 100 kbytes            # dito
}

define tapetype DEC-DLT2000 {
    comment "DEC Differential Digital Linear Tape 2000"
    length 15000 mbytes
    filemark 8 kbytes
    speed 1250 kbytes
}

# goluboff AT butch.Colorado DOT EDU
# in amanda-users (Thu Dec 26 01:55:38 MEZ 1996)
define tapetype DLT {
    comment "DLT tape drives"
    length 20000 mbytes         # 20 Gig tapes
    filemark 2000 kbytes        # I don't know what this means
    speed 1536 kbytes           # 1.5 Mb/s
}

define tapetype SURESTORE-1200E {
    comment "HP AutoLoader"
    length 3900 mbytes
    filemark 100 kbytes
    speed 500 kbytes
}

define tapetype EXB-8500 {
    comment "Exabyte EXB-8500 drive on decent machine"
    length 4200 mbytes
    filemark 48 kbytes
    speed 474 kbytes                    
}

define tapetype EXB-8200 {
    comment "Exabyte EXB-8200 drive on decent machine"
    length 2200 mbytes
    filemark 2130 kbytes
    speed 240 kbytes                    
}

define tapetype HP-DAT {
    comment "DAT tape drives"
    # data provided by Rob Browning <rlb AT cs.utexas DOT edu>
    length 1930 mbytes
    filemark 111 kbytes
    speed 468 kbytes
}

define tapetype DAT {
    comment "DAT tape drives"
    length 1000 mbytes          # these numbers are not accurate
    filemark 100 kbytes         # but you get the idea
    speed 100 kbytes
}

define tapetype MIMSY-MEGATAPE {
    comment "Megatape (Exabyte based) drive through Emulex on Vax 8600"
    length 2200 mbytes
    filemark 2130 kbytes
    speed 170 kbytes            # limited by the Emulex bus interface,
ugh
}


# dumptypes
#
# These are referred to by the disklist file.  The dumptype specifies
# certain parameters for dumping including:
#   auth        - authentication scheme to use between server and
client.
#                 Valid values are "bsd" and "krb4".  Default: [auth
bsd]
#   comment     - just a comment string
#   comprate    - set default compression rate.  Should be followed by
one or
#                 two numbers, optionally separated by a comma.  The 1st
is
#                 the full compression rate; the 2nd is the incremental
rate.
#                 If the second is omitted, it is assumed equal to the
first.
#                 The numbers represent the amount of the original file
the
#                 compressed file is expected to take up.
#                 Default: [comprate 0.50, 0.50]
#   compress    - specify compression of the backed up data.  Valid
values are:
#                 "none"        - don't compress the dump output.
#                 "client best" - compress on the client using the best
(and
#                                 probably slowest) algorithm.
#                 "client fast" - compress on the client using fast
algorithm.
#                 "server best" - compress on the tape host using the
best (and
#                                 probably slowest) algorithm.
#                 "server fast" - compress on the tape host using a fast
#                                 algorithm.  This may be useful when a
fast
#                                 tape host is backing up slow clients.
#                 Default: [compress client fast]
#   dumpcycle   - set the number of days in the dump cycle, ie, set how
often a
#                 full dump should be performed.  Default: from
DUMPCYCLE above
#   exclude     - specify files and directories to be excluded from the
dump.
#                 Useful with gnutar only; silently ignored by dump and
samba.
#                 Valid values are:
#                 "pattern"       - a shell glob pattern defining which
files
#                                   to exclude.
#                                   gnutar gets --exclude="pattern"
#                 list "filename" - a file (on the client!) containing
patterns
#                                   re's (1 per line) defining which
files to
#                                   exclude.
#                                   gnutar gets --exclude-
from="filename"
#                 Note that the `full pathname' of a file within its
#                 filesystem starts with `./', because of the way amanda
runs
#                 gnutar: `tar -C $mountpoint -cf - --lots-of-
options .' (note
#                 the final dot!)  Thus, if you're backing up `/usr'
with a
#                 diskfile entry like ``host /usr gnutar-root', but you
don't
#                 want to backup /usr/tmp, your exclude list should
contain
#                 the pattern `./tmp', as this is relative to the `/usr'
above.
#                 Please refer to the man-page of gnutar for more
information.
#                 Default: include all files
#   holdingdisk - should the holding disk be used for this dump.  Useful
for
#                 dumping the holding disk itself.  Default:
[holdingdisk yes]
#   ignore      - do not back this filesystem up.  Useful for sharing a
single
#                 disklist in several configurations.
#   index       - keep an index of the files backed up.  Default: [index
no]
#   kencrypt    - encrypt the data stream between the client and server.
#                 Default: [kencrypt no]
#   maxdumps    - max number of concurrent dumps to run on the client.
#                 Default: [maxdumps 1]
#   priority    - priority level of the dump.  Valid levels are "low",
"medium"
#                 or "high".  These are really only used when Amanda has
no
#                 tape to write to because of some error.  In that
"degraded
#                 mode", as many incrementals as will fit on the holding
disk
#                 are done, higher priority first, to insure the
important
#                 disks are at least dumped.  Default: [priority medium]
#   program     - specify the dump system to use.  Valid values are
"DUMP" and
#                 "GNUTAR".  Default: [program "DUMP"].
#   record      - record the dump in /etc/dumpdates.  Default: [record
yes]
#   skip-full   - skip the disk when a level 0 is due, to allow full
backups
#                 outside Amanda, eg when the machine is in single-user
mode.
#   skip-incr   - skip the disk when the level 0 is NOT due.  This is
used in
#                 archive configurations, where only full dumps are done
and
#                 the tapes saved.
#   starttime   - delay the start of the dump?  Default: no delay
#   strategy    - set the dump strategy.  Valid strategies are
currently:
#                 "standard" - the standard one.
#                 "nofull"   - do level 1 dumps every time.  This can be
used,
#                              for example, for small root filesystems
that
#                              only change slightly relative to a site-
wide
#                              prototype.  Amanda then backs up just the
#                              changes.
#                 "noinc"    - do level 0 dumps every time.
#                              Unfortunately, this is not currently
#                              implemented.  Use `dumpcycle 0'
#                              instead.
#                 "skip"     - skip all dumps.  Useful for sharing a
single
#                              disklist in several configurations.
#                 Default: [strategy standard]
#
# Note that you may specify previously defined dumptypes as a shorthand
way
# of defining parameters.

####
#### BEGIN USEFUL DUMPTYPES
####

define dumptype unix {
    comment "Global definitions UNIX LINUX"
    index yes
    #exclude list ".amanda.excludes"
}

define dumptype unix-tar {
    unix
    program "GNUTAR"
}

define dumptype unix-tar-d0-c0-t1 {
    unix-tar
    comment "partitions full dumped with tar not zipped"
    compress none
    dumpcycle 0
}

define dumptype unix-tar-d1-c0-t1 {
    unix-tar
    comment "partitions incr dumped with tar not zipped"
    compress none
    strategy incronly 
}

define dumptype unix-tar-d0-c1-t1 {
    unix-tar
    comment "partitions full dumped with tar zipped"
    compress client fast
    # dumpcycle 0 #changed by widhalmt 20060103
}

define dumptype unix-tar-d1-c1-t1 {
    unix-tar
    comment "partitions incr dumped with tar zipped"
    #compress client fast
    compress client best #changed by widhalmt 20060310
    # strategy incronly #changed by widhalmt 20060103
}

define dumptype unix-dump {
    unix
}

define dumptype unix-dump-d0-c0-t1 {
    unix-dump
    comment "partitions full dumped"
    compress none
    #dumpcycle 0 #changed by widhalmt 20060111
}

define dumptype unix-dump-d1-c0-t1 {
    unix-dump
    comment "partitions incr dumped"
    compress none
    strategy incronly 
}

define dumptype unix-dump-d0-c1-t1 {
    unix-dump
    comment "partitions full dumped"
    compress none
    #dumpcycle 0 #changed by widhalmt 20060111
}

define dumptype unix-dump-d1-c1-t1 {
    unix-dump
    comment "partitions incr dumped"
    compress none
    strategy incronly 
}

####
#### END USEFUL DUMPTYPES
####

define dumptype linux-holding-disk {
    unix-tar
    comment "The master-host holding disk itself"
    holdingdisk no # do not use the holding disk
    priority medium
}

define dumptype irix {
    comment "Global definitions IRIX"
    exclude list optional ".amanda_no"
}

define dumptype irix-root {
    irix
    comment "irix root partitions"
    compress none
    index
    priority low
}

define dumptype irix-high-tar {
    irix-root
    comment "irix partitions dumped with tar"
    program "GNUTAR"
    compress client fast
    priority high
}

define dumptype global {
    comment "Global definitions"
    # This is quite useful for setting global parameters, so you don't
have
    # to type them everywhere.  All dumptype definitions in this sample
file
    # do include these definitions, either directly or indirectly.
    # There's nothing special about the name `global'; if you create any
    # dumptype that does not contain the word `global' or the name of
any
    # other dumptype that contains it, these definitions won't apply.
    # Note that these definitions may be overridden in other
    # dumptypes, if the redefinitions appear *after* the `global'
    # dumptype name.
    # You may want to use this for globally enabling or disabling
    # indexing, recording, etc.  Some examples:
    # index yes
    # record no
}

define dumptype always-full {
    global
    comment "Full dump of this filesystem always"
    compress none
    priority high
    dumpcycle 0
}

define dumptype root-tar {
    global
    program "GNUTAR"
    comment "root partitions dumped with tar"
    compress none
    index
    exclude list optional ".amanda_no"
    priority low
}

define dumptype user-tar {
    root-tar
    comment "user partitions dumped with tar"
    priority medium
}

define dumptype high-tar {
    root-tar
    comment "partitions dumped with tar"
    priority high
}

define dumptype comp-root-tar {
    root-tar
    comment "Root partitions with compression"
    compress client fast
}

define dumptype comp-user-tar {
    user-tar
    compress client fast
}

define dumptype holding-disk {
    global
    comment "The master-host holding disk itself"
    holdingdisk no # do not use the holding disk
    priority medium
}

define dumptype comp-user {
    global
    comment "Non-root partitions on reasonably fast machines"
    compress client fast
    priority medium
}

define dumptype nocomp-user {
    comp-user
    comment "Non-root partitions on slow machines"
    compress none
}

define dumptype comp-root {
    global
    comment "Root partitions with compression"
    compress client fast
    priority low
}

define dumptype nocomp-root {
    comp-root
    comment "Root partitions without compression"
    compress none
}

define dumptype comp-high {
    global
    comment "very important partitions on fast machines"
    compress client best
    priority high
}

define dumptype nocomp-high {
    comp-high
    comment "very important partitions on slow machines"
    compress none
}

define dumptype nocomp-test {
    global
    comment "test dump without compression, no /etc/dumpdates recording"
    compress none
    record no
    priority medium
}

define dumptype comp-test {
    nocomp-test
    comment "test dump with compression, no /etc/dumpdates recording"
    compress client fast
}

# network interfaces
#
# These are referred to by the disklist file.  They define the
attributes
# of the network interface that the remote machine is accessed through.
# Notes: - netusage above defines the attributes that are used when the
#          disklist entry doesn't specify otherwise.
#        - the values below are only samples.
#        - specifying an interface does not force the traffic to pass
#          through that interface.  Your OS routing tables do that.
This
#          is just a mechanism to stop Amanda trashing your network.
# Attributes are:
#       use             - bandwidth above which amanda won't start
#                         backups using this interface.  Note that if
#                         a single backup will take more than that,
#                         amanda won't try to make it run slower!

define interface local {
    comment "a local disk"
    use 10000 kbps
}

define interface hme0 {
    comment "100 Mbps ethernet"
    use 100000 kbps
}

# You may include other amanda configuration files, so you can share
# dumptypes, tapetypes and interface definitions among several
# configurations.

#includefile "/usr/local/amanda.conf.main"


and the other:

org "IS"     # your organization name for reports

inparallel 1           # maximum dumpers that will run in parallel

dumpcycle 8             # the number of days in the normal dump cycle
runspercycle 0          # the number of amdump runs in dumpcycle days
tapecycle 4 tapes       # the number of tapes in rotation
                        # 4 weeks (dumpcycle) times 5 tapes per week
(just
                        # the weekdays) plus a few to handle errors that
                        # need amflush and so we do not overwrite the
full
                        # backups performed at the beginning of the
previous
                        # cycle

holdingdisk hd1 {
    comment "main holding disk for IS"
    directory "/data/amanda/IS"        # where the holding disk is
    use -1              # how much space can we use on it
#    use 70 Mb           # how much space can we use on it
                        # a negative value mean:
                        #        use all space except that value
    chunksize 2 Gb      # size of chunk if you want big dump to be
                        # dumped on multiple files on holding disks
                        #  N Kb/Mb/Gb split disks in chunks of size N
                        #  0          split disks in INT_MAX/1024 Kb
chunks
                        # -N Kb/Mb/Gb dont split, dump larger
                        #             filesystems directly to tape
                        #             (example: -2 Gb)
    }


#tapedev "/dev/rmt/0mn"        # the no-rewind tape device to be used
#da 2.band defekt... #tapedev "/dev/nst1"        # the no-rewind tape
device to be used
#tapedev "/dev/nst0"        # the no-rewind tape device to be used
tapedev "/dev/null"
tapetype DEC-TZ89             # see tapetype in amanda-shared.conf
labelstr "^IS[0-9][0-9]*$"    # label constraint regex: all tapes must
match

infofile "/var/lib/amanda/IS/curinfo"        # database filename
logdir   "/var/lib/amanda/IS"                # log directory
indexdir "/var/lib/amanda/IS/index"  # index directory

includefile "/etc/amanda/amanda-shared.conf"



the first file is used by all configs. The second one is one specific
config.

> 
> You mentioned you had to take over that server:
> Do you have any experience with Amanda already or not?

My only experience with amanda comes from experimenting with this
server.

Thanks,
Thomas

> 
> Stefan.
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iD8DBQFEFzp60mmG+9CBTHYRAutNAJsG9BKBIqf15ZM2+9qaVYjEuefiEgCgi0R5
> JEuVORm5JOWnCiwj7isFsCo=
> =Odui
> -----END PGP SIGNATURE-----
-- 

*****************************************************************
* Thomas Widhalm                             Unix Administrator *
* University of Salzburg                       ITServices (ITS) *
* Systems Management                               Unix Systems *
* Hellbrunnerstr. 34                     5020 Salzburg, Austria *
* widhalmt AT unix.sbg.ac DOT at                     +43/662/8044-6774 *
* gpg: 6265BAE6                                                 *
* http://www.sbg.ac.at/zid/organisation/mitarbeiter/widhalm.htm *
*****************************************************************



Attachment: signature.asc
Description: This is a digitally signed message part

<Prev in Thread] Current Thread [Next in Thread>