Amanda-Users

77 hour backup of 850gb?

2006-06-29 19:54:23
Subject: 77 hour backup of 850gb?
From: "Paul Graf" <grafpm AT hotmail DOT com>
To: amanda-users AT amanda DOT org
Date: Thu, 29 Jun 2006 18:48:33 -0400
I've got a strange problem with my backups. I'm running Amanda 2.5 on FC4, and the problem lies with one large directory that needs to be backed up. It's over 700 gigs, so it has to span tapes. Eventually the backup will complete, but I'm getting an average of 3 MB/s. However, if I back up something smaller (I have a few directories being backed up that are around 100 megs), I get 25 MB/s. I also did some test tar backups of a few gigs, and those went at around 25 MB/s as well.

I do have an idea about what is causing the problem, but I'd like to get some input before I try to adjust anything. My tape library is a Dell LTO-2 unit (200GB uncompressed), and going by the amanda.conf comments, I set the chunk size to 20GB. I have a feeling that this is conflicted by my holding disk space, which is also 20GB. Would increasing the holding disk space or decreasing the chunk size be helpful? Here's my amanda.conf:

org "DailyBackup"     # your organization name for reports

mailto "root"         # space separated list of operators at your site
dumpuser "amandabackup"       # the user to run dumps under

inparallel 4            # maximum dumpers that will run in parallel (max 63)
                        # this maximum can be increased at compile-time,
                        # modifying MAX_DUMPERS in server-src/driverio.h
dumporder "sssS"      # specify the priority order of each dumper
                        #   s -> smallest size
                        #   S -> biggest size
                        #   t -> smallest time
                        #   T -> biggest time
                        #   b -> smallest bandwitdh
                        #   B -> biggest bandwitdh
                        # try "BTBTBTBTBTBT" if you are not holding
                        # disk constrained

taperalgo first         # The algorithm used to choose which dump image to send
                        # to the taper.

                        # Possible values: 
[first|firstfit|largest|largestfit|smallest|last]
                        # Default: first.

                        # first         First in - first out.
                        # firstfit      The first dump image that will fit on 
the current tape.
                        # largest       The largest dump image.
                        # largestfit    The largest dump image that will fit on 
the current tape.
                        # smallest      The smallest dump image.
                        # last          Last in - first out.

displayunit "k"               # Possible values: "k|m|g|t"
                        # Default: k.
                        # The unit used to print many numbers.
                        # k=kilo, m=mega, g=giga, t=tera

netusage  6000 Kbps     # maximum net bandwidth for Amanda, in KB per sec

dumpcycle 4 weeks       # the number of days in the normal dump cycle
runspercycle 7 days     # the number of amdump runs in dumpcycle days
                        # (4 weeks * 5 amdump runs per week -- just weekdays)
tapecycle 30 tapes      # the number of tapes in rotation
                        # 4 weeks (dumpcycle) times 5 tapes per week (just
                        # the weekdays) plus a few to handle errors that
                        # need amflush and so we do not overwrite the full
                        # backups performed at the beginning of the previous
                        # cycle

bumpsize 20 Mb          # minimum savings (threshold) to bump level 1 -> 2
bumppercent 20          # minimum savings (threshold) to bump level 1 -> 2
bumpdays 1              # minimum days at each level
bumpmult 4              # threshold = bumpsize * bumpmult^(level-1)


etimeout 300            # number of seconds per filesystem for estimates.
#etimeout -600          # total number of seconds for estimates.
# a positive number will be multiplied by the number of filesystems on
# each host; a negative number will be taken as an absolute total time-out.
# The default is 5 minutes per filesystem.

dtimeout 1800           # number of idle seconds before a dump is aborted.

ctimeout 90             # maximum number of seconds that amcheck waits
                        # for each client host

tapebufs 20

runtapes 7              # number of tapes to be used in a single run of amdump
tpchanger "chg-zd-mtx"        # the tape-changer glue script
tapedev "/dev/nst0"   # the no-rewind tape device to be used
#rawtapedev "/dev/null"       # the raw device to be used (ftape only)
changerfile "/etc/amanda/daily/changer"
#changerfile "/etc/amanda/DailySet1/changer-status"
changerdev "/dev/sg7"

maxdumpsize -1          # Maximum number of bytes the planner will schedule
                        # for a run (default: runtapes * tape_length).
tapetype LTO2           # what kind of tape it is (see tapetypes below)
labelstr "^[0-9][0-9][0-9][0-9][0-9][0-9]*$" # label constraint regex: all tapes must match

amrecover_do_fsf yes            # amrecover will call amrestore with the
                                # -f flag for faster positioning of the tape.
amrecover_check_label yes       # amrecover will call amrestore with the
                                # -l flag to check the label.
amrecover_changer "null:"     # amrecover will use the changer if you restore
                                # from this device.
                                # It could be a string like 'changer' and
                                # amrecover will use your changer if you
                                # set your tape with 'settape changer'

holdingdisk hd1 {
   comment "main holding disk"
   directory "/tmp/amandahld" # where the holding disk is
   use 20000 Mb         # how much space can we use on it
                        # a non-positive value means:
                        #        use all space but that value
   chunksize 1Gb        # size of chunk if you want big dump to be
                        # dumped on multiple files on holding disks
                        #  N Kb/Mb/Gb split images in chunks of size N
                        #             The maximum value should be
                        #             (MAX_FILE_SIZE - 1Mb)
                        #  0          same as INT_MAX bytes
   }

autoflush no #

infofile "/etc/amanda/daily/curinfo"  # database DIRECTORY
logdir   "/etc/amanda/daily"          # log directory
indexdir "/etc/amanda/daily/index"    # index directory
#tapelist "@CONFIG_DIR/DailySet1/tapelist"    # list of used tapes
# tapelist is stored, by default, in the directory that contains amanda.conf

# tapetypes

define tapetype LTO2 {
   comment "LTO2/Ultrium tapes"
   length 208277 mbytes
   filemark 1049 kbytes
   speed 2339 kps
}

# dumptypes

define dumptype global {
   comment "Global definitions"
   tape_splitsize 20000 mbytes
   split_diskbuffer "/tmp/amandasplit/split"
   index yes
   holdingdisk yes
}

define dumptype normal {
   global
   compress none
   comment "Typical dump"
   program "GNUTAR"
   priority medium
   strategy standard
}

# network interfaces

define interface local {
   comment "a local disk"
   use 6000 kbps
}

define interface le0 {
   comment "100 Mbps ethernet"
   use 4000 kbps
}

I also have the mail that Amanda sent after the backup was done, if anyone wants to see it. Any help would be greatly appreciated :)

-Paul