After struggling around about a week with amanda to get it working in a
tapless network backup configuration it finally seems to be doing as it
should.
I've put together a little How-To to help others (as well as myself)
with the setup of amanda for tapeless operation. It got a bit lengthy I
fear but I think it is rather comprehensive. I am posting it here, at
the amanda-users mailing list, hoping that it will be archived for
further reference.
Please feel free to mail me about errors, mishaps, spelling or grammar
mistakes as well as suggestions for extension or corrections if you
want.
I have written this by myself, with help of the following How-To in the
Amanda Faq-o-matic archive (borrowed a bit from the structure and a
little from contents): http://amanda.sourceforge.net/fom-serve/cache/191.html
So here it is:
Amanda tapeless network backup operation
Institut fuer Nachrichtentechnik und Hochfrequenztechnik
Technische Universitaet Wien (Vienna University of Technology)
Vienna, Austria
Author:
Andreas Ntaflos
ant at overclockers in the dot at domain
Date:
2003-07-30
Contents
========
1 Assumptions/Requirements
2 Installation
3 Backup configuration
4 Restoring
5 Problems
This HOWTO covers amanda 2.4.4p1.
- - - - - - - - - - - - - - - - -
1 Assumptions/Requirements
==========================
It's helpful/mandatory to
.) know what amanda is and what it is used for,
.) know UNIX/Linux/shit,
.) know where to find config files,
.) have a thought-through backup strategy prepared,
.) etc...
The amanda backup configuration is called 'normal'.
The backup server (which generates and stores the backups) is called 'backup'.
The backup client (which holds the important data to be backed up) is called
'mobilkom'.
There are no tapes. Instead, 'backup' uses a 600GB raid-array of IDE disks.
'backup' and 'mobilkom' are connected via a 100Mbit CAT5 ethernet cable. Wow.
'mobilkom' is accessible from the outside, 'backup' only via 'mobilkom'
(mobilkom has two ethernet NICs).
Configuration files for amanda are on 'backup', located under
/usr/local/etc/amanda/normal.
The files are called amanda.conf, disklist, and changer.conf.
The backup strategy (subject to change, I believe?)
---------------------------------------------------
7 day dumpcycle, rotating 7 tapes, one for each day. Backups start at Sunday
night with a level 0 dump of the data. Each day the backup level gets incre-
mented by one.
'mobilkom' has a 400GB raid5 array on device /dev/md0, mounted on /d which
holds the data to be backed up.
'backup's 600GB raid array (the 'tapes') is also /dev/md0 (but on 'backup')
and mounted on /dump, where a subdirectory for each tape exists, according to
the configuration ('normal' in this case)
'backup' also has a mount point /d which is an NFS export share from mobilkom
of the data to be backed up (used for restoring, see 'Configuration' and
'Problems').
2 Installation
==============
The installation of amanda involves both client and server. The INSTALL
and docs/INSTALL files coming with the amanda distribution describes in detail
how amanda can be configured at build-time and what options it provides.
Server installation
-------------------
Here, the following options have been set:
--with-user=amanda
--with-group=disk
--with-config=normal
According to the INSTALL files, amanda can be built and installed thus (as
root):
# ./configure [OPTIONS]
# make
# make install
This will add a new user amanda to the system, with it's home directory in
/var/lib/amanda, which also serves as the state directory. There all the log
files will be put as well as the indexing database for use with amrecover.
It varies from system to system where the binaries are to installed to, but
common places for amanda are:
/usr/bin
/usr/sbin
/usr/local/bin
/usr/local/sbin
/usr/local/libexec
The binaries for the server are the following (possibly incomplete list):
amadmin
amcheck
amcheckdb
amcleanup
amdd
amdump
amflush
amgetconf
amlabel
ammt
amoverview
amplot
amrecover
amreport
amrestore
amrmtape
amstatus
amtape
amtapetype
amtoc
amverify
amverifyrun
Server configuration
--------------------
1) The amanda services have to be put in the /etc/services file:
amanda 10080/udp
amandaidx 10082/tcp
amidxtape 10083/tcp
2) The indexing capabilities of amanda are also needed, so the services have
to be started with inetd. The underlying systems use xinetd, so the config-
uration will look as follows:
/etc/xinetd.d/amandaidx:
service amandaidx
{
socket_type = stream
protocol = tcp
wait = no
user = amanda
group = disk
server = /usr/local/libexec/amindexd
disable = no
}
/etc/xinted.d/amidxtape:
service amidxtape
{
socket_type = stream
protocol = tcp
wait = no
user = amanda
group = disk
server = /usr/local/libexec/amidxtaped
disable = no
}
3) In order for the client to access the amanda services, they have to have
the permissions to do so; they are controlled by the .amandahosts file in
the home directory of amanda (e.g. /var/lib/amanda):
.amandahosts:
localhost amanda
localhost.localdomain amanda
mobilkom root
backup root
4) A cronjob is needed for amanda to execute the backup procedures on a
regular basis. amdump is used for the dumping of files and filesystems. It
only takes the configuration name as an argument. The cronjob must be done by
user amanda. As amanda the following command will create a new cronjob,
running every day of the week at 23:00:
$ crontab -e
SHELL=/bin/sh
00 23 * * * /usr/local/sbin/amdump normal
Client installation
-------------------
The extracted distribution of amanda contains a directory called client-src.
Cd'ing there and issuing make and make install should take care of compiling
and installing the necessary client binaries. The default paths are
/usr/local/bin and /usr/local/libexec.
Client configuration
--------------------
1) The client needs to know about the services running, too. The following
entries are to be added to /etc/services:
amanda 10080/udp
amandaidx 10082/tcp
amidxtape 10083/tcp
2) On the client the only service that needs to run is the amandad daemon. It
is located unter /usr/local/libexec/amandad. Starting it via inetd:
/etc/xinetd.d/amanda:
service amanda
{
socket_type = dgram
protocol = udp
user = amanda
group = disk
wait = yes
server = /usr/local/libexec/amandad
disable = no
}
3) Authentication on the client is also needed, so the backup server can
connect to it. This is done via the .amandahosts file in ~amanda, too.
.amandahosts:
backup
mobilkom
3 Backup configuration
======================
The three configuration files for the correct backup operation are the
aforementioned 'amanda.conf', 'disklist' and 'changer.conf'. They reside on
the backup server, under /usr/local/etc/amanda/normal in case of the
present configuration of 'backup' and 'mobilkom'. The following describes
what lines are to be edited or added to either configuration file.
amanda.conf
-----------
org "normal" # your organization name for reports
mailto "amanda" # space separated list of operators at your site
dumpuser "amanda" # the user to run dumps under
netusage 600 kbps # maximum net bandwidth for Amanda, in KB per sec
dumpcycle 7 days # the number of days in the normal dump cycle
runspercycle 7 # the number of amdump runs in dumpcycle days
tapecycle 7 # the number of tapes in rotation
# This enables a backup cycle of one week, backups on every day, with one tape
# per day.
runtapes 1 # number of tapes to be used in a single run of amdump
tpchanger "chg-multi" # the tape-changer glue script
tapedev "file:/dump/tape" # the no-rewind tape device to be used
# see later in this document
tapetype HARD-DISK # what kind of tape it is; this is defined later in
# the config file
labelstr "^normal[0-9][0-9]*$" # label constraint regex: all tapes must match
# This is the pattern which every tapelabel must match; in case of backup and
# mobilkom this would be normal01, normal02, ..., normal99.
infofile "/var/lib/amanda/normal/curinfo" # database filename
logdir "/var/lib/amanda/normal" # log directory
indexdir "/var/lib/amanda/normal/index" # index directory
tapelist "/var/lib/amanda/normal/tapelist" # list of used tapes
define tapetype HARD-DISK {
comment "Hard disk instead of tape"
length 601 gbytes
}
# This defines a tapetype HARD-DISK with a 'length' of 601GB, the size of the
# backup array.
define dumptype hard-disk-dump {
global
comment "Back up to hard disk instead of tape using dump"
holdingdisk no
index yes
priority high
kencrypt no
}
define dumptype hard-disk-tar {
comment "Back up to hard disk instead of tape - using tar"
kencrypt no
program "GNUTAR"
priority high
index yes
}
# Defining a real dump of the filesystem and tar-ing the contents of the
# directory in question. tar runs faster and more platform and more system
# indepentent (when using the GNU version of tar, available for virtually
# every existing UNIX operating system) than dump.
define interface local {
comment "a local disk"
use 1000 kbps
}
define interface eth0 {
comment "10 Mbps ethernet"
use 600 kbps
}
# Network interface definitions.
changer.conf
------------
multieject 0
gravity 0
needeject 0
ejectdelay 0
statefile /var/lib/amanda/normal/changer-status
firstslot 1
lastslot 7
slot 1 file:/dump/normal01/
slot 2 file:/dump/normal02/
slot 3 file:/dump/normal03/
slot 4 file:/dump/normal04/
slot 5 file:/dump/normal05/
slot 6 file:/dump/normal06/
slot 7 file:/dump/normal07/
This tells amanda that a tape changer is used; it has 7 slots and each slot
bears a tape labeled normal01 through normal07. Full dumps are thus supposed
to go on normal01.
disklist
--------
mobilkom md0 hard-disk-tar
The disklist defines which host is backed up, which disk on the host is used
and what kind of backup should be made.
Tape setup
----------
For this to work the directories /dump/normal* have to be created (by the
amanda user) and the 'tapes' have to be labelled accordingly.
Creating the 'tapes' on the disk:
# mkdir -p /dump/normal01
# mkdir -p /dump/normal02
.
.
# mkdir -p /dump/normal07
Labelling the tapes:
# amlabel normal normal01 slot 1
# amlabel normal normal02 slot 2
.
.
# amlabel normal normal07 slot 7
For restoring, the current tape is always referred to by /dump/tape; a
symbolic link to the currently loaded tape (e.g. normal02) ensures this.
/dump/tape has been specified in amanda.conf above.
# ln -s /dump/normal02 /dump/tape
This should be changed whenever a restore program like amrecover requires
another or the next tape to be put into the drive. The tapes are always
referred to by label.
4 Restoring
===========
The way to restore data backed up on the 'tapes' depends on the dump-type
specified in disklist. For 'dump', the UNIX program restore is used in
conjunction with amrestore and for 'tar' the GNU program tar (surprised?).
The amanda command amrecover can be used on the client to interactively
retrieve files indexed by the amanda indexing service during backups. It
does not work on the herein discussed setup. See Problems below.
On backup and mobilkom, the restoring is done from host backup, but using
the NFS export share /d from mobilkom. Thus all extracted and restored files
are automatically put on mobilkoms /d partition (the raid5 array). This is
important and necessary because when working with hundreds of GB there is no
space left for much overhead such as trying to extract data on the backup
server first and only afterwards copying it to the backup client.
restore
-------
The simplest way to work with restore is to pipe the output of amrestore
to it and run it interactively from backup (using tape normal02 in this
example):
# cd /d
# amrestore -p -f 1 file:/dump/normal01 mobilkom md0 | restore -ivb 2 -f -
amrestore:
-p pipes the output, -f 1 'rewinds' the 'tape', mobilkom specifies the
host and md0 the disk from which the backups were done.
restore:
-i provides an interactive shell (much like FTP), -v is verbose, -b 2
sets the blocksize to 2 to avoid error messages concerning 'short reads'
and -f - specifies the file to be worked on, in this case coming from the
output of amrestore.
See the man page for both commands for further options.
tar
---
Using tar works in a similar way, on backup, too:
# cd /d
To list the contents of the tar backup file/tape:
# amrestore -p -f 1 file:/dump/normal02 mobilkom md0 | tar -tvf -
To extract all files on the tape:
# amrestore -p -f 1 file:/dump/normal02 mobilkom md0 | tar -xvf -
To extract specific files on the tape:
# amrestore -p -f 1 file:/dump/normal02 mobilkom md0 | tar -xvf -
./path/to/file/on/archive/here
5 Problems
==========
Besides its nice and comprehenisve functionality amanda also seems
to have some more or less severe bugs which may or may not happen to be
caused by the specific setup in our case (using disk files instead of real
tapes and date drives). Two major problems were encountered during testing
and configuring amanda.
1) amrecover borking
---------------------
1.1) Stalling TCP connection on extraction
- - - - - - - - - - - - - - - - - - - - -
Using amrecover on the backup client should work much like an FTP or an inter-
active restore session. It is used thus:
Change the directory to the disk which used to hold the original files and
which is the place where the are to be restored to.
# cd /d
invoke amrecover, specifying the tape server, index server and the config.
# amrecover -C normal -t backup -s backup
Here files can be added to an extraction list and then extracted using
(you guessed it) extract. At least that's how it should be.
AMRECOVER Version 2.4.4p1. Contacting server on backup ...
220 backup AMANDA index server (2.4.4p1) ready.
200 Access OK
Setting restore date to today (2003-07-23)
200 Working date set to 2003-07-23.
200 Config set to normal.
200 Dump host set to mobilkom.
Trying disk /d ...
Trying disk md0 ...
$CWD '/d' is on disk 'md0' mounted at '/d'.
200 Disk set to md0.
/d
amrecover> ls
2003-07-3 testdir1/
amrecover> add testdir1
Added dir /testdir1 at date 2003-07-2
amrecover> extract
Extracting files using tape drive file:/dump/normal02 on host backup.
The following tapes are needed: normal02
Restoring files into directory /d
Continue [?/Y/n]? y
Extracting files using tape drive file:/dump/normal02 on host backup.
Load tape normal02 now
Continue [?/Y/n/s/t]? y
The above transcript illustrates how amrecover is operated up to the
point where the actual files should be extracted. After issuing the last
'y', the connection to the backup server stalls and sits there more or
less idley for hours. No files get extracted or transferred.
Solution: None as of the time of writing.
Workaround: Using amrestore and an NFS export of the client disk (see
above Installation and Configuration).
1.2) Index on index server (backup) outdated, displaying old files
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
instead of newly backupped ones
- - - - - - - - - - - - - - - -
When using amrecover the program connects to the backup server and
queries the index catalogue it made during the backup. Every single
file on the backup tapes should thus be accessible. This does not seem
to work always. Often the amanda server provides only an outdated
(several dump cycles before made) index of the backed up files. It
is not possible to recover files from a dump after the outdated index
was created.
This could be the result of a misconfiguration on the server side or
a problem during the extensive tests of amandas functionality on backup.
However, since amrecover wouldn't allow for any file recovery as
described above, this problem/bug wasn't cared about much.
Solution: None as of the time of writing.
Workaround: Using amrestore and an NFS export of the client disk (see
above: Installation and Configuration).
2) amdump failing to dump to 'tapes', making dumpfiles/tapes on disk
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
inaccessible/corrupting them
- - - - - - - - - - - - - -
This problem is indicated by the amdump mail report. A line like this
shows up:
NOTES:
driver: mobilkom md0 3 [dump to tape failed, will try again]
driver: mobilkom md0 3 [dump to tape failed, will try again]
taper: tape normal02 kb 17184 fm 3 [OK]
No further information about the failed dump can be found in any of
the log files so this is quite a cryptic problem. When trying to re-
store from such a 'tape' a message like the following comes up and
the restore process borks:
[root@backup root]# amrestore -f 1 -p file:/dump/normal02 mobilkom md0 |
restore -ivb 2 -f -
amrestore: 1: reached end of information
Verify tape and initialize maps
Input is from file/pipe
restore: Tape read error on first record
Solution: None as of the time of writing.
Workaround: Using GNUTAR and dump-type hard-disk-tar uses tar to create
an archive of the filesystem or partition in question and
stores it on the tape. This works fine and extraction and
restoring works too. See above (Restoring).
--
Andreas "ant" Ntaflos | "A cynic is a man who knows the price of
ant AT oc DOT at | everything, and the value of nothing."
Vienna, AUSTRIA | Oscar Wilde
|
