Amanda-Users

Re: Dumps cannot be retrieved from tape...

2008-09-24 12:10:57
Subject: Re: Dumps cannot be retrieved from tape...
From: Chris Hoogendyk <hoogendyk AT bio.umass DOT edu>
To: Dan Brown <monkeypants AT shaw DOT ca>
Date: Wed, 24 Sep 2008 09:59:11 -0400
Since there is a lot of detail here, and I don't want to get comments lost in the detail, I'm going to top-post.

First off, I noticed that all your dump levels in the history are level 1 or 2 incrementals. There are no fulls, going back 3 months. Looking at your amanda.conf file shows why. You have a dump cycle of 52 weeks, with a runspercycle of 30. But, in the last 3 months you have over 40 runs. This doesn't make sense.

For reference on dumpcyle, check out
http://wiki.zmanda.com/index.php/Dumpcycle and
http://wiki.zmanda.com/index.php/FAQ:How_are_backup_levels_defined_and_how_does_Amanda_use_them%3F

Typically, one uses a shorter dumpcycle, fewer runspercycle, and a longer tapecycle. So, for example, I have 1 week, 7 runs, and 35 tapes. I understand that you want to conserve tape usage, and are keeping backups on the holding disk until you have enough to fill a tape. I actually do that with incrementals only to th holding disk on weekends. So I have 7 runs in a week, but only use 5 tapes in a week.

I would suggest you try 1 month, 15 runs, and however many tapes you want; or, perhaps, 2 weeks, 6 runs, and however many tapes. It really depends on how large your full backups are going to be and how that fits on your holding disk or on your tapes. Amanda will smooth the load over the dumpcycle, and you just need to balance your priorities regarding resource usage versus risk aversion (not wanting to lose data). I'm not sure why your current schedule is irregular, or why you don't want to run every night to catch incremental changes and distribute the load of full backups more.

I also noticed that your tapedev was "tape:/dev/st0". You'll have to check your devices and man pages, I'm not a linux user. However, should that be "/dev/nst0"? In other words, pick the non-rewinding device? On my systems, I have /dev/rmt/0 for the plain device and /dev/rmt/0n for the non-rewinding version. I just googled on your device string and found the nst0, but that's not proof of anything, just a possible clue. I could be off.

Let us know what you find on your tapes and what your thoughts were on your schedule. With some input on what you were trying to achieve, I'm sure there will be more suggestions on how to accomplish that.


---------------

Chris Hoogendyk

-
  O__  ---- Systems Administrator
 c/ /'_ --- Biology & Geology Departments
(*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst
<hoogendyk AT bio.umass DOT edu>

---------------
Erdös 4




Dan Brown wrote:
Chris Hoogendyk wrote:
> > First, it should work. amrecover should get things off the holding disk > > or off tape as appropriate. When you say "came to realize", I think we > > need some more detail on what you mean by that. What errors did you get? > > How did you try? Did you use setdate to get the file you wanted from a
> > particular backup?

Here's what I see when I use amrecover.

[root@ministryofinformation __designmac_design_resources_]# amrecover Macs could not open conf file "/etc//amanda/amanda-client.conf": No such file or
directory
AMRECOVER Version 2.6.0p1. Contacting server on ministryofinformation ...
220 ministryofinformation AMANDA index server (2.5.2p1) ready.
Setting restore date to today (2008-09-23)
200 Working date set to 2008-09-23.
200 Config set to Macs.
200 Dump host set to ministryofinformation.
Use the setdisk command to choose dump disk to recover
amrecover> setdisk //designmac/design_resources/
200 Disk set to //designmac/design_resources/.
amrecover> setdate ---16
200 Working date set to 2008-09-16.
No index records for cwd on new date
Setting cwd to mount point
amrecover>

It does this for anything before 2008-09-19, which is for when I still
have backups still waiting in the holding disk. The file I want is from the
16th however.


> > Did you try "history" in amrecover to see what it
> > thought you had after you had sethost and setdisk? Did you sethost to
> > where the samba share was mounted and backed up from? As opposed to the
> > machine it originated on?

The backup server is the "ministryofinformation" (Brazil reference), hence
all
samba shares on Mac OS X clients are backed up from being mounted by the
server.

ministryofinformation   //designmac/design_resource_archive/    user-tar
ministryofinformation   //designmac/design_resources/           user-tar

A history command shows the following:

amrecover> setdisk //designmac/design_resources/
200 Disk set to //designmac/design_resources/.
amrecover> history
200- Dump history for config "Macs" host "ministryofinformation" disk
//designmac/design_resources/
201- 2008-09-22-10-45-01 2
/dumps/holding-mac/20080922104501/ministryofinformation.__designmac_design_r
esources_.2:0
201- 2008-09-21-10-45-01 2
/dumps/holding-mac/20080921104501/ministryofinformation.__designmac_design_r
esources_.2:0
201- 2008-09-20-10-45-01 2
/dumps/holding-mac/20080920104501/ministryofinformation.__designmac_design_r
esources_.2:0
201- 2008-09-19-10-45-01 2
/dumps/holding-mac/20080919104501/ministryofinformation.__designmac_design_r
esources_.2:0
200 Dump history for config "Macs" host "ministryofinformation" disk
//designmac/design_resources/
amrecover>

As you can see, these are only dumps still in holding.

On the otherhand according to amadmin:

amanda@ministryofinformation ~]$ amadmin Macs find ministryofinformation
//designmac/design_resources/


date host disk lv
tape or file
file part status
2008-06-21 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-012
1  1/1 OK
2008-06-25 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-012
46  1/1 PARTIAL
2008-06-25 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-013
1  1/1 OK
2008-06-26 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-013
21  1/1 OK
2008-06-28 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-013
52  1/1 PARTIAL
2008-06-28 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-014
1  1/1 OK
2008-06-29 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-014
19  1/1 OK
2008-06-30 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-015
2  1/1 OK
2008-07-04 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-015
34  1/1 OK
2008-07-08 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-016
74  1/1 PARTIAL
2008-07-08 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-017
1  1/1 OK
2008-07-17 10:45:02 ministryofinformation //designmac/design_resources/ 1
Macs-017
95  1/1 PARTIAL
2008-07-17 10:45:02 ministryofinformation //designmac/design_resources/ 1
Macs-018
1  1/1 OK
2008-07-18 10:45:02 ministryofinformation //designmac/design_resources/ 1
Macs-018
23  1/1 OK
2008-07-19 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-019
2  1/1 OK
2008-07-22 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-000
1  1/1 OK
2008-07-22 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-019
36  1/1 PARTIAL
2008-07-23 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-000
15  1/1 OK
2008-07-24 10:45:01 ministryofinformation //designmac/design_resources/ 1
Macs-001
8  1/1 OK
2008-07-27 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-001
61  1/1 OK
2008-07-29 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-001
74  1/1 PARTIAL
2008-07-29 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-002
1  1/1 OK
2008-08-07 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-003
2  1/1 OK
2008-08-14 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-004
6  1/1 OK
2008-08-15 10:45:02 ministryofinformation //designmac/design_resources/ 2
Macs-004
29  1/1 PARTIAL
2008-08-15 10:45:02 ministryofinformation //designmac/design_resources/ 2
Macs-005
1  1/1 OK
2008-08-16 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-005
20  1/1 OK
2008-08-17 10:45:02 ministryofinformation //designmac/design_resources/ 2
Macs-006
12  1/1 OK
2008-08-28 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-006
68  1/1 PARTIAL
2008-08-28 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-008
1  1/1 OK
2008-09-06 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-008
65  1/1 PARTIAL
2008-09-06 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-010
1  1/1 OK
2008-09-07 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-010
15  1/1 OK
2008-09-10 10:45:02 ministryofinformation //designmac/design_resources/ 2
Macs-011
2  1/1 OK
2008-09-11 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-011
19  1/1 OK
2008-09-12 10:45:01 ministryofinformation //designmac/design_resources/ 2
0   -- FAILED (driver) [can't dump required holdingdisk]
2008-09-16 10:45:01 ministryofinformation //designmac/design_resources/ 2
Macs-009
12  1/1 OK
2008-09-19 10:45:01 ministryofinformation //designmac/design_resources/ 2 /dumps/holding-mac/20080919104501/ministryofinformation.__designmac_design_r
esources_.2    0   -- OK
2008-09-20 10:45:01 ministryofinformation //designmac/design_resources/ 2 /dumps/holding-mac/20080920104501/ministryofinformation.__designmac_design_r
esources_.2    0   -- OK
2008-09-21 10:45:01 ministryofinformation //designmac/design_resources/ 2 /dumps/holding-mac/20080921104501/ministryofinformation.__designmac_design_r
esources_.2    0   -- OK
2008-09-22 10:45:01 ministryofinformation //designmac/design_resources/ 2 /dumps/holding-mac/20080922104501/ministryofinformation.__designmac_design_r
esources_.2    0   -- OK

So it does appear that Amanda knows about the dump, just not under root.


> > If you look at the tape directly using OS utilities, the first file on
> > the tape tells you what to do. There are detailed descriptions of
> > various approaches to reading the tapes here:
> > http://wiki.zmanda.com/index.php/Restoring_files, and if you scan down > > that page looking for dd, you will find how to do it without any Amanda
> > tools.

Well seeing as I can use tar on this, I'll give that a shot. My users are ridiculously redundant most of the time so it's pretty rare that something
actually needs restoration.

> >
> > If you need help with your configuration, you will have to post it.

Here it is, I see nothing obvious which would prevent indexing from being
recorded when backups are flushed to tape.

org      "Macs" # your organization name for reports
mailto   "danb AT zu DOT com"  # space separated list of operators at your site
dumpuser "amanda"       # the user to run dumps under

inparallel 16 # maximum dumpers that will run in parallel (max 63) dumporder "BTBTBTBTBTBTBTBT" # specify the priority order of each dumper

taperalgo first # The algorithm used to choose which dump image to
send

displayunit "m"         # Possible values: "k|m|g|t"

netusage  10000 Kbps    # maximum net bandwidth for Amanda, in KB per sec

dumpcycle 52 weeks      # the number of days in the normal dump cycle
runspercycle 30         # the number of amdump runs in dumpcycle days
tapecycle 15 tapes      # the number of tapes in rotation

bumpsize 20 Mb # minimum savings (threshold) to bump level 1 -> 2 bumppercent 20 # minimum savings (threshold) to bump level 1 -> 2
bumpdays 1              # minimum days at each level
bumpmult 4              # threshold = bumpsize * bumpmult^(level-1)

etimeout 600            # number of seconds per filesystem for estimates.
dtimeout 1800 # number of idle seconds before a dump is aborted.
ctimeout 600            # maximum number of seconds that amcheck waits

tapebufs 128            # A positive integer telling taper how many

usetimestamps yes

runtapes 1 # number of tapes to be used in a single run
of amdump
tapedev "tape:/dev/st0" # the no-rewind tape device to be used

maxdumpsize -1 # Maximum number of bytes the planner will
schedule
tapetype LTO3-400               # what kind of tape it is (see tapetypes
below)
labelstr "^Macs-[0-9][0-9][0-9]*$" # label constraint regex: all tapes
must match

amrecover_do_fsf yes            # amrecover will call amrestore with the
                                # -f flag for faster positioning of the
tape.
amrecover_check_label yes       # amrecover will call amrestore with the
                                # -l flag to check the label.
amrecover_changer "/dev/null"   # amrecover will use the changer if you
restore
                                # from this device.
                                # It could be a string like 'changer' and
                                # amrecover will use your changer if you
                                # set your tape with 'settape changer'
holdingdisk hd1 {
    comment "main Mac holding disk"
    directory "/dumps/holding-mac"      # where the holding disk is
    use -100 Mb                 # how much space can we use on it
                                # a non-positive value means:
                                # use all space but that value
    chunksize 2Gb       # size of chunk if you want big dump to be
    }
reserve 30 # percent
autoflush no #

infofile "/etc/amanda/Macs/curinfo"     # database DIRECTORY
logdir   "/etc/amanda/Macs"             # log directory
indexdir "/etc/amanda/Macs/index"               # index directory
tapelist "/etc/amanda/Macs/tapelist"    # list of used tapes

define tapetype LTO3-400 {
    comment "Dell PV124T LTO3 (hardware compression off)"
    length 402432 mbytes
    filemark 0 kbytes
    speed 71189 kps
}

define tapetype LTO2-200 {
    comment "Dell PV124T LTO3 (degraded LTO2 tapes)"
    length 201216 mbytes
    filemark 0 kbytes
    speed 71189 kps
}



define dumptype global {
    comment "Global definitions"
    index yes
    estimate client
    holdingdisk required
    record yes
    fallback_splitsize 64m
    auth "bsdtcp"
    maxdumps 3
}

define dumptype user-tar {
    global
    program "GNUTAR"
    comment "user partitions dumped with tar"
    priority high
    compress none
    index
}


define interface local {
    comment "a local disk"
    use 1000 kbps
}

define interface le0 {
    comment "10 Mbps ethernet"
    use 400 kbps
}

define interface eth0 {
    comment "1Gbps ethernet"
    use 921600 kbps
}



----
Dan Brown
monkeypants AT shaw DOT ca

<Prev in Thread] Current Thread [Next in Thread>