Amanda-Users

Re: error recovery

2005-09-06 09:51:41
Subject: Re: error recovery
From: Jon LaBadie <jon AT jgcomp DOT com>
To: amanda users mailing list <amanda-users AT amanda DOT org>
Date: Tue, 6 Sep 2005 09:41:11 -0400
On Tue, Sep 06, 2005 at 01:53:05PM +0100, Rodrigo Ventura wrote:
> 
> I was finally able to pass the estimate timeout problem I was having
> (solution: a combination of decreased cpu/swap intensive processes and
> increased etimeout value). However, a couple of troubles came up:
> 
> I got failed dumps (I was expecting this) because of end-of-tape:
> 
> ======================================================================
> These dumps were to tape ISR005.
> *** A TAPE ERROR OCCURRED: [[writing file: No space left on device]].
> Some dumps may have been left in the holding disk.
> Run amflush to flush them to tape.
> The next tape Amanda expects to use is: ISR001.
> [...]
>   omni       /home/nt lev 0 FAILED [out of tape]
>   omni       / lev 1 FAILED [can't dump no-hold disk in degraded mode]
> [...]
> NOTES:
>   planner: Last full dump of omni:/home/ag on tape ISR005 overwritten on this 
> run.
>   planner: Last full dump of omni:/var/spool/imap/user/hm on tape ISR005 
> overwritten on this run.
>   planner: Last full dump of omni://new/E$ on tape ISR005 overwritten on this 
> run.
>   planner: Incremental of damiao:/ bumped to level 2.
>   planner: Incremental of omni:/var/spool/imap/user/hm bumped to level 2.
>   planner: gtisr /usr 20050906 0 [dumps too big, 398656 KB, full dump delayed]
>   planner: damiao /usr 20050906 0 [dumps too big, 598668 KB, full dump 
> delayed]
>   planner: gtisr / 20050906 0 [dumps too big, 716448 KB, full dump delayed]
>   planner: omni / 20050906 0 [dumps too big, 3878447 KB, full dump delayed]
>   planner: omni /home/hm 20050906 0 [dumps too big, 6360061 KB, full dump 
> delayed]
>   planner: omni //new/E$ 20050906 0 [dumps too big, 3386713 KB, full dump 
> delayed]
>   planner: omni /home/ag 20050906 0 [dumps too big, 5373985 KB, full dump 
> delayed]
>   planner: omni /var/spool/imap/user/hm 20050906 0 [dumps too big, 5761742 
> KB, full dump delayed]
>   taper: tape ISR005 kb 30871904 fm 21 writing file: No space left on device
>   driver: going into degraded mode because of tape error.
> [...]


Your DLE's that are "too big" will never be backed up.  They need to be split 
into
smaller, multiple DLE's (don't ask, read first) or you need larger tapes.


> DUMP SUMMARY:
>                                      DUMPER STATS            TAPER STATS 
> HOSTNAME     DISK        L ORIG-KB OUT-KB COMP% MMM:SS  KB/s MMM:SS  KB/s
> -------------------------- --------------------------------- ------------
> damiao       /           0 1141360 429995  37.7  13:48 519.3   3:012370.4
> damiao       /boot       0    4930   3654  74.1   0:12 302.0   0:021759.7
> damiao       /raid1      1    3600    290   8.1   0:35   8.2   0:02 130.4
> damiao       /raid1/www  1   22370   8621  38.5   1:15 114.4   0:061473.3
> damiao       /usr        1    5510    532   9.7   1:04   8.3   0:09  59.4
> gtisr        /           1   31710   3496  11.0   0:47  74.7   0:021938.1
> gtisr        /boot       0    6820   4709  69.0   0:031360.6   0:022932.4
> gtisr        /usr        1    5650    562   9.9   0:25  22.7   0:14  39.4
> omni         /           1 FAILED ---------------------------------------
> omni         //new/C$    0 32633101670382  51.2  16:381673.1  11:022523.3
> omni         //new/E$    1  961248 103532  10.8   3:12 538.2   0:422470.3
> omni         /boot       0    6750   5038  74.6   0:031458.7   0:031511.8
> omni         /home/ag    1   11510   2649  23.0   0:45  58.5   0:05 576.7
> omni         /home/hm    1 12223101195526  97.8   7:352629.9   7:502542.7
> omni         /home/nt    0 1493210012752875  85.4  60:573487.4  FAILED  ----
> omni         /home/uz    0 28596102189626  76.6  14:052590.4  13:542626.1
> omni         /root       0  432700 347637  80.3   2:172542.0   2:152578.8
> omni         /usr        0 33126401578130  47.6  29:11 901.0  10:352485.7
> omni         -ap/user/ag 0 78292705183322  66.2  45:381892.9  33:282581.7
> omni         -ap/user/hm 2  345390 191659  55.5   2:461157.9   1:132615.7
> omni         -ap/user/nt 0 67581704649497  68.8  31:582423.7  29:542591.1
> omni         -ap/user/uz 0 20160101059867  52.6  28:49 613.0   7:012517.0


Look into the columnspec parameter to adjust the column spacing.

> 
> (brought to you by Amanda version 2.4.4p4)
> ======================================================================
> 
> 
> Now, I have a bunch of questions about this report:
> 
> 1. what's the meaning of "Some dumps may have been left in the holding
> disk."? I mean, can I expect them to be dumped on the next amdump? Or

User configurable, parameter autoflush.

> do I have to call amcleanup/amflush (which?) to check them out to
> tape? In this latter case, I'll be overwriting a tape of my cycle. I

You don't "have to", but you can.  And yes, it will overwrite the next
tape.  But it seems you may have too much data for your tape and tapecycle.

> have one spare tape (my cycle is 4 tapes, and I have a magazine of 5
> loaded).

I know they might be expensive, but if your data is important buy more
tapes.  You are at the very minimum that amanda should ever work with.

> 
> 2. what does it mean to go "into degraded mode"?
> 

No place to tape your dumps and no unreserved holding disk space.
Amanda believe you care more about the changed files rather than
new copies of unchanged files.  So it does not do full backups
even if scheduled.

> 3. let's take omni:/home/ag for instance; in NOTES amanda says "Last
> full dump of omni:/home/ag on tape ISR005 overwritten on this run",
> but in DUMP SUMMARY it says
>                                      DUMPER STATS            TAPER STATS 
> HOSTNAME     DISK        L ORIG-KB OUT-KB COMP% MMM:SS  KB/s MMM:SS  KB/s
> -------------------------- --------------------------------- ------------
> omni         /home/ag    1   11510   2649  23.0   0:45  58.5   0:05 576.7
> 
> meaning it was level 1. My interpretation is: since the last full dump
> was overwritten, and the current one is level 1, I LOST the backup. Is
> this correct? (Shall I panic?)

No, buy more tapes.  You can still get the files from the incremental
dump (i.e. the changed ones) but not the unchanged ones from the now
overwritten full dump.  Buy more tapes.

Aside from my archive tapes (never reused) I rotate enough tapes to
have 3 to 5 dumpcycles worth of data around.  For me that is 24 tapes
at a nominal 6 per week.

> 
> 4. How can I figure out the *current* *state* of the whole backup
> system? I'd like to know what full backups exist in what tapes, and
> what levels and where, and whether I am able to recover everything

amoverview

> from the current tape set. The amanda mail report are great, but they
> only reflect the last amdump operation. I'd like to know about the
> actual state of the whole tape set.
> 
> 5. I read somewhere in amanda docs that whenever there is a
> end-of-tape, the next one is automatically loaded and overwritten with
> the dumps that did not fit on the previous one. Is this true? However,
> in this amanda report I quoted, "These dumps were to tape ISR005" and
> "The next tape Amanda expects to use is: ISR001" which seems ok (I
> only have ISR001-ISR005). So can I conclude that amanda *only* used
> one tape? The dumps that did not fit are on holding disk?

You have not told amanda it is allowed to use more than one tape per
amdump run (runtapes parameter).


-- 
Jon H. LaBadie                  jon AT jgcomp DOT com
 JG Computing
 4455 Province Line Road        (609) 252-0159
 Princeton, NJ  08540-4322      (609) 683-7220 (fax)

<Prev in Thread] Current Thread [Next in Thread>