Amanda-Users

RE: amcheck hang

2005-01-07 10:21:05
Subject: RE: amcheck hang
From: "Dan Tomlinson" <dan AT azuro DOT com>
To: <amanda-users AT amanda DOT org>
Date: Fri, 7 Jan 2005 15:00:39 -0000

My amandad debug file looks like this:

 

************************

amandad: debug 1 pid 32572 ruid 34 euid 34 start time Thu Jan  6 10:52:00 2005

amandad: version 2.4.2p2

amandad: build: VERSION="Amanda-2.4.2p2"

amandad:        BUILT_DATE="Tue Apr 2 21:24:21 UTC 2002"

amandad:        BUILT_MACH="Linux cyberhq 2.4.18pre2 #1 SMP Tue Jan 8 18:13:43 PST 2002 i686 unknown"

amandad:        CC="gcc"

amandad: paths: bindir="/usr/sbin" sbindir="/usr/sbin"

amandad:        libexecdir="/usr/lib/amanda" mandir="/usr/share/man"

amandad:        AMANDA_TMPDIR="/tmp/amanda" AMANDA_DBGDIR="/tmp/amanda"

amandad:        CONFIG_DIR="/etc/amanda" DEV_PREFIX="/dev/"

amandad:        RDEV_PREFIX="/dev/r" DUMP="/sbin/dump"

amandad:        RESTORE="/sbin/restore" SAMBA_CLIENT="/usr/bin/smbclient"

amandad:        GNUTAR="/bin/tar" COMPRESS_PATH="/bin/gzip"

amandad:        UNCOMPRESS_PATH="/bin/gzip" MAILER="/usr/bin/Mail"

amandad:        listed_incr_dir="/var/lib/amanda/gnutar-lists"

amandad: defs:  DEFAULT_SERVER="localhost" DEFAULT_CONFIG="DailySet1"

amandad:        DEFAULT_TAPE_SERVER="localhost"

amandad:        DEFAULT_TAPE_DEVICE="/dev/null" HAVE_MMAP HAVE_SYSVSHM

amandad:        LOCKING=POSIX_FCNTL SETPGRP_VOID DEBUG_CODE

amandad:        AMANDA_DEBUG_DAYS=4 BSD_SECURITY USE_AMANDAHOSTS

amandad:        CLIENT_LOGIN="backup" FORCE_USERID HAVE_GZIP

amandad:        COMPRESS_SUFFIX=".gz" COMPRESS_FAST_OPT="--fast"

amandad:        COMPRESS_BEST_OPT="--best" UNCOMPRESS_OPT="-dc"

got packet:

--------

Amanda 2.4 REQ HANDLE 000-D8490608 SEQ 1105008720

SECURITY USER backup

SERVICE selfcheck

OPTIONS ;

GNUTAR /var/lib/mysql/ 0 OPTIONS |;bsd-auth;compress-best;index;exclude-list=/etc/amanda/exclude.gtar;

GNUTAR /etc/ 0 OPTIONS |;bsd-auth;compress-best;index;exclude-list=/etc/amanda/exclude.gtar;

GNUTAR /dev/vg0/perforce 0 OPTIONS |;bsd-auth;compress-best;index;exclude-list=/etc/amanda/exclude.gtar;

--------

 

sending ack:

----

Amanda 2.4 ACK HANDLE 000-D8490608 SEQ 1105008720

----

 

bsd security: remote host platinum.azuro.com user backup local user backup

amandahosts security check passed

amandad: running service "/usr/lib/amanda/selfcheck"

************************

 

The selfcheck debug file looks like this:

 

************************

selfcheck: debug 1 pid 32575 ruid 34 euid 34 start time Thu Jan  6 10:52:00 2005

/usr/lib/amanda/selfcheck: version 2.4.2p2

selfcheck: checking disk /var/lib/mysql/

************************

 

And that, as they say, is that.  Not very enlightening as far as I can see :o(  It just reports that it is checking the disk /var/lib/mysql and then stops all output, leaving the dodgy selfcheck and amandad processes hanging around and preventing the dump from continuing.

 

Another thing I might mention is, Amanda is successfully backing up directories from other machines on our network.  It is just failing for this one particular machine (which also happens to be running the Amanda demon and has the tape drive).

 

Any ideas would be greatly appreciated…

 

Dan

 


From: Dan Tomlinson
Sent: 06 January 2005 14:20
To: amanda-users AT amanda DOT org
Subject: amcheck hang

 

Hi all,

amanda has been failing during my nightly dumps:

FAILURE AND STRANGE DUMP SUMMARY:
   mymachine    /var/lib/mysql/ lev 0 FAILED [mymachine NAK: amandad busy]

 

Trawled newsgroups for answers and it appears that this particular error can be caused by amandad processes hanging around after a failed amanda operation. Did a quick "ps -ef | grep amanda" and saw there were "amandad" and "..amanda/selfcheck" processes active. Tried to kill them, but only the amandad would die :o( Eventually managed to kill the selfcheck with a kill -9

I labeled up another new tape to attempt to manually continue the dumpcycle, and ran an "amcheck" after labelling, only to have it fail to finish. Another "ps -ef | grep amanda" found that there was now a new selfcheck process and a new amandad.

So the problem seems to be that the selfcheck process is hanging during amcheck execution and preventing the dump from finishing.  As to how to solve it? Various in the mailing lists suggest rebooting machine but a) this is inconvienient to say the least, and b) how do I know this wont just happen again next time amanda runs...

I am using amanda version 2.4.2p2 on debian with a 2.4 kernel, any ideas?  Do we need to upgrade our amanda?

Many thanks in advance

Dan T

<Prev in Thread] Current Thread [Next in Thread>