Amanda-Users

Re: amverify stuck, using 95% of cpu

2004-11-16 22:34:40
Subject: Re: amverify stuck, using 95% of cpu
From: Gene Heskett <gene.heskett AT verizon DOT net>
To: amanda-users AT amanda DOT org
Date: Tue, 16 Nov 2004 22:13:26 -0500
On Tuesday 16 November 2004 09:05, Gene Heskett wrote:

Synopsis: Set runtapes to 2 to let amanda get caught up.

Humm, a test run of amverify has been hung for several hours now.

Here is the output from the shell its running in:
[amanda@coyote amanda]$ amverify Daily 22 2
Tape changer is chg-disk...
2 slots...
Verify summary to root AT coyote.coyote DOT den
Defects file is /tmp/amanda/amverify.25796/defects
amverify Daily
Tue Nov 16 17:13:52 EST 2004

Loading 22 slot...
Using device file:/amandatapes/Dailys/
Waiting for device to go ready...
Rewinding...
Processing label...
Volume Dailys-22, Date 20041116
Rewinding...
Checked coyote._tmp.20041116.1
Checked coyote._home.20041116.0
Checked coyote._root.20041116.1
Checked coyote._usr_src.20041116.1
Checked coyote._var.20041116.1
Checked coyote._usr_dlds-misc_FC3-i386-disk2.iso.20041116.0
Checked coyote._usr_dlds-misc_FC3_FC3-i386-SRPMS-disk2.iso.20041116.1
Checked coyote._usr_dlds-misc_FC3_FC3-i386-disk3.iso.20041116.0
Checked coyote._usr_dlds-misc_FC3_FC3-i386-SRPMS-disk4.iso.20041116.1
Checked coyote._usr_dlds-misc_FC3_FC3-i386-disc1.iso.20041116.0
Checked coyote._usr_dlds-misc_FC3_FC3-i386-SRPMS-disc1.iso.20041116.1
Checked coyote._usr_dlds-misc_FC3_FC3-i386-SRPMS-disk3.iso.20041116.1
Checked coyote._usr_share.20041116.1
** Error detected (coyote._usr_dlds-misc_FC3_FC3-i386-disk4.iso.20041116.1)
amrestore: WARNING: not at start of tape, file numbers will be offset
amrestore:   0: restoring 
coyote._usr_dlds-misc_FC3_FC3-i386-disk4.iso.20041116.1
amrestore:   1: reached end of information
/bin/gtar: Unexpected EOF in archive
/bin/gtar: Error is not recoverable: exiting now
64+0 in
64+0 out
---------------
and thats where its stuck.  When it started, it gave me a file
to look at in case there were errors.
-------------
[amanda@coyote amanda]$ tail -f /tmp/amanda/amverify.25796/defects
Dailys-22 (coyote._usr_dlds-misc_FC3_FC3-i386-disk4.iso.20041116.1):
amrestore: WARNING: not at start of tape, file numbers will be offset
amrestore:   0: restoring 
coyote._usr_dlds-misc_FC3_FC3-i386-disk4.iso.20041116.1
amrestore:   1: reached end of information
/bin/gtar: Unexpected EOF in archive
/bin/gtar: Error is not recoverable: exiting now
64+0 in
64+0 out
---------------
in other words the same as the cli reported.
------------
Looking at system usage according to kpm, or htop, this from htop:
PID   USER  PRI  NI  VIRT  RES  SHR  State CPU%  MEM%  COMM
26278 amanda 25   0  1776  588  472    R    92.1  0.1  
/usr/local/sbin/amrestore -h -p file:/amandatapes/Dailys/

Note that it does not specify the data dir in the command as issued.
Thats the parent dir, and there are no actual files for amrestore
to read there.  That doesn't seem to make sense to me.  Can one of the
hackers please comment?

I just gave the amverify cli a ctl-c, and it exited, again with a
memory allocation related error I believe:
---------------
aborted!
Null message body; hope that's ok
/tmp/RsuPHJDu: No space left on device
[amanda@coyote amanda]$

/tmp is a directory on /, and a df says its 89% full.  Cleaned
house a bit now but its still 89%, but that 89% means there is 5GB
of space on the device.  I think the message above is a red herring.

Can someone please tell me whats going on?

[... old posts on this subject]

As a temp fix, I've gone back to a runtapes of 1, rm'd the
last 10 vtapes (which includes the last 4 backups made :()
to make room for a bigger vtape & we'll see what
happens tonight.  A bit frustrating.  However, I did just
now find that the tapelist is not being updated by an amcheck,
only the /configdir/chg-disk-slot is updated, so at least
I can read that and save it to define what starting slot I feed
amverify.

That will remove one element of confusion in this scene.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.29% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attorneys please note, additions to this message
by Gene Heskett are:
Copyright 2004 by Maurice Eugene Heskett, all rights reserved.

<Prev in Thread] Current Thread [Next in Thread>