Veritas-bu

Re: [Veritas-bu] Client freezing media.

2009-12-08 14:19:30
Subject: Re: [Veritas-bu] Client freezing media.
From: <judy_hinchcliffe AT administaff DOT com>
To: <jeff.cleverley AT avagotech DOT com>, <VERITAS-BU AT mailman.eng.auburn DOT edu>
Date: Tue, 8 Dec 2009 13:18:47 -0600

My first thought is to run the database consistency check.

 

From: veritas-bu-bounces AT mailman.eng.auburn DOT edu [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Jeff Cleverley
Sent: Tuesday, December 08, 2009 12:39 PM
To: VERITAS-BU AT mailman.eng.auburn DOT edu
Subject: [Veritas-bu] Client freezing media.

 

Greetings,

I'm going nuts trying to figure out what is happening with one of my SAN media servers (NBU 5.1, hpux 11.11 servers).  Everything was fine on Saturday morning.  Backups went as planned.  Saturday night, one of the media servers wiped out all 65 scratch tapes in the library.  On the master I have taken those media and run the following against each media:

/usr/openv/netbackup/bin/admincmd/bpmedia -unfreeze -m $TAPENAME
echo y | /usr/openv/netbackup/bin/admincmd/bpexpdate -d 0 -m $TAPENAME > /dev/null
echo y | /usr/openv/netbackup/bin/admincmd/bpexpdate -deassignempty -m $TAPENAME > /dev/null
/usr/openv/volmgr/bin/vmquery -deassignbyid $TAPENAME $POOLNAME 0
/usr/openv/volmgr/bin/vmdelete -m $TAPENAME

I've then done a vmquery -m $TAPENAME and nothing shows up in a database on either the master or the media server.  I then re-scan the library and the tapes go back to the scratch pool.  When I start a new backup, I find this in the media server bptm log file:

11:10:22.859 [12387] <2> vmdb_query_scratch_bypool2: server returned:  1 BK7028 ------ 6 BK7028L2 -------- 8 21 coda 00_021_TLD - 41 0 0 0 0 root root 2 backup scratch 1260295150 1260295814 0 0 0 0 0 0 0 - 0 0 50 0 0 0 0 0 - 0 0 0 0 0 0 0 0 0 0 0 - 0 0 0 0 0 0 0 0 0 0 0 0 - ---
11:10:22.860 [12387] <2> db_byid: search for media id BK7028
11:10:22.860 [12387] <2> db_byid: BK7028 found at offset 70
11:10:22.860 [12387] <2> select_media: media id BK7028 just obtained from Media Manager is already in the media database, requesting a new one
11:10:22.860 [12387] <2> select_media: getting new media id for retention level 5

It then marks the tape as frozen and wipes everything out again.  Where is the process querying the database at that the vmquery doesn't find?  I've stopped and started all processes on both the master and media server multiple times.

One thing that does seem odd is the media server vmquery does not show any valid media.  Everything was old.  I did a vmdelete of all of them.  After wiping out all the scratch tapes the media manager still shows no media.

Any help figuring out what bit of corruption it is looking at would be greatly appreciated.

Thanks,

Jeff

--
Jeff Cleverley
Unix Systems Administrator
4380 Ziegler Road
Fort Collins, Colorado 80525
970-288-4611

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
<Prev in Thread] Current Thread [Next in Thread>