ADSM-L

Re: [ADSM-L] 6.3.3.000 server wont HALT

2012-12-03 15:14:01
Subject: Re: [ADSM-L] 6.3.3.000 server wont HALT
From: Zoltan Forray <zforray AT VCU DOT EDU>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Mon, 3 Dec 2012 10:28:26 -0500
Thanks Andy.  Glad to know I am not the only one.   Will collect the info
and open a PMR.  Just another reason they call it the "bleeding edge" ;--)


On Mon, Dec 3, 2012 at 9:34 AM, Andrew Raibeck <storman AT us.ibm DOT com> 
wrote:

> Hi Zoltan,
>
> Looks like we have one or two other customers reporting this. My
> recommendation would be for you to open a PMR. Optionally, if you want to
> be as pro-active as possible, perform the following steps in advance (this
> is the current set of doc being requested for this issue):
>
> 1. Stop TSM, kill if necessary.
>
> 2. Start dsmserv in the foreground.
>
> 3. Wait for TSM to come completely up.
>
> 4. In the TSM server console, enable tracing by issuing:
>
>     trace enable ADM DB THREAD TM ADDMSG
>     trace begin /tmp/tsmhalttrace.out
>
> (for the "trace begin" command, specify whatever directory and trace file
> name if you want)
>
> 5. Issue the HALT command on the TSM server console
>
> 6. Wait 10 minutes.
>
> 7. Open a root terminal on the system where TSM is running and issue the
> following commands (and save the output):
>
>     ps -ef | grep dsmserv
>     ps -ef | grep db2
>
> 8. Locate the process ID (PID) for dsmserv AND db2sysc from the ps output.
>
> 9. Issue the pstack command against the dsmserv PID and save the output,
> e.g.:
>
>     $ procstack 1234
>
> 10. Issue the pstack command against the db2sysc PID and save the output,
> e.g.:
>
>     $ procstack 1235
>
> 10. Wait 10 minutes.
>
> 11. Repeat steps 7, 8, 9, and 10.  Save the output.
>
> 13. Repeat steps 7, 8, 9, and 10 again, saving the output.
>
> 14. Copy and save all of the on-screen output from the TSM console.
>
> 15. Restart TSM.
>
> 16. Login as instance owner, issue "db2support -d tsmdb1 -c -g -s", save
> the db2support.zip generated
>
> Once you have the PMR created, you can send in the doc that you collected:
>
> 1.The TSM server console output
>
> 2. The TSM server trace file
>
> 3. The three iterations of the "ps" output
>
> 4. The three iterations of pstack output against dsmserv
>
> 5. The three iterations of pstack output against db2sysc
>
> 6. The db2support.zip file
>
> Best regards,
>
> Andy Raibeck
> IBM Software Group
> Tivoli Storage Manager Client Product Development
> Level 3 Team Lead
> Internal Notes e-mail: Andrew Raibeck/Hartford/IBM@IBMUS
> Internet e-mail: storman AT us.ibm DOT com
>
> IBM Tivoli Storage Manager support web page:
>
> http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Storage_Manager
>
> "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu> wrote on 2012-12-03
> 08:45:17:
>
> > From: Zoltan Forray <zforray AT VCU DOT EDU>
> > To: ADSM-L AT vm.marist DOT edu,
> > Date: 2012-12-03 08:55
> > Subject: Re: 6.3.3.000 server wont HALT
> > Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu>
> >
> > This is now becoming a consistent / persistent problem.  I had to kill -9
> > to stop the dsmserv process.  I restarted the server (via service ..
> >  start) and there didn't seem to be any damage done.
> >
> > However, attempting to stop/halt it, again, produced the same result -
> > dsmserv using 200% CPU and after 2-hours I had to kill -9.
> >
> > So, obviously there are big enough changes in 6.3.3 vs 6.3.2, to cause
> > problems like this, since none of my 6.3.x or 6.2.x servers exhibit
> > this behavior.
> >
> > Any suggestions on how to diagnose this "issue" before I contact IBM and
> > open a PMR?
> >
> >
> > On Thu, Nov 29, 2012 at 2:04 PM, Zoltan Forray <zforray AT vcu DOT edu> 
> > wrote:
> >
> > > Just did my first install/conversion of a 6.2.3 TEST server to
> 6.3.3.000
> > > (RH Linux)
> > >
> > > While the install and startup went fine, it won't HALT.
> > >
> > > After the install/upgrade, I got in via dsmadmc just fine.  Checked the
> > > actlog - saw all the schema changes/upgrades.  Updated/registered the
> > > licenses and then issued HALT.  Got the usually warning and said YES.
> > >
> > > Now it has been sitting for >25-minutes since the halt.
> > >
> > > Can't get back in via dsmadmc.
> > >
> > > Top shows dsmserv using >200% CPU.
> > >
> > > I tried standard kills, with no luck.   I hate to do a kill -9 but will
> if
> > > I don't have a choice.
> > >
> > > What the heck is it doing?  Should I wait longer or just kill it with
> > > extreme prejudice?
> > >
> > > --
> > > *Zoltan Forray*
> > > TSM Software & Hardware Administrator
> > > Virginia Commonwealth University
> > > UCC/Office of Technology Services
> > > zforray AT vcu DOT edu - 804-828-4807
> > > Don't be a phishing victim - VCU and other reputable organizations will
> > > never use email to request that you reply with your password, social
> > > security number or confidential personal information. For more details
> > > visit http://infosecurity.vcu.edu/phishing.html
> > >
> > >
> >
> >
> > --
> > *Zoltan Forray*
> > TSM Software & Hardware Administrator
> > Virginia Commonwealth University
> > UCC/Office of Technology Services
> > zforray AT vcu DOT edu - 804-828-4807
> > Don't be a phishing victim - VCU and other reputable organizations will
> > never use email to request that you reply with your password, social
> > security number or confidential personal information. For more details
> > visit http://infosecurity.vcu.edu/phishing.html
> >
>



--
*Zoltan Forray*
TSM Software & Hardware Administrator
Virginia Commonwealth University
UCC/Office of Technology Services
zforray AT vcu DOT edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html