ADSM-L

Re: [ADSM-L] 6.3.3.000 server wont HALT

2012-12-03 13:49:03
Subject: Re: [ADSM-L] 6.3.3.000 server wont HALT
From: Paul Zarnowski <psz1 AT CORNELL DOT EDU>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Mon, 3 Dec 2012 13:43:44 -0500
Can anyone say whether this problem is limited to Linux, or does it exhibit 
itself on other server platforms?  Thanks.

At 09:34 AM 12/3/2012, Andrew Raibeck wrote:
>Hi Zoltan,
>
>Looks like we have one or two other customers reporting this. My
>recommendation would be for you to open a PMR. Optionally, if you want to
>be as pro-active as possible, perform the following steps in advance (this
>is the current set of doc being requested for this issue):
>
>1. Stop TSM, kill if necessary.
>
>2. Start dsmserv in the foreground.
>
>3. Wait for TSM to come completely up.
>
>4. In the TSM server console, enable tracing by issuing:
>
>    trace enable ADM DB THREAD TM ADDMSG
>    trace begin /tmp/tsmhalttrace.out
>
>(for the "trace begin" command, specify whatever directory and trace file
>name if you want)
>
>5. Issue the HALT command on the TSM server console
>
>6. Wait 10 minutes.
>
>7. Open a root terminal on the system where TSM is running and issue the
>following commands (and save the output):
>
>    ps -ef | grep dsmserv
>    ps -ef | grep db2
>
>8. Locate the process ID (PID) for dsmserv AND db2sysc from the ps output.
>
>9. Issue the pstack command against the dsmserv PID and save the output,
>e.g.:
>
>    $ procstack 1234
>
>10. Issue the pstack command against the db2sysc PID and save the output,
>e.g.:
>
>    $ procstack 1235
>
>10. Wait 10 minutes.
>
>11. Repeat steps 7, 8, 9, and 10.  Save the output.
>
>13. Repeat steps 7, 8, 9, and 10 again, saving the output.
>
>14. Copy and save all of the on-screen output from the TSM console.
>
>15. Restart TSM.
>
>16. Login as instance owner, issue "db2support -d tsmdb1 -c -g -s", save
>the db2support.zip generated
>
>Once you have the PMR created, you can send in the doc that you collected:
>
>1.The TSM server console output
>
>2. The TSM server trace file
>
>3. The three iterations of the "ps" output
>
>4. The three iterations of pstack output against dsmserv
>
>5. The three iterations of pstack output against db2sysc
>
>6. The db2support.zip file
>
>Best regards,
>
>Andy Raibeck
>IBM Software Group
>Tivoli Storage Manager Client Product Development
>Level 3 Team Lead
>Internal Notes e-mail: Andrew Raibeck/Hartford/IBM@IBMUS
>Internet e-mail: storman AT us.ibm DOT com
>
>IBM Tivoli Storage Manager support web page:
>http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Storage_Manager
>
>"ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu> wrote on 2012-12-03
>08:45:17:
>
>> From: Zoltan Forray <zforray AT VCU DOT EDU>
>> To: ADSM-L AT vm.marist DOT edu,
>> Date: 2012-12-03 08:55
>> Subject: Re: 6.3.3.000 server wont HALT
>> Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu>
>>
>> This is now becoming a consistent / persistent problem.  I had to kill -9
>> to stop the dsmserv process.  I restarted the server (via service ..
>>  start) and there didn't seem to be any damage done.
>>
>> However, attempting to stop/halt it, again, produced the same result -
>> dsmserv using 200% CPU and after 2-hours I had to kill -9.
>>
>> So, obviously there are big enough changes in 6.3.3 vs 6.3.2, to cause
>> problems like this, since none of my 6.3.x or 6.2.x servers exhibit
>> this behavior.
>>
>> Any suggestions on how to diagnose this "issue" before I contact IBM and
>> open a PMR?
>>
>>
>> On Thu, Nov 29, 2012 at 2:04 PM, Zoltan Forray <zforray AT vcu DOT edu> 
>> wrote:
>>
>> > Just did my first install/conversion of a 6.2.3 TEST server to
>6.3.3.000
>> > (RH Linux)
>> >
>> > While the install and startup went fine, it won't HALT.
>> >
>> > After the install/upgrade, I got in via dsmadmc just fine.  Checked the
>> > actlog - saw all the schema changes/upgrades.  Updated/registered the
>> > licenses and then issued HALT.  Got the usually warning and said YES.
>> >
>> > Now it has been sitting for >25-minutes since the halt.
>> >
>> > Can't get back in via dsmadmc.
>> >
>> > Top shows dsmserv using >200% CPU.
>> >
>> > I tried standard kills, with no luck.   I hate to do a kill -9 but will
>if
>> > I don't have a choice.
>> >
>> > What the heck is it doing?  Should I wait longer or just kill it with
>> > extreme prejudice?
>> >
>> > --
>> > *Zoltan Forray*
>> > TSM Software & Hardware Administrator
>> > Virginia Commonwealth University
>> > UCC/Office of Technology Services
>> > zforray AT vcu DOT edu - 804-828-4807
>> > Don't be a phishing victim - VCU and other reputable organizations will
>> > never use email to request that you reply with your password, social
>> > security number or confidential personal information. For more details
>> > visit http://infosecurity.vcu.edu/phishing.html
>> >
>> >
>>
>>
>> --
>> *Zoltan Forray*
>> TSM Software & Hardware Administrator
>> Virginia Commonwealth University
>> UCC/Office of Technology Services
>> zforray AT vcu DOT edu - 804-828-4807
>> Don't be a phishing victim - VCU and other reputable organizations will
>> never use email to request that you reply with your password, social
>> security number or confidential personal information. For more details
>> visit http://infosecurity.vcu.edu/phishing.html
>>


--
Paul Zarnowski                            Ph: 607-255-4757
CIT Infrastructure / Storage Services     Fx: 607-255-8521
719 Rhodes Hall, Ithaca, NY 14853-3801    Em: psz1 AT cornell DOT edu