ADSM-L

Re: [ADSM-L] 6.3.3.000 server wont HALT

2012-12-03 09:36:47
Subject: Re: [ADSM-L] 6.3.3.000 server wont HALT
From: Andrew Raibeck <storman AT US.IBM DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Mon, 3 Dec 2012 09:34:50 -0500
Hi Zoltan,

Looks like we have one or two other customers reporting this. My
recommendation would be for you to open a PMR. Optionally, if you want to
be as pro-active as possible, perform the following steps in advance (this
is the current set of doc being requested for this issue):

1. Stop TSM, kill if necessary.

2. Start dsmserv in the foreground.

3. Wait for TSM to come completely up.

4. In the TSM server console, enable tracing by issuing:

    trace enable ADM DB THREAD TM ADDMSG
    trace begin /tmp/tsmhalttrace.out

(for the "trace begin" command, specify whatever directory and trace file
name if you want)

5. Issue the HALT command on the TSM server console

6. Wait 10 minutes.

7. Open a root terminal on the system where TSM is running and issue the
following commands (and save the output):

    ps -ef | grep dsmserv
    ps -ef | grep db2

8. Locate the process ID (PID) for dsmserv AND db2sysc from the ps output.

9. Issue the pstack command against the dsmserv PID and save the output,
e.g.:

    $ procstack 1234

10. Issue the pstack command against the db2sysc PID and save the output,
e.g.:

    $ procstack 1235

10. Wait 10 minutes.

11. Repeat steps 7, 8, 9, and 10.  Save the output.

13. Repeat steps 7, 8, 9, and 10 again, saving the output.

14. Copy and save all of the on-screen output from the TSM console.

15. Restart TSM.

16. Login as instance owner, issue "db2support -d tsmdb1 -c -g -s", save
the db2support.zip generated

Once you have the PMR created, you can send in the doc that you collected:

1.The TSM server console output

2. The TSM server trace file

3. The three iterations of the "ps" output

4. The three iterations of pstack output against dsmserv

5. The three iterations of pstack output against db2sysc

6. The db2support.zip file

Best regards,

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Hartford/IBM@IBMUS
Internet e-mail: storman AT us.ibm DOT com

IBM Tivoli Storage Manager support web page:
http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Storage_Manager

"ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu> wrote on 2012-12-03
08:45:17:

> From: Zoltan Forray <zforray AT VCU DOT EDU>
> To: ADSM-L AT vm.marist DOT edu,
> Date: 2012-12-03 08:55
> Subject: Re: 6.3.3.000 server wont HALT
> Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu>
>
> This is now becoming a consistent / persistent problem.  I had to kill -9
> to stop the dsmserv process.  I restarted the server (via service ..
>  start) and there didn't seem to be any damage done.
>
> However, attempting to stop/halt it, again, produced the same result -
> dsmserv using 200% CPU and after 2-hours I had to kill -9.
>
> So, obviously there are big enough changes in 6.3.3 vs 6.3.2, to cause
> problems like this, since none of my 6.3.x or 6.2.x servers exhibit
> this behavior.
>
> Any suggestions on how to diagnose this "issue" before I contact IBM and
> open a PMR?
>
>
> On Thu, Nov 29, 2012 at 2:04 PM, Zoltan Forray <zforray AT vcu DOT edu> wrote:
>
> > Just did my first install/conversion of a 6.2.3 TEST server to
6.3.3.000
> > (RH Linux)
> >
> > While the install and startup went fine, it won't HALT.
> >
> > After the install/upgrade, I got in via dsmadmc just fine.  Checked the
> > actlog - saw all the schema changes/upgrades.  Updated/registered the
> > licenses and then issued HALT.  Got the usually warning and said YES.
> >
> > Now it has been sitting for >25-minutes since the halt.
> >
> > Can't get back in via dsmadmc.
> >
> > Top shows dsmserv using >200% CPU.
> >
> > I tried standard kills, with no luck.   I hate to do a kill -9 but will
if
> > I don't have a choice.
> >
> > What the heck is it doing?  Should I wait longer or just kill it with
> > extreme prejudice?
> >
> > --
> > *Zoltan Forray*
> > TSM Software & Hardware Administrator
> > Virginia Commonwealth University
> > UCC/Office of Technology Services
> > zforray AT vcu DOT edu - 804-828-4807
> > Don't be a phishing victim - VCU and other reputable organizations will
> > never use email to request that you reply with your password, social
> > security number or confidential personal information. For more details
> > visit http://infosecurity.vcu.edu/phishing.html
> >
> >
>
>
> --
> *Zoltan Forray*
> TSM Software & Hardware Administrator
> Virginia Commonwealth University
> UCC/Office of Technology Services
> zforray AT vcu DOT edu - 804-828-4807
> Don't be a phishing victim - VCU and other reputable organizations will
> never use email to request that you reply with your password, social
> security number or confidential personal information. For more details
> visit http://infosecurity.vcu.edu/phishing.html
>