ADSM-L

Re: [ADSM-L] 6.3.3.000 server wont HALT

2012-12-05 07:20:25
Subject: Re: [ADSM-L] 6.3.3.000 server wont HALT
From: Zoltan Forray <zforray AT VCU DOT EDU>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Wed, 5 Dec 2012 07:03:16 -0500
As I mentioned, initial startup showed messages about schema updates but
the server itself doesn't become available/accessible until after that is
completed.  During my many test startup/shutdowns after Andy's initial
response, I did see various messages about DB reorgs kicking in at various
times but all were finished/idle when I do the halt.

I have my "DR" server I am going to upgrade to 6.3.3.000 and I will do the
first startup interactively to monitor what happens and see if it exhibits
the same behaviour (and activate tracing, of course).


On Tue, Dec 4, 2012 at 11:29 AM, Paul Zarnowski <psz1 AT cornell DOT edu> wrote:

> Just a thought...  Is it possible that immediately after you upgrade a TSM
> server that there is some database conversion or reorganization that is
> getting done, that takes some time, and which cannot be interrupted?  And
> that this could delay a HALT until the HALT thread can acquire locks?  Just
> a wild-ass guess...
>
> At 11:16 AM 12/4/2012, Andrew Raibeck wrote:
> >Hi Zoltan,
> >
> >> It is now shutting down normally.
> >
> >This includes when you start the server normally, as a background process?
> >Interesting... at this point I don't have anything more I can offer. If
> you
> >encounter this again, I would say to go ahead and open a PMR. It is
> >possible that collecting the stack traces for one (next) occurrence will
> be
> >sufficient. We can better facilitate information exchange through the
> >formal support process.
> >
> >Meanwhile, if I hear anything more (such as a confirmed APAR number), I'll
> >post here.
> >
> >Best regards,
> >
> >Andy Raibeck
> >IBM Software Group
> >Tivoli Storage Manager Client Product Development
> >Level 3 Team Lead
> >Internal Notes e-mail: Andrew Raibeck/Hartford/IBM@IBMUS
> >Internet e-mail: storman AT us.ibm DOT com
> >
> >IBM Tivoli Storage Manager support web page:
> >
> http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Storage_Manager
> >
> >"ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu> wrote on 2012-12-04
> >10:58:29:
> >
> >> From: Zoltan Forray <zforray AT VCU DOT EDU>
> >> To: ADSM-L AT vm.marist DOT edu,
> >> Date: 2012-12-04 11:03
> >> Subject: Re: 6.3.3.000 server wont HALT
> >> Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu>
> >>
> >> Well, I don't get it.  It is now shutting down normally.  I have stopped
> >> and restarted it 10-times (this is a test server) with no issues.  The
> >CPU
> >> usage never spikes above 100% after I enter HALT.  Then it is stopped
> >> within 1-minute.
> >>
> >> I have another server I am going to upgrade from 6.2.3.0 to 6.3.3.000.
>  I
> >> will be sure to perform the first startup manually and capture the logs
> >and
> >> start tracing as soon as I can, just in case it hangs like this one did.
> >>
> >>
> >> On Mon, Dec 3, 2012 at 10:17 AM, Schneider, Jim
> ><jschneider AT ussco DOT com>wrote:
> >>
> >> > I meant 'dsmadmc -id=<id> -pass=<pass> -console' from a telnet/ssh
> >session.
> >> > It displays all console messages as they are generated.  I use it to
> >> > monitor the progress of long-running processes (deletion of
> filespaces,
> >> > macros that update volume access, etc.).
> >> >
> >> > Jim
> >> >
> >> > -----Original Message-----
> >> > From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On 
> >> > Behalf
> >Of
> >> > Zoltan Forray
> >> > Sent: Monday, December 03, 2012 8:12 AM
> >> > To: ADSM-L AT VM.MARIST DOT EDU
> >> > Subject: Re: [ADSM-L] 6.3.3.000 server wont HALT
> >> >
> >> > Not sure what you mean by "console session".  I use TSMManager and
> >issue
> >> > the halt via it's console/monitor window, if that is what you are
> >referring
> >> > to - and yes, it shows the halt is issued and then the server goes
> >> > non-responsive, as suspected.  Then from ssh/Linux the process dsmserv
> >> > never halts/stops.
> >> >
> >> >
> >> > On Mon, Dec 3, 2012 at 9:00 AM, Schneider, Jim <jschneider AT ussco DOT 
> >> > com>
> >> > wrote:
> >> >
> >> > > Have you tried a -console session prior to issuing halt?
> >> > >
> >> > > Jim Schneider
> >> > >
> >> > > -----Original Message-----
> >> > > From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On
> Behalf
> >> > > Of Zoltan Forray
> >> > > Sent: Monday, December 03, 2012 7:45 AM
> >> > > To: ADSM-L AT VM.MARIST DOT EDU
> >> > > Subject: Re: [ADSM-L] 6.3.3.000 server wont HALT
> >> > >
> >> > > This is now becoming a consistent / persistent problem.  I had to
> >kill
> >> > > -9 to stop the dsmserv process.  I restarted the server (via
> >service ..
> >> > >  start) and there didn't seem to be any damage done.
> >> > >
> >> > > However, attempting to stop/halt it, again, produced the same result
> >-
> >> > > dsmserv using 200% CPU and after 2-hours I had to kill -9.
> >> > >
> >> > > So, obviously there are big enough changes in 6.3.3 vs 6.3.2, to
> >cause
> >> > > problems like this, since none of my 6.3.x or 6.2.x servers exhibit
> >> > > this behavior.
> >> > >
> >> > > Any suggestions on how to diagnose this "issue" before I contact IBM
> >> > > and open a PMR?
> >> > >
> >> > >
> >> > > On Thu, Nov 29, 2012 at 2:04 PM, Zoltan Forray <zforray AT vcu DOT edu>
> >wrote:
> >> > >
> >> > > > Just did my first install/conversion of a 6.2.3 TEST server to
> >> > > > 6.3.3.000 (RH Linux)
> >> > > >
> >> > > > While the install and startup went fine, it won't HALT.
> >> > > >
> >> > > > After the install/upgrade, I got in via dsmadmc just fine.
>  Checked
> >> > > > the actlog - saw all the schema changes/upgrades.
> >> > > > Updated/registered the licenses and then issued HALT.  Got the
> >usually
> >> > warning and said YES.
> >> > > >
> >> > > > Now it has been sitting for >25-minutes since the halt.
> >> > > >
> >> > > > Can't get back in via dsmadmc.
> >> > > >
> >> > > > Top shows dsmserv using >200% CPU.
> >> > > >
> >> > > > I tried standard kills, with no luck.   I hate to do a kill -9 but
> >will
> >> > > if
> >> > > > I don't have a choice.
> >> > > >
> >> > > > What the heck is it doing?  Should I wait longer or just kill it
> >> > > > with extreme prejudice?
> >> > > >
> >> > > > --
> >> > > > *Zoltan Forray*
> >> > > > TSM Software & Hardware Administrator Virginia Commonwealth
> >> > > > University UCC/Office of Technology Services zforray AT vcu DOT edu -
> >> > > > 804-828-4807 Don't be a phishing victim - VCU and other reputable
> >> > > > organizations will never use email to request that you reply with
> >> > > > your password, social security number or confidential personal
> >> > > > information. For more details visit
> >> > > > http://infosecurity.vcu.edu/phishing.html
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> > > --
> >> > > *Zoltan Forray*
> >> > > TSM Software & Hardware Administrator
> >> > > Virginia Commonwealth University
> >> > > UCC/Office of Technology Services
> >> > > zforray AT vcu DOT edu - 804-828-4807
> >> > > Don't be a phishing victim - VCU and other reputable organizations
> >> > > will never use email to request that you reply with your password,
> >> > > social security number or confidential personal information. For
> more
> >> > > details visit http://infosecurity.vcu.edu/phishing.html
> >> > >
> >> > >
> >**********************************************************************
> >> > > Information contained in this e-mail message and in any attachments
> >> > > thereto is confidential. If you are not the intended recipient,
> >please
> >> > > destroy this message, delete any copies held on your systems, notify
> >> > > the sender immediately, and refrain from using or disclosing all or
> >> > > any part of its content to any other person.
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > *Zoltan Forray*
> >> > TSM Software & Hardware Administrator
> >> > Virginia Commonwealth University
> >> > UCC/Office of Technology Services
> >> > zforray AT vcu DOT edu - 804-828-4807
> >> > Don't be a phishing victim - VCU and other reputable organizations
> will
> >> > never use email to request that you reply with your password, social
> >> > security number or confidential personal information. For more details
> >> > visit http://infosecurity.vcu.edu/phishing.html
> >> >
> >> > **********************************************************************
> >> > Information contained in this e-mail message and in any attachments
> >> > thereto is confidential. If you are not the intended recipient, please
> >> > destroy this message, delete any copies held on your systems, notify
> >the
> >> > sender immediately, and refrain from using or disclosing all or any
> >part of
> >> > its content to any other person.
> >> >
> >>
> >>
> >>
> >> --
> >> *Zoltan Forray*
> >> TSM Software & Hardware Administrator
> >> Virginia Commonwealth University
> >> UCC/Office of Technology Services
> >> zforray AT vcu DOT edu - 804-828-4807
> >> Don't be a phishing victim - VCU and other reputable organizations will
> >> never use email to request that you reply with your password, social
> >> security number or confidential personal information. For more details
> >> visit http://infosecurity.vcu.edu/phishing.html
> >>
>
>
> --
> Paul Zarnowski                            Ph: 607-255-4757
> CIT Infrastructure / Storage Services     Fx: 607-255-8521
> 719 Rhodes Hall, Ithaca, NY 14853-3801    Em: psz1 AT cornell DOT edu
>



--
*Zoltan Forray*
TSM Software & Hardware Administrator
Virginia Commonwealth University
UCC/Office of Technology Services
zforray AT vcu DOT edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html