So from this, it looks like session # 55705 would be the one we would want
to cancel in order to hopefully clear this out. So is the problem currently
happening? If it is, the next question is whether or not this session is
going to cancel. Let me know what happens. If it won't cancel let me know.
Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410
On Mon, May 2, 2011 at 10:27 AM, Zoltan Forray <zforray.vcu AT gmail DOT
com>wrote:
> Aha - that is why there was so much info. Here is just that piece plus the
> thread for the "parent" and his "parent"
>
> Thread 94, Parent 1: AcceptorThread, Storage 77938528, AllocCnt 420814
> HighWaterAmt 77938528
> tid=1116109120, ptid=1971136576, det=1, zomb=0, join=0, result=0, sess=0
>
> Thread 137116, Parent 94: psSessionThread, Storage 41352, AllocCnt 2464414
> HighWaterAmt 1424944
> tid=1203480896, ptid=1116109120, det=1, zomb=0, join=0, result=0,
> sess=55704
>
> Thread 137117, Parent 137116: XiSessionThread, Storage 406459533, AllocCnt
> 204215 HighWaterAmt 2147307969
> tid=1237166400, ptid=1203480896, det=1, zomb=0, join=0, result=0,
> sess=55705
> Awaiting cond xiMaster->cSent (0x0x2b787908e870), using mutex
> xiMaster->mutex
> (0x0xff84ae8), at xicomm.c(551)
>
> On Mon, May 2, 2011 at 1:14 PM, Dave Canan <ddcanan AT gmail DOT com> wrote:
>
> > There are no parameters on the "show threads" command; that is the
> complete
> > command. When you issue that command, do you see any lines that start
> with
> > "Thread 137117" ? That will be the output we are looking for. . Then
> look
> > for the "sess=" part of the output from that thread. That is the session
> we
> > are looking for.
> >
> >
> > Dave Canan
> > IBM ATS TSM Performance
> > ddcananATUSDOTIBMDOTCOM
> > 916-723-2410
> > >
> >
> > On Mon, May 2, 2011 at 9:50 AM, Zoltan Forray <zforray.vcu AT gmail DOT com>
> > wrote:
> >
> > > Dave,
> > >
> > > Thanks for the response.
> > >
> > > As I suspect, the oldest thread is from 03/07/2011. I have 2-IMPORT
> > > processes that hung (one started 03/07/2011 the other 04/19/2011)
> since
> > > the
> > > exporting server had a tape-mount problem and the exports died. I have
> > > long
> > > since tried to cancel the imports, to no avail. A "q proc" just shows
> > > "cancel in process".
> > >
> > > I wasn't sure of the proper syntax for the "show threads" command and
> > just
> > > used the ThreadID and used it. In this case:
> > >
> > > Start ThreadId=137117, Timestamp=03/07/2011 12:35:37 PM,
> > > (the oldest)
> > >
> > > and did a "show threads 137117"
> > >
> > > to get:
> > >
> > > -------------------------------------------------
> > > 12:34:20 PM SUN : show threads 137117
> > > Server PID: 8520
> > > Active threads: 126
> > > Zombie threads: 0
> > > Cached descriptors: 6
> > > Thread 1, Parent 0: main, Storage 6203696, AllocCnt 8342 HighWaterAmt
> > > 6203696
> > > tid=1971136576, ptid=<none>, det=0, zomb=0, join=0, result=0, sess=0
> > > Thread 2, Parent 1: TmDeadlockDetector, Storage 0, AllocCnt 0
> > HighWaterAmt
> > > 0
> > > tid=1086736704, ptid=1971136576, det=1, zomb=0, join=0, result=0,
> sess=0
> > > Thread 3, Parent 1: TbCacheMonitorThread, Storage 0, AllocCnt 2081959
> > > HighWaterAmt 0
> > > tid=1099311424, ptid=1971136576, det=0, zomb=0, join=0, result=0,
> sess=0
> > > Thread 4, Parent 1: AdmActivityLogThread, Storage 0, AllocCnt 20132093
> > > HighWaterAmt 114297
> > > tid=1105582400, ptid=1971136576, det=1, zomb=0, join=0, result=0,
> sess=0
> > > Awaiting cond descP->outReady (0x0x2b7879076ee0), using mutex
> > OUTV->mutex
> > > (0x0xb5de3e8), at output.c(3292)
> > > Thread 13, Parent 1: NaDiscoverThread, Storage 733, AllocCnt 14
> > > HighWaterAmt
> > > 37761
> > > tid=1100364096, ptid=1971136576, det=1, zomb=0, join=0, result=0,
> sess=0
> > > Awaiting cond NAV->checkConfig (0x0x2b78790770a0), using mutex
> > NAV->mutex
> > > (0x0xb7be7b8), at nadiscvr.c(2137)
> > > Thread 14, Parent 1: RemAgent, Storage 0, AllocCnt 0 HighWaterAmt 0
> > > tid=1102469440, ptid=1971136576, det=1, zomb=0, join=0, result=0,
> sess=0
> > > Awaiting cond MMSV->remCond (0x0x2b7879077180), using mutex
> MMSV->mutex
> > > (0x0x2aaab0009638), at mmsop.c(1409)
> > > Thread 15, Parent 1: PollAgent, Storage 0, AllocCnt 0 HighWaterAmt 0
> > > tid=1101416768, ptid=1971136576, det=1, zomb=0, join=0, result=0,
> sess=0
> > >
> > > (lot of snippage to not anger the listserv gods....................)
> > >
> > >
> > > Awaiting cond descP->outReady (0x0x2b78790900f0), using mutex
> > OUTV->mutex
> > > (0x0xb5de3e8), at output.c(3292)
> > > Thread 517131, Parent 517130: SmAdminCommandThread, Storage 253205,
> > > AllocCnt
> > > 4574 HighWaterAmt 321609
> > > tid=1120319808, ptid=1201375552, det=0, zomb=0, join=0, result=0,
> sess=0
> > > Holding mutex THRV->mutex (0x0xb5ea958), acquired at pkthread.c(2397)
> > > Server PID: 8520
> > > Active threads: 126
> > > Zombie threads: 0
> > > Cached descriptors: 6
> > >
> > > ----------------------------------------------------------------------
> > >
> > > I know there are fixes to import in 6.2.2.2 so I am going straight to
> > > 6.2.2.30
> > >
> > > On Mon, May 2, 2011 at 12:26 PM, Dave Canan <ddcanan AT gmail DOT com>
> > > wrote:
> > >
> > > > Zoltan, there is no "show logpinned" command in V6.2.2, but it is
> still
> > > > possible to figure out which session and transaction is causing the
> log
> > > to
> > > > be pinned. Here is an example of what you would need to do:
> > > > This isn't quite as straight-forward as the old show command, but
> this
> > > will
> > > > work. Call or re-post if you have questions.
> > > >
> > > >
> > > > 1. You need to start with doing a "show txnt" command. Here is an
> > example
> > > > of
> > > > one:
> > > >
> > > >
> > > > Session established with server xxxxxxx: AIX
> > > > Server Version 6, Release 2, Level 2.0
> > > > Server date/time: 05/02/11 12:09:43 Last access: 05/02/11
> > 12:09:30
> > > >
> > > >
> > > > tsm: xxxxxxx>show txnt
> > > >
> > > > Transaction hash table contents (slots=256):
> > > > slot -> 38:
> > > > Tsn=0:688678, Resurrected=False, InFlight=True, Distributed=False,
> > > > Persistent=True, Addr 113001ab8
> > > > Start ThreadId=17986, Timestamp=05/02/11 12:10:06,
> > > Creator=imdmgr.c(3552)
> > > > Last known in use by ThreadId=17986
> > > > Participants=1, summaryVote=ReadOnly
> > > > EndInFlight False, endThreadId 0, tmidx 0 0, processBatchCount 0,
> > > > mustAbort
> > > > False.
> > > > Participant DB: voteReceived=False, ackReceived=False
> > > > DB: Txn 11295a158, ReadOnly(YES), connP=1128ab1f8,
> > applHandle=23208,
> > > > openTbls=2:
> > > > DB: --> OpenP=115facf38 for table=Management.Classes.
> > > > DB: --> OpenP=116eba978 for table=Management.Class.Ids.
> > > > DB: --> RegSqlId=0x03000000 SELECT for table=Backup.Objects,
> > > > executed(Yes).
> > > > slot -> 39:
> > > > Tsn=0:688679, Resurrected=False, InFlight=True, Distributed=False,
> > > > Persistent=True, Addr 115646a78
> > > > Start ThreadId=17986, Timestamp=05/02/11 12:10:06,
> > > Creator=imdmgr.c(3673)
> > > > Last known in use by ThreadId=17986
> > > > Participants=2, summaryVote=ReadOnly
> > > > EndInFlight False, endThreadId 0, tmidx 0 0, processBatchCount 0,
> > > > mustAbort
> > > > False.
> > > > Participant DB: voteReceived=False, ackReceived=False
> > > > DB: Txn 112f855b8, ReadOnly(NO), connP=112ef88f8,
> > applHandle=23205,
> > > > openTbls=24:
> > > > DB: --> OpenP=1147eded8 for table=Activity.Summary.
> > > > DB: --> OpenP=118b7e038 for table=Group.Objects.ByLeader.
> > > > DB: --> OpenP=112fbdcf8 for table=Backup.Objects.
> > > > DB: --> OpenP=11300d3d8 for table=Group.Leaders.
> > > > DB: --> OpenP=11730b3b8 for table=AS.Volume.Status.
> > > > DB: --> OpenP=11307a7b8 for table=AF.Vol.Clusters.
> > > > DB: --> OpenP=11743acd8 for table=AF.Clusters.
> > > > DB: --> OpenP=117335198 for table=BF.Dereferenced.Chunks.
> > > > DB: --> OpenP=112fa2bb8 for table=DF.Segments.
> > > > DB: --> OpenP=113084658 for table=AS.Segments.
> > > > DB: --> OpenP=1160b65b8 for table=AF.Damaged.
> > > > DB: --> OpenP=112fe8b78 for table=BF.Bitfile.Extents.
> > > > DB: --> OpenP=115641438 for table=AF.Segments.
> > > > DB: --> OpenP=115f7d578 for table=AF.Bitfiles.
> > > > DB: --> OpenP=114536ad8 for table=DF.Bitfiles.
> > > > DB: --> OpenP=11303ff78 for table=BF.Aggregate.Attributes.
> > > > DB: --> OpenP=118c62eb8 for table=BF.Aggregated.Bitfiles.
> > > > DB: --> OpenP=1170581d8 for table=ARCH_EXPDIRS.
> > > > DB: --> OpenP=1147ec498 for table=Archive.Objects.
> > > > DB: --> OpenP=11604ddb8 for table=Management.Class.Ids.
> > > > DB: --> OpenP=116e91c98 for table=Management.Classes.
> > > > DB: --> OpenP=113034e98 for table=Filespaces.
> > > > DB: --> OpenP=1144d3e38 for table=Nodes.
> > > > DB: --> OpenP=114d1ce58 for table=Policy.Domain.Members.
> > > > Participant IM: voteReceived=False, ackReceived=False
> > > > Locks held by Tsn=0:688679 :
> > > > Type=19002(im node filespace), NameSpace=0, SummMode=xLock,
> > > Mode=xLock,
> > > > Key='154.1'
> > > > Type=19001(im node), NameSpace=0, SummMode=ixLock, Mode=ixLock,
> > > > Key='154'
> > > > Type=19000(im universe), NameSpace=0, SummMode=ixLock,
> Mode=ixLock,
> > > > Key=''
> > > > slot -> 165:
> > > > Tsn=0:692901, Resurrected=False, InFlight=True, Distributed=False,
> > > > Persistent=True, Addr 114532178
> > > > Start ThreadId=17984, Timestamp=05/02/11 12:10:07,
> > > Creator=imdmgr.c(2460)
> > > > Last known in use by ThreadId=17984
> > > > Participants=1, summaryVote=ReadOnly
> > > > EndInFlight False, endThreadId 0, tmidx 0 0, processBatchCount 0,
> > > > mustAbort
> > > > False.
> > > >
> > > > 2. What you're going to want to do is look at the lines that state
> > "Start
> > > > Threadid=nnnnn, Timestamp=". You need to find the oldest Timestamp.
> > That
> > > > will also give you the threadid. From there you do a "show threads"
> > > > command.
> > > > Here is a piece from that command :
> > > >
> > > > Thread 17980, Parent 42: psSessionThread, Storage 0, AllocCnt
> 490
> > > > HighWaterAmt
> > > > 174080
> > > > tid=253c, ptid=1e2a, det=1, zomb=0, join=0, result=0, sess=4612
> > > > Holding mutex OUTV->mutex (0x112018018), acquired at output.c(3267)
> > > >
> > > > Stack trace:
> > > > 0x0900000000595f30 _cond_wait_global
> > > > 0x0900000000596abc _cond_wait
> > > > 0x09000000005977ac pthread_cond_wait
> > > > 0x0000000100008414 pkWaitConditionTracked
> > > > 0x00000001000130f8 outGetNext
> > > > 0x0000000100870e58 SmAdminOutput
> > > > 0x000000010086ffb8 SmExecuteAdminCommand
> > > > 0x000000010086f888 SmAdminSession
> > > > 0x00000001003f123c DoAdminGeneral
> > > > 0x00000001003ec9bc smExecuteSession
> > > > 0x000000010009ad48 psSessionThread
> > > > 0x000000010001b688 StartThread
> > > >
> > > >
> > > > 3. Look for the thread Id for the longest running transaction in the
> > > first
> > > > step. This will also point you to the session matching that thread.
> > From
> > > > there, the session (that is the longest running transaction), can be
> > > > canceled.
> > > >
> > > >
> > > > Dave Canan
> > > > IBM ATS TSM Performance
> > > > ddcananATUSDOTIBMDOTCOM
> > > > 916-723-2410
> > > >
> > > >
> > > > On Fri, Apr 29, 2011 at 5:50 AM, Zoltan Forray <
> zforray.vcu AT gmail DOT com
> > > > >wrote:
> > > >
> > > > > Yes, I realize 6.x is DB2 but I was wondering is there is an
> > equivalent
> > > > to
> > > > > "show logpinned". I seem to be in some kind of situation where the
> > logs
> > > > are
> > > > > not being released and growing daily, eventhough I have performed
> > > > numerous
> > > > > full DB backups and backup volhist.
> > > > >
> > > > > I am going to have to bounce it soon, if I can't kill whatever is
> > > holding
> > > > > things up.
> > > > >
> > > > > My suspicion is a hung import. I have been moving numerous nodes
> > from
> > > an
> > > > > old 5.5 server and along the way at least 2-export/imports have
> > crashed
> > > > for
> > > > > various reasons and left zombie imports on the 6.2.2 server (yes I
> > > tried
> > > > > cancel proc with no results other than "cancel in progress")
> > > > >
> > > > > --
> > > > > Zoltan Forray
> > > > > TSM Software & Hardware Administrator
> > > > > Virginia Commonwealth University
> > > > > UCC/Office of Technology Services
> > > > > zforray AT vcu DOT edu - 804-828-4807
> > > > > Don't be a phishing victim - VCU and other reputable organizations
> > will
> > > > > never use email to request that you reply with your password,
> social
> > > > > security number or confidential personal information. For more
> > details
> > > > > visit
> > > > > http://infosecurity.vcu.edu/phishing.html
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Zoltan Forray
> > > TSM Software & Hardware Administrator
> > > Virginia Commonwealth University
> > > UCC/Office of Technology Services
> > > zforray AT vcu DOT edu - 804-828-4807
> > > Don't be a phishing victim - VCU and other reputable organizations will
> > > never use email to request that you reply with your password, social
> > > security number or confidential personal information. For more details
> > > visit
> > > http://infosecurity.vcu.edu/phishing.html
> > >
> >
>
>
>
> --
> Zoltan Forray
> TSM Software & Hardware Administrator
> Virginia Commonwealth University
> UCC/Office of Technology Services
> zforray AT vcu DOT edu - 804-828-4807
> Don't be a phishing victim - VCU and other reputable organizations will
> never use email to request that you reply with your password, social
> security number or confidential personal information. For more details
> visit
> http://infosecurity.vcu.edu/phishing.html
>
|
