This seems to be the cause. I see the same diagnostics.
IC48429: MIGRATE STGPOOL PREVENTS UPDATE STGPOOL FROM RUNNING AND HANGS
SUBSEQUENT QUERY VOLUME AND SELECT FROM VOLUMES COMMANDS.
APAR status
Closed as program error.
Error description
A MIGRATE STGPOOL command prevents UPDATE STGPOOL command from
running at the same time. If the UPDATE STGPOOL command is
issued while the MIGRATE STGPOOL is running, it will also
prevents QUERY VOLUME and SELECT FROM VOLUMES commands from
running as well.
A show lock command will show 1 transaction holding lock type
34000 with other transactions waiting for the same lock.
For example :
slot -> 79:
LockDesc: Type=34000, NameSpace=0, SummMode=sLock, Key=''
Holder: Tsn=0:1.4265401449, Mode=sLock
Waiter: Tsn=0:1.4265407080, Mode=ixLock
Waiter: Tsn=0:1.4265407084, Mode=sLock
Waiter: Tsn=0:1.4265407118, Mode=sLock
Waiter: Tsn=0:1.4265407433, Mode=sLock
Waiter: Tsn=0:1.4265407491, Mode=sLock
In the above output, transaction Tsn=0:1.4265401449 is holding
lock type 34000. A show txnt output will reveal for example :
slot -> 105:
Tsn=0:1.4265401449, Resurrected=False, InFlight=True,
Distributed=False, Addr 82a778f8
ThreadId=67, Timestamp=01/16/06 15:00:27,
Creator=admcmd.c(1864) Participants=1, summaryVote=ReadOnly
EndInFlight False, endThreadId 67, tmidx 0 0,
processBatchCount 0, mustAbort False.
Participant DB: voteReceived=False, ackReceived=False
Locks held by Tsn=0:1.4265401449 :
Type=34000, NameSpace=0, SummMode=sLock, Mode=sLock, Key=''
A show thread will show that the corresponding thread
( thread 67 in this case) belongs to the MIGRATE STGPOOL
command. For example :
Thread 67: CsRunCmdThread
tid=17183, ktid=2384073, ptid=62, det=1, zomb=0, join=0,
result=0, sess=0
Awaiting cond pCtrl->allDone (0x18a5447e0),
using mutex pCtrl->mutex
(0x186dcf6d8), in AdmMigrateStgPool (0x1007b3e74)
Stack trace:
0x09000000003848fc _cond_wait_global
0x090000000038530c _cond_wait
0x0900000000385d6c pthread_cond_wait
0x000000010000d91c pkWaitCondition
0x00000001007b3e7c AdmMigrateStgPool
0x00000001001d9728 AdmCommandLocal
0x00000001001da668 admCommand
0x0000000100547b34 SmExecScheduledCommand
0x0000000100547d4c smScheduledConsoleSession
0x0000000100545510 CsRunCmdThread
0x000000010000e9dc StartThread
0x090000000036f460 _pthread_body
TSM Versions Affected:
TSM 5.3 Servers on all platforms.
Additional Keywords:
hang hung wait performance lock conflict SS_LOCK_UNIVERSE
AdmMigrateStgPool AdmUpdateStgPool
Local fix
Do not run the UPDATE STGPOOL command while the MIGRATE STGPOOL
command runs.
Problem summary
****************************************************************
* USERS AFFECTED: All users of TSM 5.3.x servers that use *
* command migration with WAIT=YES option *
* enabled. *
****************************************************************
* PROBLEM DESCRIPTION: See Error Description. *
****************************************************************
* RECOMMENDATION: Apply fixing level when available. *
* This problem is currently projected to be *
* fixed in level 5.3.3. Note that this is *
* subject to change at the discretion of *
* IBM. *
****************************************************************
Due to lock contention, MIGRATE STGPOOL WAIT=YES command
could prevent the other commands from running for
the duration of the migration.
Problem conclusion
The problem has been fixed.
Temporary fix
Comments
APAR information
APAR number IC48429
Reported component name TSM SERVER
Reported component ID 5698ISMSV
Reported release 53A
Status CLOSED PER
PE NoPE
HIPER NoHIPER
Submitted date 2006-01-24
Closed date 2006-02-13
Last modified date 2006-02-13
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
PK19694 <http://www-1.ibm.com/support/docview.wss?uid=swg1PK19694>
Modules/Macros
Fix information
Fixed component name TSM SERVER
Fixed component ID 5698ISMSV
Applicable component levels
R53A PSY UP
R53H PSY UP
R53L PSY UP
R53S PSY UP
R53W PSY UP
Rate this page
Orville L. Lantto
Glasshouse Technologies, Inc.
________________________________
From: ADSM: Dist Stor Manager on behalf of Rainer Wolf
Sent: Wed 3/1/2006 3:01 AM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] Q STG hangs during reclamation
Hi,
shortly had the same effect. There are various reasons for this behaviour -
often hardware-related
sometimes maybe just software.
Our last 'hang of query stg ' results from a filesystem that was offline
because of a defect FC-Adapter - so
first I would check ( on os-level ) if all
related disk-hardware -filesystems or raw partitions- are ok and really usable.
In tsm you may also check the client events of the last 24 hours.
You should call ibm for support.
Greetings
Rainer
Orville Lantto wrote:
>
> We are having problems on a TSM server instance on AIX. Some simple commands
> such as "Q Stg" appear to hang, along with their sessions, while reclamation
> is in process. Other commands like "Q Pr' do not hang. The server is AIX
> 5.2.5 with six instances of TSM Server 5.3.2.0 on it. The disk is DS4300
> with a 3584 tape library shared between the instances.
>
> Anyone have a guess as to why?
>
> Orville L. Lantto
> Glasshouse Technologies, Inc.
> Cell: 952-738-1933
>
>
> ________________________________
>
> From: ADSM: Dist Stor Manager on behalf of Prather, Wanda
> Sent: Tue 2/28/2006 4:50 PM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: [ADSM-L] 3584 help
>
> Having a problem with the Tape Library Specialist/web app, so I want to
> know what level of microcode is on this 3584.
>
> Can you get that information from the web app, or for that matter, from
> the front LED panel of the 3584?
>
> Thanks!
--
------------------------------------------------------------------------
Rainer Wolf eMail: rainer.wolf AT uni-ulm DOT de
kiz - Abt. Infrastruktur Tel/Fax: ++49 731 50-22482/22471
Universitaet Ulm wwweb: http://kiz.uni-ulm.de
|
