ADSM-L

Re: [ADSM-L] Expire Inventory Hangs?

2010-07-05 17:44:02
Subject: Re: [ADSM-L] Expire Inventory Hangs?
From: "Sheppard, Sam" <SSheppard AT SDDPC DOT ORG>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Mon, 5 Jul 2010 14:43:20 -0700
The two traces we were asked to provide are as follows:

You can enable instrumentation traces with the following commands in dsmadmc:
instrument begin
(gather instrumentation for about 30 minutes of DEL FI)
instrument end file=your_filename.txt

and

1) start "del fi ..." process if not already running (or the expire inventory)
2) do "show threads" to see which thread the delete fi is
3) do "trace enable DBSTMT"
4) do "trace begin /tmp/delfitrace.txt bufsize=4096"
5) Let run for only a few seconds due to size of trace file
6) do "trace end"

Another serious symptom we experienced was that after the DEL FI or EXP INVENT 
were running for a while on the problem filespace, the buffer pool hit ratio 
declined drastically from 98+% all the way down to ~65%.  Also, I misspoke as 
to the client causing the problem; it was Windows 7.

Sam Sheppard
San Diego Data Processing Corp.
(858)-581-9668

________________________________________
From: ADSM: Dist Stor Manager [ADSM-L AT VM.MARIST DOT EDU] On Behalf Of 
Christian Svensson [Christian.Svensson AT CRISTIE DOT SE]
Sent: Monday, July 05, 2010 12:29 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: [ADSM-L] SV: Expire Inventory Hangs?

Hi Sam,
Can you send over what trace flag you are using to debug Expire inventory?
Sometimes can you see a bullet point when you go though the Trace Logs.

Best Regards
Christian Svensson

Cell: +46-70-325 1577
E-mail: Christian.Svensson AT cristie DOT se
Skype: cristie.christian.svensson
Supported Platform for CPU2TSM:: 
http://www.cristie.se/cpu2tsm-supported-platforms

________________________________________
Från: Sheppard, Sam [SSheppard AT SDDPC DOT ORG]
Skickat: den 5 juli 2010 20:20
Till: ADSM-L AT VM.MARIST DOT EDU
Ämne: Re: Expire Inventory Hangs?

I have an open apar on a similar problem. We had a Win2008 client that backed 
up several million system objects (full backup was happening every day for a 
while) with 14 copies being kept. The client then stopped backing up the system 
objects and after 30 days all but the last copy were eligible for expiration. 
That's when we started having expiration problems. It seems the further into 
expiration on this node we got, the slower it ran.  We ended up excluding this 
node from expiration and everything was back to normal.

We were then asked by support to run a DELETE FILESPACE on the system in 
question and it produced the same results; ran ok for a while and then slowed 
to a crawl. Tivoli support had us take some traces and it appears this is a DB2 
bug. I have received a test fix which includes apar IC69156 that I'm going to 
install this week. This is projected to be fixes in 6.1.4.

I'll report back with the results.

Sam Sheppard
San Diego Data Processing Corp.
(858)-581-9668

________________________________________
From: ADSM: Dist Stor Manager [ADSM-L AT VM.MARIST DOT EDU] On Behalf Of Fred 
Johanson [Fred AT UCHICAGO DOT EDU]
Sent: Monday, July 05, 2010 7:01 AM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] Expire Inventory Hangs?

I have a similar one.  It's 4.5M objects with odd retention.  Everything before 
and after it in the expiration process completes in my 12 hour window.  It 
takes 2+ days.  When I see expiration hung, I cancel and restart without 
specifying duration.  It's a weekly procedure for me.


________________________________________
From: ADSM: Dist Stor Manager [ADSM-L AT vm.marist DOT edu] On Behalf Of 
Christian Svensson [Christian.Svensson AT CRISTIE DOT SE]
Sent: Monday, July 05, 2010 8:46 AM
To: ADSM-L AT vm.marist DOT edu
Subject: [ADSM-L] SV: Expire Inventory Hangs?

Hi,
Their is no backup or restore session active.
And the Expire inventory can hangs for over 24 hours without any problem on the 
same node and on the same filespace. So I,m guessing it is a corruption in the 
TSM Database.

Best Regards
Christian Svensson

Cell: +46-70-325 1577
E-mail: Christian.Svensson AT cristie DOT se
Skype: cristie.christian.svensson
Supported Platform for CPU2TSM:: 
http://www.cristie.se/cpu2tsm-supported-platforms

________________________________________
Från: Nick Laflamme [dplaflamme AT GMAIL DOT COM]
Skickat: den 5 juli 2010 15:22
Till: ADSM-L AT VM.MARIST DOT EDU
Ämne: Re: Expire Inventory Hangs?

On Jul 5, 2010, at 5:29 AM, Christian Svensson wrote:

> Hi,
> I have a TSM Server 6.1.3.4 running on a Windows Server 2008 R2.
> When I run Expire Inventory on my server it seams to hang on a unique node 
> and I think I have found the exact node it hangs on.
> My question is.
> How can I solve this? Is it possible to run a online audit on the database 
> now when the database is on DB2 or is it only offline audit availbile?

Does the client have a session open with the server while you're running EXPIRE 
INVENTORY? (That probably was one of the first things you checked, but you 
didn't say so, so I ask. :-) )

> Best Regards
> Christian Svensson

Nick
<Prev in Thread] Current Thread [Next in Thread>