ADSM-L

Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

2011-04-18 12:26:28
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
From: Nancy L Leugemors <Leugemors.Nancy AT HEALTHNOW DOT ORG>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Mon, 18 Apr 2011 12:20:07 -0400
Sorry Paul, Just seeing your reply in the thread..

We had no issues with recovery log pinning at 5.5.4.0 and now are
experiencing them at 5.5.5.0,but will put in to upgrade the TSM Server to
go to .5.5.5.2 for the IC58852 next week.   I want to let this code run
this week and I need to finish our DD Clean Process to regain Data Domain
Space back.



Did fix for us in the 5.5.5.0 Code:

Going from 5.5.4.0 to 5.5.5.0 did fix the deadlock issues for us when you
ran a delete volhist=dbb and had the relabelscratch=yes turned on.


Tivoli Storage Manager server may encounter a deadlock situation
when delete volhistory and label libvol activity are
running at the same time.
Show output at the time of this problem reveals:
>From show resqueue - waiters for type=77778
>From show locks - waiting for lock type=77778



http://www-01.ibm.com/support/docview.wss?uid=swg1IC67947







Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979



From:
Paul van Dongen <Paul.vanDongen AT VANCIS DOT NL>
To:
ADSM-L AT VM.MARIST DOT EDU
Date:
04/18/2011 03:33 AM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
"ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>



Hello,

I don't know the behavior on 5.5.5 (we went straight to 5.5.5.2 because of
the SQL SELECT bug), but on 5.5.4 we had a lot of issues with logs being
pinned due to IC65781.

Problems have disappeared since the upgrade.


Paul

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of
Bob Booth
Sent: zondag 17 april 2011 16:17
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0
code

On Sat, Apr 16, 2011 at 11:41:13PM -0400, Nancy L Leugemors wrote:
> Hello,
>
> We just upgraded our TSM Server this  from 5.5.4.0 to 5.5.5.0  to fix
APAR
>  IC66116.   Since the server upgrade we have experienced  several
recovery
> log pinning incidents from a few different database backup clients.   I
> haven't found any hits on new bugs related to our issue and I just
opened
> up a case with support but was wondering if anyone had a similar issue
> with this level of server and client code or know of any APARS this
might
> fit?   Our recovery log has always been set to 13GB so I can't expand it
> anymore.     Running a database backup doesn't free any space while the
> client database backup is running either.   We run 1 full TSM database
> backup and multiple incrementals throughout the day.   The only
difference
> is the upgrade to the latest code.   I'm not seeing any other errors,
just
> showing different database clients pinning the log after issuing show
log
> pinned command.   Sometimes the client backup finishes before hitting
80%
> and sometimes TSM server cancels the longest running backup that is
> pinning the log.

We are still at 5.5.4, and have seen more instances of log pins and
database
lock conflicts, however, look to see if you have any of these factors:

Increased the size of your database, possibly to sub-optimal
disks/controllers,
or RAID devices.

Longer running expirations, and or additions of systems that may have an
increased number of files (say in the millions).

Change in client levels on some nodes, due to compatibility problems (or
some
other issue).  Mac's and TDP's come to mind. --

Our database is in an almost constant state of backup, as we use
rollforward
recovery.  I would be most interested in anything that IBM support says,
so
pass on the good word if you hear anything.  We are not quite ready for V6
yet, since we have to stand up new infrastructure, and get prepared for
that
whole other set of problems.

The suggestion that you back your log off to 12GB is a good one, since you
will very screwed if you fill up the log and fall over.

My questions for the list are this,

Would it help to get out of rollforward recovery mode and just to periodic
database backups at some standard hour?  I know the reason for
rollforward, but
I'm willing to live with the losses if it fixes this problem.

Is there any way to stop the server from killing the oldest transaction?
It
never seems to be the one that is pinning the log, and all it does is
force
the long running jobs to start over, usually making the issue worse.  We
do
a lot of image backups which take several hours, and it would be nice to
just
allow them to finish and get it over with.

Is there a fix for the error message about the log transaction delay?  I
saw
an APAR that showed the 3ms text is bogus, and is actually 1s.   This
should
have been fixed before my current level.

TSM: 5.5.4.0
AIX 5.3 - 11
TSM clients 5.5.X - 6.X
DB size 243GB
Log size 12GB

Good luck.

> example of error:
>                           29857)
> 04/16/11   04:03:26      ANR0524W Transaction failed for session 24418
for
> node
>                           DEVNODE_API (SQL-BACKTRACK) -  data transfer
> interrupted.
>                           (SESSION: 24418)
> 04/16/11   04:03:26      ANR2997W The server log is 81 percent full. The
> server
>                           will delay transactions by 3 milliseconds.
> (SESSION:
>                           29846)
>
>
> TSM Background:
>
> TSM Server:  5.5.5.0
> TSM OS:  AIX 5300 -12
> TSM Clients:  5.5.2.0
> TSM Client OS:  AIX 5.3 (UDB-DB2 & SQL-Backtrack Sybase seen so far)





CONFIDENTIALITY NOTICE: This email message and any attachments are for the sole 
use of the intended recipient(s) and may contain proprietary, confidential, 
trade secret or privileged information.  Any unauthorized review, use, 
disclosure or distribution is prohibited and may be a violation of law.  If you 
are not the intended recipient or a person responsible for delivering this 
message to an intended recipient, please contact the sender by reply email and 
destroy all copies of the original
message.

<Prev in Thread] Current Thread [Next in Thread>