ADSM-L

Re: [ADSM-L] TDP Oracle issue: thousands of near simultaneous sessions per node

2013-05-29 18:07:59
Subject: Re: [ADSM-L] TDP Oracle issue: thousands of near simultaneous sessions per node
From: Richard Cowen <rcowen AT CPPASSOCIATES DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Wed, 29 May 2013 22:06:03 +0000
Rick,

I have seen something similar recently.  More than 1 million client sessions in 
24 hours from a single DP Oracle node.  My 32-bit perl script ran out of 
memory- a 64-bit version handled it...
I haven't investigated any further, just a lot of ANR0406I/ANR0403I pairs. A 
few hundred sessions actually sent data.

Richard


-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of 
Rick Adamson
Sent: Wednesday, May 29, 2013 2:44 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] TDP Oracle issue: thousands of near simultaneous sessions 
per node

Ricky,
Obviously there are multiple scripts each scheduled at different times, but 
appears that each one allocates one channel.
The TSM server has max sessions set at 300, and there have been no 
modifications to the defaults for resource utilization on the client.

Thanks,
~Rick


-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of 
Plair, Ricky
Sent: Wednesday, May 29, 2013 1:03 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] TDP Oracle issue: thousands of near simultaneous sessions 
per node

Rick,
How many channels do the DBA have set in RMAN to use for the backups. Also how 
many max sessions are set up on the server node and  resourceutilization in the 
dsm.sys file on the client

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of 
Rick Adamson
Sent: Wednesday, May 29, 2013 11:50 AM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: [ADSM-L] TDP Oracle issue: thousands of near simultaneous sessions per 
node

Looking for some direction...

I recently inherited a TSM server along with many TDP for Oracle nodes.
The TSM server is 5.4 (upgrade on the horizon).
The Oracle clients run on mostly AIX, but some Suse Linux, the TDP client is 
ver.5.5.x and the BA clients are primarily 6.2.

All of the database and log backups are initiated by CRON and when they start a 
single Oracle node will start and stop literally thousands of sessions on the 
TSM server within a minute or two.

Oracle and RMAN are new to me and I have talked with our DBA team regarding the 
problem but to no surprise they imply it is a TSM issue.
When this occurs the obvious impact to the TSM server is devastating, 
performance is crushed and the console churn is a blur.

In reading the Oracle and TDP documentation I am suspicious of the RMAN scripts 
as the cause, but before I go testing changes or opening a support case, I 
thought I would ask the group.

The RMAN scripts do perform several "crosschecks" during the process and I 
suspect that each backup piece it queries for starts and stops a client session 
on the TSM server.

Has anyone seen this issue before? Is there any way to get the TDP client to 
perform all operations during a single, or minimal,  sessions?
I have included the RMAN script I suspect below.....

All feedback welcome,
~Rick

===================================================

rman target backup/sqbkup3 catalog rman10/sqbkup3@rcat1_catalog.world <<EOF #> 
/opt/oracle/log/rman_full_csp1.log 2>&1 run { allocate channel tdp1 type 
'sbt_tape'
parms 'ENV=(TDPO_OPTFILE=/opt/tivoli/tsm/client/oracle/bin64/tdpo.csp1.opt)';
setlimit channel tdp1 kbytes 1073741824 maxopenfiles 32 readrate 200;

sql 'alter system archive log current';
resync catalog;
change archivelog all validate;
backup database include current controlfile format 'df_%t_%s_%p'; backup 
archivelog all delete input format 'al_%t_%s_%p'; release channel tdp1; } EOF #

rman <<EOF #> /opt/oracle/log/backup_delete_csp1.log 2>&1 connect target 
backup/sqbkup3 connect catalog rman10/sqbkup3@rcat1_catalog.world
allocate channel for delete type 'sbt_tape'
parms 'ENV=(TDPO_OPTFILE=/opt/tivoli/tsm/client/oracle/bin64/tdpo.csp1.opt)';
configure retention policy to recovery window of 14 days; list backup of 
database; list backup of database summary; list backup of archivelog all 
summary; crosscheck backup; delete noprompt obsolete; delete noprompt expired 
backup; crosscheck backup; release channel; allocate channel for delete type 
disk; crosscheck backup; delete noprompt obsolete; delete noprompt expired 
backup; crosscheck backup; list backup of database; EOF

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
CONFIDENTIALITY NOTICE: If you have received this email in error, please 
immediately notify the sender by e-mail at the address shown.This email 
transmission may contain confidential information.This information is intended 
only for the use of the individual(s) or entity to whom it is intended even if 
addressed incorrectly. Please delete it from your files if you are not the 
intended recipient. Thank you for your compliance.

<Prev in Thread] Current Thread [Next in Thread>