job schedule failed rc=1 using rman

[merouan]

i planned a backup of oracle database using rman , i have 2 drives and when i check out the log rman here's the results :
channel t1: starting piece 1 at 15-JUL-15
RMAN-03009: failure of backup command on t1 channel at 07/15/2015 23:21:00
ORA-19502: write error on file "/SFC6PRD_data_hot_diff_216631_1/", blockno 16897 (blocksize=512)
ORA-27030: skgfwrt: sbtwrite2 returned error
ORA-19511: Error received from media manager layer, error text:
ANS1017E (RC-50) Session rejected: TCP/IP connection failure
channel t1 disabled, job failed on it will be run on another channel
channel t2: starting incremental level 1 datafile backupset
channel t2: specifying datafile(s) in backupset
............................................................................
Finished backup at 16-JUL-15

Starting Control File and SPFILE Autobackup at 16-JUL-15
piece handle=c-739160211-20150716-00 comment=API Version 2.0,MMS Version 5.4.1.0
Finished Control File and SPFILE Autobackup at 16-JUL-15

released channel: t1
released channel: t2
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================
RMAN-03002: failure of release command at 07/16/2015 05:03:35
RMAN-06012: channel: t1 not allocated

Recovery Manager complete.

here's the server side tsm option values CommTimeOut 14,400 IdleTimeOut 240
there's no adminstrative backup schedule in the same time of my database oracle backup

thanks for help

 

[LED888]

Is the TDP for Oracle a new installation?
Did the schedule backup worked before?
If so, what changed?

If we attempt a manual backup of Oracle, does the process complete or does it fail?

The system that is hosting the TDP and Oracle, is there a TSM B/A Client installed?
If so, does the schedule backup for the TSM B/A Client work?
If not, if we attempt a manual backup via the TSM B/A Client. Does the backup complete?

What are the messages in the following logs -

There may be two set of the following logs, one for the B/A Client and another set for the API.
Should be looking in the set for the API
If the B/A and the API are writing to the same log files, good luck on determine which app wrote the message.

dsmerror.log
dsmsched.log

dsierror.log <- This log may or may not exist.

tdpoerror.log

...tsm option values CommTimeOut 14,400 IdleTimeOut 240...

With the high time out values, did we review the TSM Server actlog to confirm that the limits
were not reach. Also, in the act log were there any message that we may have ran out of schedule sessions?

Good Luck,
Sias

 

[merouan]

tsm client ba and tdp for oracle in the same machine, and client ba work very well.
tdperror content
sessSendVerb: Error sending Verb, rc: -50
ANS1017E Session rejected: TCP/IP connection failure
ANS1017E Session rejected: TCP/IP connection failure
sessSendVerb: Error sending Verb, rc: -50
yes ites true , B/A AND tdp oracle writes in the same log files dsmerror.log dsmsched.log

 

[LED888]

BTW: What is the version of the TSM Server, TDP, TSM API and Oracle that we are backing up?

Historically the messages that we are seeing in the tdpoerorr.log are due to either a network issue or a time out issue.

...CommTimeOut 14,400 IdleTimeOut 240

Did we recently increase the time out values?
Or have these values been set some time ago?
If so, as a test. Increase the values, granted if we do not have enough sessions other nodes will not be able to connect to the TSM Server.

...yes ites true , B/A AND tdp oracle writes in the same log files dsmerror.log dsmsched.log

BTW: TDP Oracle writes to the tdpoerror.log. The TSM B/A Client and TSM API write to the dsmerror.log and dsmsch.log.
The TDP does not have a mechanism to communicate with the TSM Server, so it use the TSM API to communicate with the TSM Server.

Since we have two applications writing to the same logs, we have no idea if the messages are from the TSM B/A Client or from the TSM API Client.

For the TSM API, do suggest that you add the parameter errorlogname and schedlogname in the dsm.opt (Windows) or in the dsm.sys (Unix) so that the TSM API will write to its own logs.

Okay. so the TSM B/A Client backup complete.

Did we, or are we able to, perform a back up via RMan out side its schedule backup?

Does the dsierror.log exist? If so , are there any messages?

On the TSM Server, in the actlog around the time of when the time out occur in the tdpoerror.log are there any error message regarding the TDP node in question?



Good Luck,
Sias

 

[merouan]

TSM SERVER 6.3
TSM BA 5.5.2.7
TDP ORACLE:5.4.1.0
i recently increased the time out
Did we, or are we able to, perform a back up via RMan out side its schedule backup?
i cant perform a rman backup outside scheduled

when time out occure here's the actlog shows
ANR0492I All drives in use Session xxxxxxx for node XXXXXX (AIX) being preempted by higher priority operation. (SESSION: ******)

 

[LED888]

I'm sure that you are already aware of this, TSM 5.4 Server/Client are no longer support.
The TSM B/A and API Client 5.4 are not compatible with TSM Server 6.3 . Technically TSM B/A and API Client 5.X should work with TSM Server 6.3, but the results are unknown and unsupported by IBM. The TDP 5.4 does not depend on the version of the TSM Server as long as we meet the minimum requirement for the TSM API.

... ANR0492I All drives in use Session xxxxxxx for node XXXXXX (AIX) being preempted by higher priority operation. (SESSION: ******)

If the above ANR message occurred for the TDP Node, some higher process preempted the TDPO backup. Reset the timeout values back to what it was so other clients can connect with the TSM Server.

Your solutions:
- Temporary ignore the message, the next schedule back should run.

If we can not wait for the next schedule backup, run a manual backup via RMan.
If you can not run a manual backup via RMan, no choice to wait for the next schedule backup to run and see if the TDP schedule backup will complete. The higher process may have been temporary.
If we have a higher process that keep preempting the TDP backup, need to find out what higher process that is running at the same time as the schedule TDP backup.
Once the higher process is id, set up an admin schedule to run the higher process before the TDPO schedule backup or adjust the TDPO schedule to run early or later than normal.
​

- Disable preemtion on the TSM Server.

Higher process will have to wait until there is a tape drive available.
With the exception, backup db will still be able to preempt lower process.
​

- Add more tape drives to the tape library.

Good Luck,
Sias