TSM Baclient failed 12

[zack]

Hello Gurus,

I have experience this error (schedule job ended with failed12 but the job actually completed with no error) after upgrade TSM Server to 7.1.7 (from 7.1.6) . Most of the clients impacted by this. Sometime admin session were impacted as well. All job were fine before TSM server upgrade.

TSM client scheduler will set a global return code
of 12 indicating the scheduled operation has failed if
the client initially experiences a network communication
failure but is able to successfully connect to the TSM
server on retry and complete the schedule.

The dsmsched.log will report messages indicating the
scheduled event is being retried:

Retrying scheduled event: TESTSCHED

The dsmerror.log will report messages indicating a
problem connecting to the TSM server:

ANS5216E Could not establish a TCP/IP connection with address
XXX. The TCP/IP error is 'Unknown error' (errno = 10065).
ANS4039E Could not establish a session with a TSM
server or client agent. The TSM return code is -50.

The TSM scheduled operation should not be flagged as failing
with RC 12 if the scheduler is able to establish a connection
within the schedule's startup window and complete successfully
without other errors.

This issue has been reported when TSM client systems are set
to Standby mode when the scheduled operation is to run.

Do anyone have clue where should I start to troubleshoot.

Thank you in advance.

 

[balirake]

What status code comes when you refire the same schedule?

 

[balirake]

Also share the actlog

 

[zack]

The actlog just showing the client session started and ended. Thats all.

From Actlog:


11/17/2016 04:00:07 ANR0403I Session 51111 ended for node TSMCLIENTNODENAME().
(SESSION: 22)
11/17/2016 04:00:07 ANR0403I Session 51110 ended for node TSMCLIENTNODENAME().
(SESSION: 22)
11/17/2016 04:00:08 ANR1959I Status monitor collecting current data at
04:00:08.
11/17/2016 04:00:15 ANR0406I Session 51117 started for node TSMCLIENTNODENAME(WinNT)
(Tcp/Ip 10.176.203.13(58536)). (SESSION: 51117)
11/17/2016 04:00:15 ANR0403I Session 51117 ended for node TSMCLIENTNODENAME(WinNT).
more... (<ENTER> to continue, 'C' to cancel)

(SESSION: 51117)
11/17/2016 04:00:17 ANR0406I Session 51118 started for node TSMCLIENTNODENAME(WinNT)
(Tcp/Ip 10.176.203.12(63350)). (SESSION: 51118)
11/17/2016 04:00:18 ANR0403I Session 51118 ended for node TSMCLIENTNODENAME(WinNT).
(SESSION: 51118)
11/17/2016 04:00:19 ANR0406I Session 51120 started for node TSMCLIENTNODENAME(WinNT)
(Tcp/Ip 10.176.203.13(58538)). (SESSION: 51120)



From dsmerror:

17-11-2016 04:01:14 ANS5216E Could not establish a TCP/IP connection with address 'TSMSERVERIP:1500'. The TCP/IP error is 'Unknown error' (errno = 10060).
17-11-2016 04:01:14 ANS9020E Could not establish a session with a TSM server or client agent. The TSM return code is -50.

From dsmsched.log

17-11-2016 04:00:19 --- SCHEDULEREC QUERY BEGIN
17-11-2016 04:00:19 --- SCHEDULEREC QUERY END
17-11-2016 04:00:19 Next operation scheduled:
17-11-2016 04:00:19 ------------------------------------------------------------
17-11-2016 04:00:19 Schedule Name: INC_DAILY_0400
17-11-2016 04:00:19 Action: Incremental
17-11-2016 04:00:19 Objects:
17-11-2016 04:00:19 Options:
17-11-2016 04:00:19 Server Window Start: 04:00:00 on 17-11-2016
17-11-2016 04:00:19 ------------------------------------------------------------
17-11-2016 04:00:19
Executing scheduled command now.
-------
17-11-2016 04:00:19 --- SCHEDULEREC OBJECT BEGIN INC_DAILY_0400 17-11-2016 04:00:00
17-11-2016 04:16:48 --- SCHEDULEREC STATUS BEGIN
17-11-2016 04:16:48 Total number of objects inspected: 177,720
17-11-2016 04:16:48 Total number of objects assigned: 109,540
17-11-2016 04:16:48 Total number of objects backed up: 1,260
17-11-2016 04:16:48 Total number of objects updated: 4
17-11-2016 04:16:48 Total number of objects rebound: 0
17-11-2016 04:16:48 Total number of objects deleted: 0
17-11-2016 04:16:48 Total number of objects expired: 130
17-11-2016 04:16:48 Total number of objects failed: 0
17-11-2016 04:16:48 Total number of subfile objects: 0
17-11-2016 04:16:48 Total number of bytes inspected: 41.00 GB
17-11-2016 04:16:48 Total number of bytes transferred: 1.47 GB
17-11-2016 04:16:48 Data transfer time: 29.94 sec
17-11-2016 04:16:48 Network data transfer rate: 51,492.58 KB/sec
17-11-2016 04:16:48 Aggregate data transfer rate: 1,559.86 KB/sec
17-11-2016 04:16:48 Objects compressed by: 0%
17-11-2016 04:16:48 Total data reduction ratio: 96.42%
17-11-2016 04:16:48 Subfile objects reduced by: 0%
17-11-2016 04:16:48 Elapsed processing time: 00:16:28
17-11-2016 04:16:48 --- SCHEDULEREC STATUS END
17-11-2016 04:16:48 --- SCHEDULEREC OBJECT END INC_DAILY_0400 17-11-2016 04:00:00
17-11-2016 04:16:48 ANS1512E Scheduled event 'INC_DAILY_0400' failed. Return code = 12.
17-11-2016 04:16:48 Sending results for scheduled event 'INC_DAILY_0400'.
17-11-2016 04:16:48 Results sent to server for scheduled event 'INC_DAILY_0400'.


17-11-2016 04:16:48 ANS1483I Schedule log pruning started.
17-11-2016 04:16:48 ANS1484I Schedule log pruning finished successfully.
17-11-2016 04:16:48 TSM Backup-Archive Client Version 7, Release 1, Level 0.3
17-11-2016 04:16:48 Querying server for next scheduled event.
17-11-2016 04:16:48 Node Name: TSMCLIENTNODENAME
17-11-2016 04:17:30 ANS1017E Session rejected: TCP/IP connection failure.
17-11-2016 04:17:30 Will attempt to get schedule from server again in 15 minutes.
17-11-2016 04:32:30 TSM Backup-Archive Client Version 7, Release 1, Level 0.3
17-11-2016 04:32:30 Querying server for next scheduled event.

 

[LED888]

What is the full version of the TSM Client that is backing up to the TSM Server 7.1.7 ?

Did the issue started when the TSM Server was upgraded to 7.1.7?

What happen when you do a manual backup?
What is the schedmode set to?

From the os that is hosting the TSM Client, attempt a connection via FTP to the os that is hosting the TSM Server. Does the FTP connect to the TSM Server hosting os?

Good Luck,
Sias

 

[marclant]

RC 12 means that there was a major error during the backup.

17-11-2016 04:01:14 ANS5216E Could not establish a TCP/IP connection with address 'TSMSERVERIP:1500'. The TCP/IP error is 'Unknown error' (errno = 10060).
17-11-2016 04:01:14 ANS9020E Could not establish a session with a TSM server or client agent. The TSM return code is -50.

That would be considered major if it wasn't able to connect to the server once. I'm sure if you spot those errors in the dsmsched.log, you will see it retried and connected again, otherwise the final stats/results would not have been sent to the server. So these are networking errors, the upgrade is just a coincidence.