ADSM-L

[ADSM-L] Bad rman/TSM for Oracle restore performance

2012-10-19 15:39:18
Subject: [ADSM-L] Bad rman/TSM for Oracle restore performance
From: "Ehresman,David E." <deehre01 AT LOUISVILLE DOT EDU>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Fri, 19 Oct 2012 19:35:38 +0000
I have a Linux box running oracle and using rman/TSM for Oracle for backup and 
restores.

TDPO is 5.5.1.0.  The TSM client API is 6.3.0.0.  The TSM Server, running on 
AIX,  is 6.2.4.0.

Backups times are acceptable, approximated 250GB in an hour using two rman 
channels.  This drives the TSM server 1G Ethernet link at about 80MB/sec.

Restores are so slow I do not have a time yet for restoring the 250GB.  The 
linux server has no load; top shows:

top - 14:54:09 up  4:39,  8 users,  load average: 0.03, 0.02, 0.00
Tasks: 274 total,   1 running, 273 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.1%sy,  0.0%ni, 99.9%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  16463088k total, 16368212k used,    94876k free,   305156k buffers
Swap:  8388600k total,        0k used,  8388600k free, 15201560k cached

TSM "q sess f=d" commands against the two TDPO sessions issued every 15 seconds 
shows each thread in a SendW state that lasts about 3.5 minutes, then the 
client receives a chunk of data, then goes into another 3.5 minute SendW.  This 
repeats:

IBM Tivoli Storage Manager
Command Line Administrative Interface - Version 6, Release 2, Level 2.0
(c) Copyright by IBM Corporation and other(s) 1990, 2010. All Rights Reserved.

Session established with server ULTSM: AIX
  Server Version 6, Release 2, Level 4.0
  Server date/time: 10/19/12   14:53:02  Last access: 10/19/12   14:43:04

ANS8000I Server command: 'q sess 2508358 f=d'

              Sess Number: 2,508,358
             Comm. Method: Tcp/Ip
               Sess State: SendW
                Wait Time: 3.6 M
               Bytes Sent: 477.4 M
              Bytes Recvd: 693
                Sess Type: Node
                 Platform: TDPO LinuxAMD64
              Client Name: DBULSD4T
      Media Access Status: Current input volumes:  VR0742L3,(2615 Seconds)
                User Name:
Date/Time First Data Sent:
   Proxy By Storage Agent:
                  Actions: ObjRtrv


ANS8002I Highest return code was 0.

IBM Tivoli Storage Manager
Command Line Administrative Interface - Version 6, Release 2, Level 2.0
(c) Copyright by IBM Corporation and other(s) 1990, 2010. All Rights Reserved.

Session established with server ULTSM: AIX
  Server Version 6, Release 2, Level 4.0
  Server date/time: 10/19/12   14:53:02  Last access: 10/19/12   14:43:04

ANS8000I Server command: 'q sess 2508359 f=d'

              Sess Number: 2,508,359
             Comm. Method: Tcp/Ip
               Sess State: SendW
                Wait Time: 3.6 M
               Bytes Sent: 546.1 M
              Bytes Recvd: 693
                Sess Type: Node
                 Platform: TDPO LinuxAMD64
              Client Name: DBULSD4T
      Media Access Status: Current input volumes:  VR0746L3,(2615 Seconds)
                User Name:
Date/Time First Data Sent:
   Proxy By Storage Agent:
                  Actions: ObjRtrv


ANS8002I Highest return code was 0.

IBM Tivoli Storage Manager
Command Line Administrative Interface - Version 6, Release 2, Level 2.0
(c) Copyright by IBM Corporation and other(s) 1990, 2010. All Rights Reserved.

Session established with server ULTSM: AIX
  Server Version 6, Release 2, Level 4.0
  Server date/time: 10/19/12   14:53:17  Last access: 10/19/12   14:43:04

ANS8000I Server command: 'q sess 2508358 f=d'

              Sess Number: 2,508,358
             Comm. Method: Tcp/Ip
               Sess State: SendW
                Wait Time: 9 S
               Bytes Sent: 517.4 M
              Bytes Recvd: 693
                Sess Type: Node
                 Platform: TDPO LinuxAMD64
              Client Name: DBULSD4T
      Media Access Status: Current input volumes:  VR0742L3,(2630 Seconds)
                User Name:
Date/Time First Data Sent:
   Proxy By Storage Agent:
                  Actions: ObjRtrv


ANS8002I Highest return code was 0.

IBM Tivoli Storage Manager
Command Line Administrative Interface - Version 6, Release 2, Level 2.0
(c) Copyright by IBM Corporation and other(s) 1990, 2010. All Rights Reserved.

Session established with server ULTSM: AIX
  Server Version 6, Release 2, Level 4.0
  Server date/time: 10/19/12   14:53:17  Last access: 10/19/12   14:53:17

ANS8000I Server command: 'q sess 2508359 f=d'

              Sess Number: 2,508,359
             Comm. Method: Tcp/Ip
               Sess State: SendW
                Wait Time: 9 S
               Bytes Sent: 591.8 M
              Bytes Recvd: 693
                Sess Type: Node
                 Platform: TDPO LinuxAMD64
              Client Name: DBULSD4T
      Media Access Status: Current input volumes:  VR0746L3,(2631 Seconds)
                User Name:
Date/Time First Data Sent:
   Proxy By Storage Agent:
                  Actions: ObjRtrv


ANS8002I Highest return code was 0

So what is rman/oracle doing in the 3.5 minutes between its contacts with TSM?  
Any ideas for to fix this or at least how to troubleshoot it?

David Ehresman
University of Louisville

<Prev in Thread] Current Thread [Next in Thread>