ADSM-L

Re: [ADSM-L] Poor TSM server performance on Sun.

2007-05-14 17:43:23
Subject: Re: [ADSM-L] Poor TSM server performance on Sun.
From: Ben Bullock <bbullock AT MICRON DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Mon, 14 May 2007 15:42:38 -0600
No offence taken. I too have my own reservations about iSCSI for a TSM
DB and log. Unfortunately a remote site decided to put this
configuration in production before we could qual it and now I am
expected to wave my magic wand to diagnose and correct their performance
issue. I guess that is why we pay for that expensive TSM support.

        I'll let the list know if I ever figure out what the issue is.

Ben
 

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of
Kelly Lipp
Sent: Monday, May 14, 2007 3:35 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: Poor TSM server performance on Sun.

Ben,

Sorry!  Not trying to be a pain in the ass!

I guess I would worry about the overall performance since this is adding
another protocol to the mix: SCSI on TCP/IP, and a somewhat slower (and
that is the debate, isn't it) 1 Gb/sec path.

TSM is such an I/O pig (and I mean that in a good way) that I'm
skeptical about it being able to tolerate the additional overhead of
iSCSI.  I like the idea of using it for a remote copy pool, but for the
main data structures, DB and LOG, I'm thinking not.  I think it will
work, but I doubt that it will fast.

Now, back to you: if you learn differently, I'm all ears!  I would love
for this to work well as it will provide another good option for storage
in a TSM environment. 


Kelly J. Lipp
VP Manufacturing & CTO
STORServer, Inc.
485-B Elkton Drive
Colorado Springs, CO 80907
719-266-8777
lipp AT storserver DOT com

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of
Ben Bullock
Sent: Monday, May 14, 2007 3:30 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] Poor TSM server performance on Sun.

What gives you such pessimism about the configuration? Which piece of
the set-up do you think is the issue? The iSCSI? Sun OS? The alignment
of the planets? ;-)

Ben
 

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of
Kelly Lipp
Sent: Monday, May 14, 2007 1:38 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: Poor TSM server performance on Sun.

I'll be very interested in the outcome of this one!  I guess I'm not
surprised that it's slow.  I will be surprised if it ever works well.
But that's just me! 


Kelly J. Lipp
VP Manufacturing & CTO
STORServer, Inc.
485-B Elkton Drive
Colorado Springs, CO 80907
719-266-8777
lipp AT storserver DOT com

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of
Ben Bullock
Sent: Monday, May 14, 2007 12:01 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: [ADSM-L] Poor TSM server performance on Sun.

        Folks,
        I have a TSM server running on a Sun host running SOL 9, 16GB
RAM, and TSM 5.3.4 

        The problem is very poor performance, expire inventory goes
extremely slowly, even archives of 20GB files going to disk or tape are
very slow. Just slow slow slow.
        But all of the typical things I look at (the bufpool, log wait,
accounting logs, iostat, etc) look ok. No MediaW or IdleW, everything
seems to be in a "Run" state, but going slowly.

        All the storage (DB, LOG and STG) is on ISCSI out to an EMC
Clariion. The disk ~seems~ to be OK, but running a TSM DB and LOG on
iSCSI is a new configuration for us. At the OS side, it doesn't think it
is waiting for I/O (with an 'iostat' command), but I'm not sure if the
iSCSI protocol may be hiding the i/o waits from the OS, Any comments,
good or bad, from someone running TSM DB & LOG on iSCSI?


        I ran a little instrumentation on the host for less than a
minute and the final output looks like this:

TOTAL SERVER SUMMARY
Operation       Count  Tottime  Avgtime  Maxtime InstTput RealTput
Total KB
------------------------------------------------------------------------
----
Disk Read         435   58.100    0.134    0.963    281.2   1068.6
16340
Disk Write         97   14.632    0.151    0.504   1128.7   1080.2
16516
Tape Read           1    0.039    0.039    0.039
Tape Write        129   28.167    0.218    0.547   1154.2   2126.3
32512
Data Copy         127    0.080    0.001    0.000
Network Recv     2094  107.587    0.051    5.923    152.3   1071.5
16384
Network Send      198    0.021    0.000    0.000  39647.0     54.9
840
Acquire Latch      91   44.514    0.489    2.019
Acquire XLatch    359  148.127    0.413    4.327
Thread Wait      2192  106.077    0.048    5.958

Instrumentation output complete.


I'm going to open a performance case with Tivoli,  but it looks like
most of the time is spent in "Acquire Xlatch", anybody have an idea what
that is?

Any wild guesses are welcome.

Thanks
Ben