Ben, please let me know what the PMR number is when you open it with
Tivoli. I will take a look at the trace for the problem. Can you cut and
paste more of the trace in the post, or send it to me at the IBM email
address below?

Also, the Acquie Xlatch category of the server trace is seen when an
exclusive lock has to be gotten for a database page. When a database page
needs to read only, this is placed in the Acquire Latch category in the
server trace. When a database page needs to be UPDATED, then this is placed
in the Acquire XLatch category. This would be common to see if expiration
was running on the TSM server.

At 12:00 PM 5/14/2007 -0600, you wrote:

        Folks,
        I have a TSM server running on a Sun host running SOL 9, 16GB
RAM, and TSM 5.3.4

        The problem is very poor performance, expire inventory goes
extremely slowly, even archives of 20GB files going to disk or tape are
very slow. Just slow slow slow.
        But all of the typical things I look at (the bufpool, log wait,
accounting logs, iostat, etc) look ok. No MediaW or IdleW, everything
seems to be in a "Run" state, but going slowly.

        All the storage (DB, LOG and STG) is on ISCSI out to an EMC
Clariion. The disk ~seems~ to be OK, but running a TSM DB and LOG on
iSCSI is a new configuration for us. At the OS side, it doesn't think it
is waiting for I/O (with an 'iostat' command), but I'm not sure if the
iSCSI protocol may be hiding the i/o waits from the OS, Any comments,
good or bad, from someone running TSM DB & LOG on iSCSI?


        I ran a little instrumentation on the host for less than a
minute and the final output looks like this:

TOTAL SERVER SUMMARY
Operation       Count  Tottime  Avgtime  Maxtime InstTput RealTput
Total KB
------------------------------------------------------------------------
----
Disk Read         435   58.100    0.134    0.963    281.2   1068.6
16340
Disk Write         97   14.632    0.151    0.504   1128.7   1080.2
16516
Tape Read           1    0.039    0.039    0.039
Tape Write        129   28.167    0.218    0.547   1154.2   2126.3
32512
Data Copy         127    0.080    0.001    0.000
Network Recv     2094  107.587    0.051    5.923    152.3   1071.5
16384
Network Send      198    0.021    0.000    0.000  39647.0     54.9
840
Acquire Latch      91   44.514    0.489    2.019
Acquire XLatch    359  148.127    0.413    4.327
Thread Wait      2192  106.077    0.048    5.958

Instrumentation output complete.


I'm going to open a performance case with Tivoli,  but it looks like
most of the time is spent in "Acquire Xlatch", anybody have an idea what
that is?

Any wild guesses are welcome.

Thanks
Ben

Dave Canan
TSM Performance
IBM Advanced Technical Support
ddcanan AT us.ibm DOT com