ADSM-L

Re: Dwindling Performance

2004-01-14 10:51:14
Subject: Re: Dwindling Performance
From: "Scott, Brian" <bscott AT EDS DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Wed, 14 Jan 2004 10:50:34 -0500
Andy,

Have you thought about using the TSM Journal Service? If you're building a
ton of directories/files but not backing up much the journal will cut down
on the processing and keep your sessions down to a minimum.

Just a thought...

Brian Scott
EDS - EOGDE
GM Distributed Management Systems Engineering
MS 3234
750 Tower Drive
Troy, MI  48098

* phone: +01-248-265-4596 (8-365)
* mailto:bscott AT eds DOT com




-----Original Message-----
From: Andy Carlson [mailto:andyc AT ANDYC.CARENET DOT ORG]
Sent: Wednesday, January 14, 2004 9:07 AM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: Dwindling Performance


Thanks for the quick response.

Expiration is not finishing.  Before the main backups start, it maybe
expires 200000 objects, but during the backup window it slows to a crawl.
It picks up some during the day when the backups and migrations are running,
but since we now have some 100 sessions not finished, its slow then too.

I didn't look at randomize, but these sessions are staying out there for
hours.  I will take a look at that today.

I currently have them doing an incrbydate every other day, and a full incr
the othter.

The cache hit ratio of the database is about 98.5%, but we have about 3.5GB
of memory in the cache.  I don't think I can go much higher, but I will try
it if I can.

P.S.  The TSMI clients are Windows and Netware, the TSMU are Unix and a
couple of VMS.

Thanks for the input.


Andy Carlson                                    |\      _,,,---,,_
Senior Technical Specialist               ZZZzz /,`.-'`'    -.  ;-;;,_
BJC Health Care                                |,4-  ) )-,_. ,\ (  `'-'
St. Louis, Missouri                           '---''(_/--'  `-'\_)
Cat Pics: http://andyc.dyndns.org/animal.html


On Wed, 14 Jan 2004, Roger Deschner wrote:

> I have posted many times in the past saying you should never do a
> database unload/reload to gain performance. But this just might be the
> one case where it might make sense - the remaining half of a split
> server. But before you do something that drastic, dangerous, and time
> consuming, look for the things that are easier to fix.
>
> My basic metric of whether or not you are in trouble is, how long does
> expiration take? If you start it daily, the closer it is to 24 hours
> running time, the closer you are to doomsday. Never-ending expiration
> is the classic symptom of TSM Server Meltdown.
>
> But on the other hand, if your expiration runs nice and fast, your
> server and its database are probably OK. Look to clients as the
> problem. They can't all squeeze in the door at once, so don't let them
> try. If they use the client-polling scheduler, how long is the backup
> window, and what is your setting for Schedule Randomization
> Percentage? Make it as high as possible - SET RANDOMIZE 50. This will
> also help if you are having any kind of a network bottleneck.
>
> Look at these clients on a micro level. About how much are they each
> actually backing up? If it's not much, then your theory might be
> right, that they are very busy downloading their lists of backed up
> files. In that case, load spreading will be the best thing you could
> do. You might consider a schedule where not every client does a full
> "Incremental" every night - perhaps they only do one every other night
> and on the other nights they do an "incrbydate" backup which is much
> faster, because it goes only by the timestamps in the file system.
>
> Not to ask the obvious, but what's your Database Cache Hit Percentage?
> (Q DB F=D) If it's below 99%, it needs help. Even (especially) a badly
> fragmented database will run a lot faster if you have it swimming in
> cache.
>
> Look at other differences between your two instances - are they
> basically different types of clients?
>
> Roger Deschner      University of Illinois at Chicago     rogerd AT uic DOT 
> edu
> ============The short fortuneteller who escaped from
> prison============= ======================was a small medium at
> large.======================
>
>
>
> On Tue, 13 Jan 2004, Andy Carlson wrote:
>
> >We are having terrible performance with one of our instances of TSM.
> >I have suspicions, but I want to hear what you guys say.  Here is
> >what we
> >have:
> >
> >2 instances of TSM - TSMI and TSMU (TSMI is the problem)
> >
> >TSM 5.2.1.1
> >AIX 51.ML4
> >RS/6000 P670 - 8 processors, 16GB memory
> >Fastt700 SAN
> >STK9840 Tape drives
> >
> >The Database is 85% of 88GB (with room to expand another 50GB or so).
> >
> >Right at this moment, we have 233 sessions with TSMI.  The backup
> >sessions grind to a halt for hours at a time, with nothing apparently
> >happening.  I suspect that the directory trees are being downloaded
> >and built, but not sure
> >
> >When we split TSMI and TSMU, we created the TSMU instance, and did a
> >full backup on all the servers that moved there.  The TSMI database
> >is a restored copy of the original database, with the TSMU stuff
> >deleted out.
> >
> >Any ideas would be greatly appreciated.
> >
> >
> >Andy Carlson                                    |\      _,,,---,,_
> >Senior Technical Specialist               ZZZzz /,`.-'`'    -.  ;-;;,_
> >BJC Health Care                                |,4-  ) )-,_. ,\ (  `'-'
> >St. Louis, Missouri                           '---''(_/--'  `-'\_)
> >Cat Pics: http://andyc.dyndns.org/animal.html
> >
>

<Prev in Thread] Current Thread [Next in Thread>