I think I understand what is going on here. In the case of a filesystem you
stripe your data across a bunch of disks. But, when you define the files
you now have conflicting IO points on the disks. However, if you were to
use TSM volumes that fully filled each defined filesytem you will probably
do fine on performance given the filesystem is tuned correctly. In AIX
vmtune allows you to set the maxperm down so it will not conflict with the
memory required for the bufferpool. On Solaris, I do not know how to do
this with UFS+ file systesms or Veritas file systems, but I am sure someone
will chime in on that subject.
I also find that if you define all the volumes (files) simultaneously on a
filesytem you fragment the logical TSM volume all over the physical volumes
when what you probably want is them to have contiguous space. So, I define
them one at a time. This is particularly important on a high end disk
solution like an IBM ESS otherwise the sequential prestage can be disrupted
into looking like sequential data. Especially on the Database or the Log
because they typically write sequentially. Reads are mostly buffer hits
except during a backup. That is when you really take the hit.
File System overhead is an issue by itself, but what is probably causing
most file system implementation performance issues is the physical layout of
the data.
I have not figured out all of the gotchas for file systems yet, but I am
getting there. I will have to think about this some more. But, maybe this
will kindle some ideas from others.
Paul D. Seay, Jr.
Technical Specialist
Naptheon Inc.
757-688-8180
-----Original Message-----
From: Johnn D. Tan [mailto:jdtan AT MED.CORNELL DOT EDU]
Sent: Thursday, September 26, 2002 2:35 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: Disk volumes
Thanks Scott. Hmm, never saw anyone else mention this, though I've been on
the list for about a year. (Though, given the volume of this list, I
definitely could've missed some mails.)
Well, we are planning to move to new hardware and to TSM 5.1.1.x, so I guess
that's as good a time as any to try using raw volumes for DB/log/diskpool.
We definitely need any performance gains we can get as our daily automated
procedures run very late into the day.
Thanks again!
johnn
>John,
> We just when through setting up our disk staging pool with TSM
>5.1 on Solaris 8. We had 6x36G to work with.
>
> The initial config used large files on the filesystem. This is
>incredibly slow.
>
> We then configured Disk Suite to mirror the disks and we
>configured TSM to use the raw devices e.g. /dev/md/rdsk/d0.
>Performance was significantly better. So TSM saw 3x36. From what we
>saw, it seemed the mirroring affected the performance more than the
>lack of spindles. As others have said, TSM seems to be pretty good at
>spreading the data around so it minimizes contention on the spool
>volumes. I never tested more, smaller mirrors.
>
> We then gave TSM the raw slices /dev/rdsk/c1t0d0s0 so he saw
>6x36. This was the fastest by far. We were able to max the bandwidth
>(100M) for over an hour (38G in one hour).
>
> We lose the fault tolerance of mirrored disks, but we figure
>since it is only a staging area who cares? If a disk goes bad, we will
>lose the data backed up to it, but we can always back it up again. We
>felt the performance gains are well worth the redundancy hit. Though
>we have not tested pulling a disk from the staging pool and seeing what
>happens.
>
> I don't know your environment, but I would go with a single
>slice on all disks and tell TSM to use the raw device. If you really
>feel you need the redundancy I would just create 6 mirrors and use the
>raw mirrored device.
>
> From all of the benchmarking I've done it seems that once you
>get your setup decently tuned (don't tell TSM to use files for
>DB/LOG/DISKPOOLS) that the bottlenecks are either network capacity to
>your TSM server, or disk/cpu performance on the client (compression
>on). But I've only tested in one environment, ymmv.
>
>
> Hope this helps.
>
>scott
>
>
>Johnn D. Tan wrote:
>
>>I have 12 36-GB drives available for spool.
>>
>>Based on recommendations made to this list earlier this year, I went
>>with 12 mirrored disk spools of 16 GB each (keep in mind disk
>>overhead).
>>
>>As I understood it, the issue was you want many spools so that, as
>>Allen mentioned, you can have many threads for backups and even
>>migrations (assuming you have a good number of tape drives).
>>
>>However, you don't want so many spools per disk, otherwise there is
>>contention for head movement on the drive which would result in poorer
>>performance.
>>
>>johnn
>>
>>
>>>=> On Thu, 26 Sep 2002 08:54:01 -0400, Mahesh Tailor
>>><MTailor AT CARILION DOT COM> said:
>>>
>>>> Hopefully this is a simple question: I have fourteen 36GB drives
>>>>that are
>>>> available for the diskpool and I was wondering whether it is
>>>>better to have
>>>> seven 5GB files or three 10GB files or one 35GB file or something
>>>>else? The
>>>> drives are mounted in two IBM-2014 Ultra-Wide SCSI disk drawers
>>>>with
>>>> separate Ultra-Wide contollers. The other 14 drives are used for
>>>>DB, LOG,
>>>> and spare.
>>>
>>>
>>>You have a total of 28 spindles, 14 each on two busses, right?
>>>
>>>I'd suggest making a RAID-5 out of the fourteen free spindles, and
>>>then make the individual volumes "A reasonable size". What's a
>>>reasonable size? Uh... ;)
>>>
>>>I just did this with a drawer of 36G SSA, and I chose 10G volumes,
>>>because I have about a dozen (and growing) disk pools amongst which I
>>>need to divide
>>>things up.
>>>
>>>Even if you only have one or two disk pools, it's useful to have more
>>>than a few volumes per pool, because instantaneously only on thing
>>>can write to a
>>>volume at a time. So, for example, if you have 12 clients backing up,
>>>and one
>>>70G disk volume, there is contention for the thread controlling that one
>>>volume.
>>>
>>>So calculate the size so that you'll have as many volumes as you feel
>>>like keeping track of, but not many more than that.
>>>
>>>
>>>- Allen S. Rout
>>
>>
>>
>
>
>--
>Scott Walters
>Packet Pusher - "The world speaks IP"
>
>Mack Trucks, WHQ http://www.MackTrucks.com
>2100 Mack Boulevard Ph: 610.709.3728
>Allentown, PA 18103 Fx: 610.709.2809
|
