ADSM-L

Re: [ADSM-L] Moving the TSM DB2 database (AIX)

2014-03-27 16:01:17
Subject: Re: [ADSM-L] Moving the TSM DB2 database (AIX)
From: "Colwell, William F." <bcolwell AT DRAPER DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Thu, 27 Mar 2014 19:59:09 +0000
Hi Kevin,

glad to be of help.

If you haven't seen it yet, you can setup schedules in the netapp cmd line
admin windows (maybe in a gui too, but I wouldn't know about that) for doing 
reallocates.

I looked for a script I made to do this, but I guess I deleted it when I got off
the netapp.

For comparison, and future planning, my databases are on raid 10 arrays, with 
15k 600 GB sas disks.

For a db on a 2 x 7 array, here are 5 dbb backup points.  The array is san 
attached.
I use 2 db streams, the output files are on sata raid.

Activity           Start Time       End Time         Elapsed (hh:mm:ss)         
   Gigs
--------------     ------------     ------------     -------------------     
----------
FULL_DBBACKUP      02-23-12.31      02-23-17.46      05:14:49                   
2338.08
FULL_DBBACKUP      03-02-09.38      03-02-13.02      03:24:30                   
2335.73
FULL_DBBACKUP      03-09-10.48      03-09-15.33      04:44:29                   
2437.75
FULL_DBBACKUP      03-16-11.11      03-16-14.46      03:35:18                   
2371.32
FULL_DBBACKUP      03-23-11.19      03-23-15.31      04:12:48                   
2363.04

And for a 2 X 3 array internal to a Dell r910 using 1 stream to sata raid -

Activity           Start Time       End Time          Elapsed Min           Gigs
--------------     ------------     ------------     ------------     ----------
FULL_DBBACKUP      02-23-11.00      02-23-12.40             100.6         817.73
FULL_DBBACKUP      03-02-11.00      03-02-12.25              85.9         772.45
FULL_DBBACKUP      03-09-11.00      03-09-12.27              87.6         774.07
FULL_DBBACKUP      03-16-11.00      03-16-12.33              93.0         789.22
FULL_DBBACKUP      03-24-13.53      03-24-15.21              88.2         795.65

Good Luck!

Bill Colwell
Draper Lab

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of 
Kevin Kettner
Sent: Thursday, March 27, 2014 12:34 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: Moving the TSM DB2 database (AIX)

Thanks a ton for that info!

This is hugely promising. My volumes are definitely fragmented. I have 5
200 GB vols that make up my DB volumes on my worst server. Four of them
were recommended for reallocation. I started a reallocate on one last
night and it's 80% done. I can already see a huge improvement in my DB
backup rate. I expect it will slow down to my normal rates when it hits
the next volume, but I can already see it's helping:

DB backup rate graph

The DB backup rate started out nearly twice as fast today.



On 3/26/2014 13:04, Colwell, William F. wrote:
> When I had TSM databases on Netapp - both v5 & v6 - I had to do frequent 
> netapp
> 'reallocate' commands to get the physical order in the netapp to match
> the logical order of db2.
>
> The db2 backup is reading the database sequentially, but within the netapp it
> is completely out of order.
>
> Try doing ' wafl scan measure_layout' to get a measure of the disorder.
>
> Bill Colwell
> Draper Lab
>
> -----Original Message-----
> From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf 
> Of Kevin Kettner
> Sent: Wednesday, March 26, 2014 1:24 PM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: Re: Moving the TSM DB2 database (AIX)
>
> That is what I'm leaning towards as well. I've got NetApp looking at the
> disk end to see if its getting hit hard. I also want to find out if the
> work load is heavier on reads or write (I'm guessing read) to know what
> sort of hardware fix is best for this, cache, flash, or more spindles, etc.
>
>   From Wanda's email, they're using a DS3512. I wouldn't expect that to
> be much different, performance wise, than the NetApp 3160 that I'm
> using. That leads me to think that maybe it's not the disk afterall...
>
> On 3/26/2014 11:30, Ehresman,David E. wrote:
>> Kevin,
>>
>> My gut reaction is that your disk drives can't feed the data fast enough.  
>> If it were me, I would open up a PMR to find out what the real bottleneck is.
>>
>> David Ehresman
>> University of Louisville.
>>
>> -----Original Message-----
>> From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf 
>> Of Kevin Kettner
>> Sent: Wednesday, March 26, 2014 11:34 AM
>> To: ADSM-L AT VM.MARIST DOT EDU
>> Subject: Re: [ADSM-L] Moving the TSM DB2 database (AIX)
>>
>> On the original question, I have moved DBs on AIX with LVM several times
>> with good success. The only real concern is the performance impact. The
>> benefit over mirroring is you can do it with no outage at all.
>>
>> I have 3 servers with similar sized DBs on AIX with NetApp SAS disk on
>> the back end, backing up to IBM 3592 drives, and my DB backups take 4-6
>> hours. I'm on 6.3.4 now and I've tried using more streams but that has
>> not made much difference.
>>
>> My smallest production DB is around 200 GB and it takes about an hour to
>> backup.
>>
>> I wonder what's going wrong. Do you have any advice?
>>
>> Thanks!
>>
>> On 3/21/2014 15:42, Prather, Wanda wrote:
>>> Was just thinking the same -
>>> It's only the conversion from V5 to V6 that takes forever.  Once you are 
>>> V6/DB2, DB backup-restore is fast again.
>>>
>>> I have TSM 6.3.4 on Windows, DB is 930G used, DS3512 disk, and it will back 
>>> up to LTO5 in 90 minutes if the server isn't doing a lot else at the time.  
>>> Restore takes maybe 15 minutes longer.
>>>
>>> You've got other issues you should address, if your DB backup is taking 
>>> many hours @ 300GB....
>>>
>>> W
>>>
>>>
>>> -----Original Message-----
>>> From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On 
>>> Behalf Of Ehresman,David E.
>>> Sent: Friday, March 21, 2014 9:11 AM
>>> To: ADSM-L AT VM.MARIST DOT EDU
>>> Subject: Re: [ADSM-L] Moving the TSM DB2 database (AIX)
>>>
>>> I've used migratepv to move oracle DBs around with no problems.  I would 
>>> not expect any issues with using LVM mirroring or migratepv to move the TSM 
>>> DB.  That is what I would do in your situation.
>>>
>>> But your comments about taking days to backup and restore your TSM DB 
>>> worries me.  How long does it take to backup your DB?  I have a 600G 
>>> allocated/415G used TSM DB.  It backs up in under an hour and restore time 
>>> is about the same.
>>>
>>> David Ehresman
>>> University of Louisville
>>>
>>> -----Original Message-----
>>> From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On 
>>> Behalf Of Roger Deschner
>>> Sent: Thursday, March 20, 2014 11:35 PM
>>> To: ADSM-L AT VM.MARIST DOT EDU
>>> Subject: [ADSM-L] Moving the TSM DB2 database (AIX)
>>>
>>> Now that TSM V5 is gone from our shop and we're all TSM V6.2, it's time to 
>>> move some things around. Such as the TSM DB2 database. The manual says to 
>>> do a full database backup and restore. That could take days of downtime 
>>> with our 150-300GB databases, and a lot of angst, so that is not really 
>>> acceptable.
>>>
>>> What I'm planning to do instead, is what I've always done on AIX. It's one 
>>> of the reasons I like AIX for hosting something like TSM. That is, to 
>>> basically walk the database over to the new location using AIX LVM 
>>> mirroring. All this with TSM up and running, albeit with a performance 
>>> impact. (It's Spring Break, so the performance impact is acceptable.) The 
>>> end result will be that the database has exactly the same Unix filesystem 
>>> names, path names, and file names as before, except that it will be on a 
>>> nice new faster disk subsystem.
>>>
>>> Other than the obvious performance impact while AIX LVM is doing the 
>>> mirroring, is there anything wrong with moving a TSM DB2 database by this 
>>> method? Anybody done this and had problems?
>>>
>>> Roger Deschner      University of Illinois at Chicago     rogerd AT uic DOT 
>>> edu
>>> ======I have not lost my mind -- it is backed up on tape somewhere.=====