Move DRM question

[chad_small]

This is more of a complaint than a question and needs to be added to TSM.

Why doesn't the MOVE DRMEDIA command have the option for WHERESTATUS=FULL ???

I know you want to get the data offsite ASAP but in some cases you are reclaiming the tape almost as soon as its sent offsite due to the lack of data on the tape.

 

[heada]

You shouldn't be reclaiming the tape unless you're expiring data from that tape. Reclamation works from %reclaimable and not %full.

-Aaron

 

[KB]

It doesn't make sense to hold your offsite backups just because the partially filled tapes get reclaimed almost as soon as they go offsite. You'll regret it big time in a DR. So what can be done? I'll suggest a couple things:
- Group collocation: try to get the amount of data per tape higher.
- Don't be so aggressive with your reclaim point, otherwise you're just churning data.
- Disk only primary pools for clients with small amounts of data. Recover the whole disk pool in a DR from non-collocated tapes. Reclaims involving disk to non-collocated tape are relatively quick.

 

[JohanW]

Because you want to get the data offsite ASAP.

You will be reclaiming it, but you did backup the database before offsiting those not-quite-empty tapes didn't you? If you restore that database backup, those not-quite-empty tapes won't have been reclaimed yet, and they are delayed from returning to scratch, right? They're still offsite, pending.

If there's not much data on them, reclamation won't take long, and tapes are cheap. Well, relatively. Compared to a disaster.

Oh, and when tapes go offsite, they are reclaimed against the %util. %empty gets added to %reclaimable. (That's why they get reclaimed immediately.)

It does make sense. Let TSM do its thing. Or you'll regret it big time in a DR situation. Well, maybe small time. But you will lose more data than necessary, for saving a couple tapes and some small time reclamation.

 

[chad_small]

Heada reclamation works off of the pct_reclaimable. When a tape goes offsite at under 70% reclamable space it is part of the next reclamation batch. Plus I don't have requirements for tapes to go off immediately because the customer doesn't have it as a requirement (contrary to our urging they do go off immediately). Collocation by group wont work for me since I have my copy pools segregated by customer. Its the whole waste time and tape handling I am trying to avoid.

 

[heada]

70% of your nightly backup data is expired before the data leaves the site? That seems to defeat the purpose of sending data offsite at all (70% of the data never leaves the site) I too would be very frustrated with a situation like that.

Are you only keeping 1 copy of the data? I'm still baffled how you could have tapes leaving the site with 70% reclaimable space and have multiple backup copies of data.

If you really want to only move 100% full tapes, you could script a SQL query to look for DRM managed tapes that are mountable and 100% full and output it to a file. Then based on the contents of that file do a move drm for each volume with just a tostate and ignore the fromstate. That should move only 100% full tapes that are onsite and leave the filling tapes.

-Aaron

 

[JohanW]

Let me get this straight.

You are willing to sacrifice the data that ends up on tapes that are not full yet to a disaster, in order to save on some (but not all!) tape handling, and some (but not all) reclamation?

Worst case: half of your most important database backup hasn't made it out to offsite storage, it is wholly invalid, and you lose a day more than necessary.

I want my copypool offsite ASAP. All of it.

Seeing how there is not a WHERESTATUS=FULL option to MOVE DRMEDIA, IBM seem to agree with me.

 

[chad_small]

Heada, no 70 does not expire overnight but I tend to send at least 10 tapes off with only 30% or less utilization daily due to our requirement to segregate our clients copypools. Remember when you send tapes offsite reclamation works off the percent reclamable which is directly related to pct_utilized. So if a tape is sent offsite only 30% used it is immediately reclaimable if your reclamation threshold is 60 to 70% reclamable.

 

[chad_small]

I don't think you guys get it, NOT ALL THE CUSTOMERS ARE PAYING FOR IT! Of course I would send the data offsite (I have been doing this long enough you know!). The problem is it costs us money to do something the customer does not require and does not pay for. Have any of you seen the cost of LTO-3 media it aint cheap; plus the cost for offsite tape handling and rotation? This is a business for profit not a government agency

I need to clarify that....The customer does not pay for the fruequency of tape rotation which is daily. Most of the customers only have a vague statement that data go offsite but no requirement for frequency, typically they tended to a weekly rotation beforehand. As many of you who work for outsourced providers the people who write up the agreements tend to not have a full grasp of the costs.

 

[booman55]

You can accomplish sending off only full tapes by scripting it out. Update the location for volumes using:

upd vol * wherestg=drpool loc=abcd wherest=full whereacc=readw,reado

then do check them using:

move drm * wherest=mo copy=drpool source=dbs tostate=vault wait=yes wherelo=abcd tolo=abcd cmd=&vol cmdf=/path/to/cmdfile

Hope that helps.

 

[chad_small]

Yeah I thought about using the location parameter but I figured a way to do it using a coule select statements. The other thing I did was to consolidate the checkout of 3 TSM instances into one job by using a remove=no in the MOVE DRM command and then checking them in to the library controller and then out again to BULK. It turns three checkout processes to one.

 

[heada]

Heada, no 70 does not expire overnight but I tend to send at least 10 tapes off with only 30% or less utilization daily due to our requirement to segregate our clients copypools. Remember when you send tapes offsite reclamation works off the percent reclamable which is directly related to pct_utilized. So if a tape is sent offsite only 30% used it is immediately reclaimable if your reclamation threshold is 60 to 70% reclamable.

pct_utilized for a non-full tape is based on the est_capacity for the devclass. Pct_reclaimable is based on actual data and how much of that actual data has expired. If you have a tape that holds 100GB, only had 50GB written to it and had 10% of that 50GB expire you would have a pct_reclaimable of 10% and not 55% (50GB that was never written plus 10% of 50GB that was written) Similarly, if you send a volume that can hold 100GB offsite with only 50GB written to it, it should have a pct_reclaimable of 0 until data starts to expire off of it.

-Aaron

 

[chad_small]

Aaron,

The following is directly from the Admin Guide under Choosing A Reclamation Threshold:

The reclamation threshold indicates how much reclaimable space a volume must have before the server reclaims the volume. Space is reclaimable because it is occupied by files that have been expired or deleted from the Tivoli Storage Manager database, or because the space has never been used.

So your explanation is incorrect. If the tape goes offsite with 70% unused it still meets the reclamation threshold of 60% to 70% I use. So I think you can see my frustration.

 

[heada]

You control how much space "has not been used" by the est_capacity setting. There is no way TSM knows what a tape will truly hold until it reaches the end of the tape. If you tell it it will hold 100GB but it will actually hold 300GB then when it puts 100GB on the tape it will go to 100% full but still be in the filling status. By the same token, if you tell TSM that a tape will hold 300GB but actually only holds 100GB, it will jump from 33% full and filling status to 100% full and full status once it reaches the end of the tape. This allows you to send a tape offsite that is less than full (i.e. reached the end of the tape) but still be 100% utilized. Simply by changing the est_capacity to something less than the actual capacity of the tapes can prevent the less than full tapes from being reclaimed immediately when sent offsite.

-Aaron

 

[chad_small]

I don't see why I would want to do that just for this issue. I have never played around with the est. capacity.

 

[heada]

I set it lower than the actual size of the tape for just this purpose(I assumed most people did). I use 3592-J1A tapes that have a native capacity of 300GB and I normally get between 900GB and 1.2TB per tape with compression. I have the est capacity for the 3592 device class set to 200GB. On average I send off between 10 and 15 tapes per datacenter and none of them have less than 100% on them (but not all of them are full).

For me, its easier to buy extra tapes than to worry about the extra tape handling for reclamation in a scenario like this.

-Aaron

 

[JohanW]

Chad, I had read into your second post that you urged customers to *not* have tapes go offsite immediately. I read wrong. Sorry.

Rewrite the contracts. Best technical solution.

Don't use MOVE DRMEDIA *, but spell out which tapes you want moved. Best cheapskate solution.

I first wrote best business solution, but I'm not convinced the businesses at hand actually understand their own best interest.

I like the temp location solution. It has a certain elegance.

 

[chad_small]

Johan,

You'd be surprised how many customers are just plain cheap. As for the solution I came up with...I do a select that generates a list of all volumes in copy pools with a status of full and appends the DB Backups list. I then run that list through a MOVE DR command. It works pretty well.

 

[JohanW]

I might very well be. There might be a reason I work for a boss, not for an agency.

Somehow, combining straight up TSM commands seems more elegant than combining selects and commands. The driver for that is not business necessity, it's job satisfaction, which paying customers will never understand ..