Reclamation Failing - Resulting in lack of scratch tapes

[Finnelly]

Hi All

This is my first post so please take it easy on me! lol

TSM ver: 5.4 - Windows 2003

I have had the responsibility of TSM pushed onto me at my current company and am facing a few problems which I have yet to find a solution to.

It seems that over the last few weeks the system has been deprived of scratch tapes and as a result reclamation has fallen over and the LOG has filled up.

The system shows that there are 74 scratch tapes that can be made available, I have been told that our TSM will only be able to release 10 at a time, due to this everytime reclamation runs it fails as its trying to release all 74 tapes! The LOG has filled up as a consequence of the system continually retrying this procedure.

I need help with how to manually release the tapes and then run reclamation on our onsite storage pool.

Any help would be much appreciated.

Thanks

Shane

 

[moon-buddy]

What do mean by 'release all 74 tapes' - what are these tapes - online pool or copy pool tapes?

Technically, reclaiming online or off site tapes should not hinder any process and will continue (depending on duration settings) until all reclaimable tapes were processed.

Can you clarify a bit more?

 

[sjl]

It seems that over the last few weeks the system has been deprived of scratch tapes and as a result reclamation has fallen over and the LOG has filled up.

These statements seem disconnected to me - whether there are scratch tapes or not has no direct relevance to the TSM database log. Or are you referring to a different sort of log?

The system shows that there are 74 scratch tapes that can be made available, I have been told that our TSM will only be able to release 10 at a time, due to this everytime reclamation runs it fails as its trying to release all 74 tapes! The LOG has filled up as a consequence of the system continually retrying this procedure.

How do you get the "74 scratch tapes that can be made available" figure? What command is giving you that result? The more detail you can give, the easier it will be to make suggestions on how best to proceed.

For example, try these commands:

SELECT status,count(*) from volumes where devclass_name in (select devclass_name from devclasses where devtype not in ('DISK','FILE')) and devclass_name <> 'DISK' group by status

SELECT stgpool_name,count(*),sum(100-pct_reclaim)/100 from volumes where pct_reclaim >= 80 group by stgpool_name

The first one will tell us how many tapes you have, and what state they're in - in this case, I'm interested in how many empty and pending cartridges you have: those are ready to be returned to the library to become scratch (not quite true for pending cartridges, but if you're absolutely desperate, it's a starting point, and can help you get out of trouble in the short term - just don't make a habit of it.) The second one will tell you how many cartridges in each storage pool are ready for reclamation, based upon an 80% threshold, and how many cartridges (approximately!) they'll need to hold the reclaimed data. Feel free to adjust the threshold level as you see fit; 80% is a reasonable starting point if reclamation hasn't run for a while, and you can lower it as time goes on and more tapes are reclaimed.

 

[herveymtz]

I had an issue with TSM that I ran into a while back very similiar to yours and it was due to tapes being in "filling" status as opposed to "full". When reclamation runs, it looks at the tapes that are "full" and from here it determines what tapes are eligible to be reclaimed. When you display the tapes on your various pools, do look for the "filling" status. Especially your copy pools, these are tapes that are offsite but will never be "full". You have to force a manual recycle of these tapes.

 

[sjl]

I had an issue with TSM that I ran into a while back very similiar to yours and it was due to tapes being in "filling" status as opposed to "full". When reclamation runs, it looks at the tapes that are "full" and from here it determines what tapes are eligible to be reclaimed. When you display the tapes on your various pools, do look for the "filling" status. Especially your copy pools, these are tapes that are offsite but will never be "full". You have to force a manual recycle of these tapes.

Sorry, but this is completely wrong. After your daily jobs run, and tapes are sent offsite, check a tape that's just gone offsite with a state of "filling", and less than 100% utilisation. You will see that the reclaimable space is set to (100-utilisation)% - eg, if the tape has 75% utilisation, the reclaimable space is set to 25%. Now, it's completely true that that value is garbage, inasmuch as you don't know how much of the tape is used (if the data is very compressible, it might be only 10%; if it's not compressible at all, it might be closer to 100%) - but that doesn't matter. All that matters is what TSM thinks is used, and that's set against the nominal capacity of the tape.

So when it comes time to reclaim offsite tapes, TSM looks only at the reclaimable space (pct_reclaim in the volumes table), not at the status (full or filling) of the tape. If you have an offsite filling tape that is only 1% used, it will be eligible for reclamation.

There are a few factors that might cause offsite tapes to not be reclaimed. Number one on that list is not letting reclamation run to completion. When reclaiming offsite volumes, TSM collates the information about what's stored on the volumes to be reclaimed (as determined by the reclamation threshold, and the limit on the number of volumes to reclaim), and mounts the onsite volumes one at a time, reading the necessary data off each in turn. This reduces the number of tape mounts required for the reclamation process. But it does mean that if you cancel reclamation early, you might not reclaim any volumes: there might be data on that last tape TSM has to read for all of the volumes to be reclaimed, so although you've reclaimed a large volume of data, you actually haven't freed up any tapes at all.

The other factor that springs to mind is availability of the data on the TSM server (are the primary pool tapes TSM needs readable and in the library?)

So in short: it's not the status of the tape that matters, but how much space TSM thinks is available for reclamation that determines whether or not a given tape is reclaimed - and for offsite tapes, that's a function of how much data has been written, not whether the tape was filled to capacity. In fact, I have seen TSM reclaim an onsite, filling cartridge: it had been filled to 80+%, never written to again (known issue, see the ADSM quickfacts for details), the data on it had expired to bring it down to 30% usage - and 50% reclaimable space - so it went through reclamation. Makes sense - reclaimable space on a filling tape is just as significant as on a full tape (except if it happens to be at the end of the written data: an unlikely scenario).

 

[herveymtz]

Well, I opened a ticket up with IBM when I first ran into this issue and spent quite a few weeks working on this with them. By the way, I have to run daily manual data moves on "filling" tapes because of this. Are you willing to look into my TSM set up to determine why the "filling" tapes are not being reclaimed? By the way, I run TSM under zOS.