TSM requires tapes not in my library during reclamation

[squig]

Some times when I am reclaiming tapes, I find tsm will request a tape that is not in my library to complete the reclamation process. I understand that this will occur if a file or aggregate spans two (or more) tapes.I am wondering If there is any way to determine this so I can load the tapes I am going to need for reclamation into the library before it asks me. I found the following post that mentions how to determine which tapes have some thing that spans a drive, but no way to determine what it flows onto. http://adsm.org/lists/html/ADSM-L/2004-12/msg00249.html That was almost a decade ago, so I wondered if some one had come across a way in the mean time. <snip
For full use of volumes, and where files may even be larger thanvolumes, TSM will span a file (Aggregate) from one volume to anotheras it fills one volume and goes on to another.You can use 'Query CONTent <Volname> F=D Count=1' on the subjectof your Move Data to see if it starts with data which spans fromanother volume (and Count=-1 to see if it spans to another vol).There is no ready way to determine what the other volume is, shortof initiating the Move Data, or researching the Activity Log forwhen the volume transition occurred./snip>

 

[moon-buddy]

To reclaim tapes properly, make sure that you have all onsite tapes loaded in the library. Do a 'q volume stgpool=<primary_stgpool_name> and q libvol. The tapes listed in the first command should be the same as in the second. Correct any difference.

Then make sure that no primary tapes are unavailable.

It is hard to determine which tape reclamation needs. I just make sure that all primary tapes are in the library.

 

[squig]

unfortunately I am a few hundred slots short of the number I need to do that.

 

[opermty]

could you try put outside of library some tapes, just some to give space at those what are you looking to reclaim.

 

[squig]

could you try put outside of library some tapes, just some to give space at those what are you looking to reclaim.

I have hundreds more tapes than slots, and its only growing. Surely there must be some way to query that data? TSM seems to be able to calculate it in seconds.

 

[tsm_dude]

We have 2000 slots and over 14,000 tapes so we use the 'move media' commands to get FULL high utilized tapes out of library and recall tapes that have high pct reclaimable. To do what you want, you need to find the object ID in the backups table that relates to the clients name for file, the do a show bfo (object ID), it will list all primary/copy tapes containing the object.

 

[moon-buddy]

We have 2000 slots and over 14,000 tapes so we use the 'move media' commands to get FULL high utilized tapes out of library and recall tapes that have high pct reclaimable. To do what you want, you need to find the object ID in the backups table that relates to the clients name for file, the do a show bfo (object ID), it will list all primary/copy tapes containing the object.

squig,

You can do what tsm_dude is saying but be aware that this is a very long and tedious process. It works well but be prepared to run with this for a long time.

I have done reclaiming tapes using this method and I would not do it again! I convinced my management to buy more slots!.

 

[squig]

We have 2000 slots and over 14,000 tapes so we use the 'move media' commands to get FULL high utilized tapes out of library and recall tapes that have high pct reclaimable. To do what you want, you need to find the object ID in the backups table that relates to the clients name for file, the do a show bfo (object ID), it will list all primary/copy tapes containing the object.

Hi tsm_dude, thanks its got me on the right track at least.

Is querying the sql to find the object_id terribly slow? I wonder if there are any indexes on the tables.

Any idea of what a reasonably fast way to find object_id is?

 

[squig]

squig,

You can do what tsm_dude is saying but be aware that this is a very long and tedious process. It works well but be prepare to run with this for a long time.

I have done reclaiming tapes using this method and I would not do it again! I convinced my management to buy more slots!.

I will soon not have enough space to fit a robot that could hold my tapes, let alone afford it!

 

[squig]

Hi tsm_dude, thanks its got me on the right track at least.

Is querying the sql to find the object_id terribly slow? I wonder if there are any indexes on the tables.

Any idea of what a reasonably fast way to find object_id is?

I tried this and its at least possible.

select OBJECT_ID from backups where node_name='TSM01' and -
filespace_name='/mnt/gpfs' -
and hl_name = 'path/' -
and ll_name='filename'

 

[tsm_dude]

The danger SQL in TSM is the contents table not the backup and archive table. That is why the show bfo is used to get info regarding contents.
I once tried an SQL on contents (Test TSM server) and it ran for 2 days then timed out with no rows returned.

Here is the steps I use to find the 2/2 tape in a spanned object; (oh yes its the achive table but ok on backup also)

Find a spanner;
tsm: TSM1>q content G04949L4 count=1 f=d
Node Name: AGUNXRP1_SAP
Type: Arch
Filespace Name: /tdpmux
Hexadecimal Filespace Name:
FSID: 2
Client's Name for File: /XRK___A0H2ZXYLMW_298
Hexadecimal Client's Name for File:
Aggregated?: No
Stored Size: 81,942.50 M
Segment Number: 1/2
Cached Copy?: No
Linked: No

Find the Object ID;
select * from archives where node_name='AGUNXRP1_SAP' and LL_NAME='XRK___A0H2ZXYLMW_298'
NODE_NAME: AGUNXRP1_SAP
FILESPACE_NAME: /tdpmux
FILESPACE_ID: 2
TYPE: FILE
HL_NAME: /
LL_NAME: XRK___A0H2ZXYLMW_298
OBJECT_ID: 492711983
ARCHIVE_DATE: 2012-06-03 20:01:53.000000
OWNER: oraxrk
DESCRIPTION: XRK___A0H2ZXYLMW_TSM1_SAP@SAPDATA_MC#530298 00035
CLASS_NAME: SAPDATA_MC

tsm: TSM1>show bfo 492711983
Bitfile Object: 492711983
**Archival Bitfile Entry
Bitfile Type: PRIMARY Storage Format: 22
Bitfile Size: 85922943991 Number of Segments: 2, flags: 0
Storage Pool ID: 7 Volume ID: 58401 Volume Name: G04949L4
Storage Pool ID: 7 Volume ID: 58456 Volume Name: G05372L4
**Archival Bitfile Entry
Bitfile Type: COPY Storage Format: 22
Bitfile Size: 85922943991 Number of Segments: 1, flags: 0
Storage Pool ID: -4 Volume ID: 58447 Volume Name: G05608L4

So G04949L4 is 1/2
and G05372L4 is 2/2
tsm: TSM1>move data G04949L4
ANR2232W This command will move all of the data stored on volume G04949L4 to other volumes within the same storage pool;
the data will be inaccessible to users until the operation completes.
Do you wish to proceed? (Yes (Y)/No (N)) y
ANR2411E MOVE DATA: Unable to access associated volume G05372L4 - access mode is set to "unavailable".
ANS8001I Return code 14.

So yes G05372L4 is the 2/2 and no it does not contain visable data (q content reveals no client file the same name as on G04949L4)
the tape G05372L4 will be mounted for short time, for the move data to work.

 

[krybczyk]

Instead of reclaiming, would it be possible to move data on tapes back up to a disk stgpool and then let it migrate? Not the optimal solution, but you can at least free up some scratch

 

[tsm_dude]

The diff between move data and reclaim is that reclaim works only on FULL volumes, move data doesn`t care. But essentially its the same thing.
Yes - You can setup a reclamation disk pool, this was desgined for small TSM servers that do not have enough tape drives(only1)
I have never bothered with it.

 

[squig]

The diff between move data and reclaim is that reclaim works only on FULL volumes, move data doesn`t care. But essentially its the same thing.
Yes - You can setup a reclamation disk pool, this was desgined for small TSM servers that do not have enough tape drives(only1)
I have never bothered with it.

we actually do use FILE pool staging.

some thing like
FileSystem -backup-> FILE_STGPOOL -migrate-> TAPE_STGPOOL
<-reclaim-

We also use GPFS/HSM and I have more stgpools than drives, and found filling tapes would be constantly inserted, removed, fast forward to get to the end of the tape, as we changed stgpool.

 

[squig]

The danger SQL in TSM is the contents table not the backup and archive table. That is why the show bfo is used to get info regarding contents.
I once tried an SQL on contents (Test TSM server) and it ran for 2 days then timed out with no rows returned.

When I looked at contents in the db2 directly I could see it had no indexes, I really want to try and speed up my query by adding an index to the db2 database.

Your method is the same as mine, but it still takes about 20 minutes for me to find a file, so im going to have to come up with some form of cache. I imagine running the query once a day looking at all the full tapes (and then only new ones) calculating its friendly tapes and then storing its friends list into a database (probably nosql hashed on the tapename and last write date).

 

[tsm_dude]

Well probably server little slow, my queries are practically instant and my DB is over 300GB. Anyway its a better/quicker method than what you posted originally with the query content and activity log stuff.