Weird process won't go away

[kfiresmith]

Hi everyone,

I've got a weird process holding me up:
====================================
Process Process Description Status
Number
-------- -------------------- -------------------------------------------------
10 Space Reclamation Volume S07149 (storage pool BACKUP_T), Moved
Files: 0, Moved Bytes: 0, Unreadable Files: 0,
Unreadable Bytes: 0. Current Physical File
(bytes): 5,369,365,080 Current input volume:
S07095. Current output volume:
/tsm_data/seq/rec-vols/00002007.BFS.
=================================================

Note that it says reclamation on vol s07149, but current input vol is s07095!
I can't get this process to cancel, it just says 'cancel pending', and it's been over a day!

Help!

 

[cheffern]

This is because it *has* to finish moving the current physical file -
and in your case - this is HUGE so it will take a long time...

Current Physical File
(bytes): 5,369,365,080

You'll just have to be patient as ever...

The volume thing could be because the one file is actually spanning multiple volumes ...

-Chef.

 

[kfiresmith]

Thanks, I'll keep waiting. 5GB is a lot in TSM terms?

 

[cheffern]

It can be, yes.

Do this - q con S07149 and q con S07095 -
See what comes up - if it's a lot of stuff, just ignore it...
But since it says it has not moved any files yet and is hung up on that one file,
I'd be curious to see if you can see it spanning the tapes...

 

[chad_small]

5GB is not a lot. Not big enough to cause the process to be pending cancellation for over a day. You can try SHOW LOGPINNED CANCEL to see if it will clear the process but in many cases TSM has to be cycled and I haven't seen anything from Tivoli or in the forum that is an undocumented FORCE cancel.

 

[kfiresmith]

I've actually tried a power cycle, it starts right back again and hangs on the same file. Thanks for all the new stuff to try though, I'll keep looking on my end!

Results of show logpinned cancel
==================================================
tsm: BS0A>show logpinned cancel
Dirty page Lsn=305445.198.2918, Last DB backup Lsn=303402.254.1585, Transaction table Lsn=305446.19.1718, Running DB backup Lsn=0.0.0, Log truncation Lsn=303402.254.1585

Lsn=303402.254.1585, Owner=DB, Length=64
Type=Update, Flags=C2, Action=UpdRightSib, Page=1851795, Tsn=0:1250748962, PrevLsn=303402.254.431, UndoNextLsn=0.0.0, UpdtLsn=303363.115.1801 ===> BeforeAddr=4299684,
AfterAddr=6250983
The recovery log is pinned by the last data base backup. Space will not be freed until data base backup is run again.
===================================================

 

[cheffern]

Are there errors showing up on one of those tapes it is reading from?
Did you check to see if the file spanned tapes?

 

[sgabriel62]

It appears you are reclaiming to a virtual volume? Is there enough space on the receiving array - and take a look at your device class info to see if your file size is large enough and you have enough scratch allotted to it.
If this is a VTL - what emulation are you running. Lastly - you can disable reclamation by inserting the correct value in the dsmserv.opt file so that if you have to recycle again you can launch reclamation manually or via a know admin schedule

Let us know

 

[kfiresmith]

Our Linux admin took the server down to remove some old drives, and apparently all our SCSI devs reordered themselves , so the tape/process wedged itself. Got the admin to fix the SCSI stuff and the problem went away.

 

[sgabriel62]

Well There ya Go - another fix for the Linux users in here.

 

[cheffern]

oh wow - never heard of that one before...
so were other things working ok during this time for the other drives?
I have found that when a drive goes bad and you replace it, you usually end up having to reorder everything in TSM...

 

[kfiresmith]

It had just wedged the one drive (out of two), but even losing one when you have two makes everything a pain to do.
This is my one cursed server out of the three. Every few weeks, it's some new stumper.:down:

Now I have one tape that wont check in, and I do these all the time. It just ends in failure, don't have the error handy, but there's no reason for it to fail. Drives are good now, paths are good, drives are empty, vol is in the backup pool, has the right status... Doesn't even get to the "Request pending" point, just fails...

 

[heada]

Is the volume that fails to checkin handled by DRM? Is it in the volhist file? It should tell you exactly why it wont check it in if you look at the activity log.

-Aaron

 

[kfiresmith]

Haven't checked volhist yet, but the act log doesn't give more than a very generic failure error. Usually these are pretty detailed. It's in the local DLT_class copy pool. I'll be looking into it more today, wish me luck!

 

[heada]

Since the volume is in a copypool, it is managed by DRM. If you are attempting to check the volume in as scratch, it will fail because thats not allowed. You should be able to check it in as private thought. Also, if TSM thinks its already checked into the library, you can't check it in again. You'll need to perform a checkout without remove (logically check it out only) and then you should be able to check it back in as private.

-Aaron

 

[cheffern]

Are you on the latest tape drivers for your lib on Linux?
Not saying it would help, but you never know...