No worries, was trying to think outside the box in a supported way
And yes, need more tapes....
My immediate concern is of your database.
Your script has this line:
5 backup db devclass=TS3200DEVCL type=FULL wait=yes
So, your database full backups are going to tape and is going to use scratch tapes. As you have no scratch tapes, its likely failing. How many previous database backups are you keeping?
You could do a
to see your oldest BACKUPFULL media. Please post here the list if you can.
I was thinking, you might be able to get a few scratch tapes from those old database backup tapes. Do you have any scripts running a del volhist type=dbb? However, I recommend you do
not manually run a del volhist type=dbb just yet. Lets see what volumes you are using for old database backups first.
That said, you may want to look at 'q db f=d' and look for the line 'Last Complete Backup Date/Time' as well. Then a 'q log f=d'. How much space do you have free in Archive Log Directory? I'm concerned if you don't have scratch tapes for reclamation, you don't have any tapes for your database backup. If your database doesn't backup, Archive Log becomes full and your TSM server comes down. It will be much harder to recover if TSM halts vs trying to reclaim a storage pool. At that point, a PMR would likely be needed to IBM. Two weeks without any database backups is asking for failure IMO.
Have you had a look at printing your own labels?
https://tapelabel.de/
DazRaz, that's very handy. I'll have to keep it tucked away!
If you can, what DazRaz has stated could tide you over till your real labels come on site.
In addition to what DazRaz said in their post, you may want to temporally comment out the migration line of your daily maintenance script if you can - and if you DISK_DD pool has space for a night or two of backups, till you successfully reclaim a volume from your primary tape pool. Depending on when your script is running, it could as you stated above, immediately take the tape you just made available and use it for its own.
To recap as I started to write this and got distracted a few times...
At this point, let us know what the results of:
- q volhist type=dbb
- q db f=d (look for Last Complete backup Date/Time)
- q log f=d Look for the Free Space on File System something like below:
Code:
Archive Log Directory: /tsminst1/tsmarclog
Total Space of File System (MB): 783,360.00
Used Space on File System (MB): 343,257.45
Free Space on File System (MB): 440,102.55
- Do you have any tape volumes in a pending state for your primary tape pool? As moon-buddy posted above, but this time do a q stg <primary tape pool name> and look for 'Delay Period for Volume Reuse'
Next steps:
- If you have a lot (greater than 10 days) of database backup volumes and no recent db backup I suggest, at your own risk, of removing an old database backup, and first running a new full backup till completion. This will clear out your Active Log and keep the system running that much longer.
- May not be a bad idea to do a 'q proc' and cancel the migration process if you see it waiting for a mount point before doing any thing such as removing old database backups. Which I'll imagine is the case till it times out. If you don't the migration process will come in and take your tape.
- If you have a lot (greater than 10 days) and a recent backup volume (of within 24 hours or so and log space) I suggest, at your own risk, of removing an old database backup, and then trying to use that tape as your reclaim tape for your primary storage pool.
- Again, look at q proc and cancel the migration process if you see it waiting for mount.
Over all sounds like your resources are pretty lean so you may have to make sacrifices of daily backups from clients till your TSM server gets healthier.
I just want to make it clear that I feel the people on the forums here are some of the best in the world when it comes to TSM, and if any of them contradict what I've posted above, or calls it just plain wrong take their advice over mine! Just trying to look at the problem from a different view point.