TSM STGPOOL is full

[Out]

Dear TSM experts,

First of all want to mention that I am not TSM guy even not IBM. I am Oracle DBA and I have problem with TSM.
Recently I got error from one of my DB that TSM storage is full and that's why DB backup is failed.
I checked TSM system which installed in Windows system where we have DISK of 6TB and this is really full. But we have Tapes and as I understood TSM must transfer from DISK to tape and after remove from disk. Is it like this ? I check all configuration everything is online. In tapes we have space. And I think that the TSM transfer from DISK to TAPE but not reclaiming after from DISK. I found few errors in activity log of TSM


ANR1217E There is an insufficient number of mount points available for removable media.

check with Google , this forum found something but cannot fix. I check also with IBM pages again found information about errors but again in out configuration everything seems be OK. And this TSM is worked for 2 and more years.



Currently we don't have TSM specialist but instead we have TSM problem, I hope that someone can help me to at least understand why our DISK is not reclaimed.

Thanks in advance


some information
---------------------------------------------
Volume Name Storage Device Estimated Pct Volume
Pool Name Class Name Capacity Util Status
------------------------ ----------- ---------- --------- ----- --------
F:\BACKUPPOOL\DSM000 BACKUPPOOL DISK 512.0 G 100.0 On-Line
F:\BACKUPPOOL\DSM001 BACKUPPOOL DISK 512.0 G 91.3 On-Line
F:\BACKUPPOOL\DSM002 BACKUPPOOL DISK 512.0 G 100.0 On-Line
F:\BACKUPPOOL\DSM003 BACKUPPOOL DISK 512.0 G 100.0 On-Line
F:\BACKUPPOOL\DSM004 BACKUPPOOL DISK 512.0 G 98.8 On-Line
F:\BACKUPPOOL\DSM005 BACKUPPOOL DISK 512.0 G 100.0 On-Line
F:\BACKUPPOOL\DSM006 BACKUPPOOL DISK 512.0 G 99.6 On-Line
F:\BACKUPPOOL\DSM007 BACKUPPOOL DISK 512.0 G 99.7 On-Line
F:\BACKUPPOOL\DSM008 BACKUPPOOL DISK 512.0 G 100.0 On-Line
F:\BACKUPPOOL\DSM009 BACKUPPOOL DISK 512.0 G 100.0 On-Line
F:\BACKUPPOOL\DSM00A BACKUPPOOL DISK 512.0 G 99.6 On-Line
F:\BACKUPPOOL\DSM00B BACKUPPOOL DISK 512.0 G 96.4 On-Line
V7DG000L TAPEPOOL LTOTAPE 3.1 T 50.7 Full
V7DG003L COPYPOOL LTOTAPE 3.0 T 64.8 Full
V7DG005L TAPEPOOL LTOTAPE 3.9 T 43.3 Full
V7DG007L COPYPOOL LTOTAPE 3.9 T 44.8 Full
V7DG008L COPYPOOL LTOTAPE 3.6 T 54.6 Full
V7DG009L COPYPOOL LTOTAPE 3.5 T 62.8 Full
V7DG010L TAPEPOOL LTOTAPE 3.0 T 0.4 Filling
V7DG011L COPYPOOL LTOTAPE 3.5 T 73.3 Full
V7DG012L TAPEPOOL LTOTAPE 3.4 T 57.3 Full
V7DG013L COPYPOOL LTOTAPE 3.0 T 25.6 Filling
V7DG014L COPYPOOL LTOTAPE 3.0 T 0.6 Filling
V7DG015L TAPEPOOL LTOTAPE 3.8 T 36.4 Full
V7DG016L TAPEPOOL LTOTAPE 3.0 T 35.4 Filling
V7DG017L TAPEPOOL LTOTAPE 3.4 T 50.2 Full
V7DG018L TAPEPOOL LTOTAPE 3.1 T 61.3 Full
V7DG019L TAPEPOOL LTOTAPE 3.2 T 60.9 Full

---------------------

tsm: MTSM107>q stg BACKUPPOOL f=d
Session established with server MTSM107: Windows
Server Version 6, Release 3, Level 0.0
Server date/time: 02/05/2015 16:06:51 Last access: 02/0

Storage Pool Name: BACKUPPOOL
Storage Pool Type: Primary
Device Class Name: DISK
Estimated Capacity: 6,000 G
Space Trigger Util: 98.8
Pct Util: 98.8
Pct Migr: 98.8
Pct Logical: 99.6
High Mig Pct: 90
Low Mig Pct: 70
Migration Delay: 0
Migration Continue: Yes
Migration Processes: 1
Reclamation Processes:
Next Storage Pool: TAPEPOOL
Reclaim Storage Pool:
Maximum Size Threshold: No Limit
Access: Read/Write
Description:
Overflow Location:
Cache Migrated Files?: Yes
Collocate?:
Reclamation Threshold:
Offsite Reclamation Limit:
Maximum Scratch Volumes Allowed:
Number of Scratch Volumes Used:
Delay Period for Volume Reuse:
Migration in Progress?: No
Amount Migrated (MB): 0.00
Elapsed Migration Time (seconds): 0
Reclamation in Progress?:
Last Update by (administrator): ADMINISTRATOR
Last Update Date/Time: 10/01/2012 13:00:29
Storage Pool Data Format: Native
Copy Storage Pool(s):
Active Data Pool(s):
Continue Copy on Error?: Yes
CRC Data: No
Reclamation Type:
Overwrite Data when Deleted:
Deduplicate Data?: No
more... (<ENTER> to continue, 'C' to cancel)

Processes For Identifying Duplicates:
Duplicate Data Not Stored:
Auto-copy Mode: Client
Contains Data Deduplicated by Client?: No

 

[moon-buddy]

How many tape drives do you have?

Can you access TSM via CLI? If so, can you run: "q drive", "q path", "q mount" "select count(*) from libvolumes where status='Scratch'" and post output?

 

[Out]

How many tape drives do you have?

Can you access TSM via CLI? If so, can you run: "q drive", "q path", "q mount" "select count(*) from libvolumes where status='Scratch'" and post output?

Thanks for reply.

CLI I have but I am not sure from where I can run SQL code ?????

Here the output .

tsm: MTSM107>q drive

Library Name Drive Name Device Type On-Line
------------ ------------ ----------- -------------------
BEANR-TAPLI- TAPE0 LTO Unavailable Since
B101
10/05/2014 01:54:18
BEANR-TAPLI- TAPE1 LTO Unavailable Since
B101
11/05/2014 01:00:39
----------------------------------------------------------------------------------------------------------
tsm: MTSM107>q path

Source Name Source Type Destination Destination On-Line
Name Type
----------- ----------- ----------- ----------- -------
MTSM107 SERVER BEANR-TAPL- LIBRARY Yes
IB101
MTSM107 SERVER TAPE0 DRIVE Yes
MTSM107 SERVER TAPE1 DRIVE Yes

----------------------------------------------------------------------------------------------------------
tsm: MTSM107>q mount
ANR2034E QUERY MOUNT: No match found using this criteria.

 

[moon-buddy]

There is your problem: the tape drives are OFFLINE.

Run these commands from the CLI

update drive BEANR-TAPLIB102 TAPE0 online=yes
update drive BEANR-TAPLIB102 TAPE1 online=yes

After running the above commands, re-run "q drive" and see if they go online.

By the way, the SQL Select command is also run on the CLI. This is a DB2 select command that TSM uses.

 

[Out]

There is your problem: the tape drives are OFFLINE.

Run these commands from the CLI

update drive BEANR-TAPLIB102 TAPE0 online=yes
update drive BEANR-TAPLIB102 TAPE1 online=yes

After running the above commands, re-run "q drive" and see if they go online.

By the way, the SQL Select command is also run on the CLI. This is a DB2 select command that TSM uses.

Thanks. Can you please explain how they can be offline but when I run the command to see the status I see filing of tapes ?

V7DG010L TAPEPOOL LTOTAPE 3.0 T 0.4 Filling

I just want to understand the logic .

Abut SQL Select command I tried from CLI but got error message.

No match found using this criteria.

Thanks

 

[moon-buddy]

The tape drives went offline for many reasons - one may be due to a communication issue between the host and TSM.

The SQL error message means you have NO scratch tapes for storing data. Check in some scratch tapes.

Really, you need a TSM admin or consultant to look after this. You cannot survive running TSM without maintenance.

 

[Out]

The tape drives went offline for many reasons - one may be due to a communication issue between the host and TSM.

The SQL error message means you have NO scratch tapes for storing data. Check in some scratch tapes.

Really, you need a TSM admin or consultant to look after this. You cannot survive running TSM without maintenance.

What I really don't understand. If the drives are offline then why the TSM still working and showing that tapes are filling ?

 

[moon-buddy]

Tapes are filling from a previous status - it is not yet full. It has nothing to do with the Tape drives being offline.

If you look at the activity log of TSM, there has been no tape movement since the tape drives went offline.

 

[Out]

Tapes are filling from a previous status - it is not yet full. It has nothing to do with the Tape drives being offline.

If you look at the activity log of TSM, there has been no tape movement since the tape drives went offline.

Thanks again. Can you please help me. How I can find in activity log that "there has been no tape movement since the tape drives went offline." I need some evidences.

thanks

 

[Jarzi]

q actlog begindate=-30 s=V7DG0

That searches actlog for term V7DG0 from last 30 days.

If you have longer act log retention than 30 days you can change the begin date but IIRC 30 days is the default value.

 

[Out]

Thank you very much I will check it.

Thanks for help !