Server out of data storage space

[ramzeso]

Hi, I am getting error "Server out of data storage space" for some nodes. Consider screens below, can someone explain me how its possible when pct util is just 34% of 99GB

Then what does it mean Pct util? Its 9.5 for 00005A58.bfs and it says full. I am a little bit confused.

What can I do now?


As you can see, I got 10 vol declared for DEDUP stg pool, how is it filling? I thought its filling by turn.

 

[Harry_Redl]

Hello,

this is all about how TSM measures and reports used space plus the effect of expiration. If you have 100GB volume and write 100GB data to it (end-of-file/volume is reached), then the volume is reported as FULL (EOF - cannot be appended as it is sequential) and 100% full.
If first 50GB expire (obsolete data), then it is still FULL, but only 50% of capacity used/reported.
What you need to do is to add scratch volumes/files to your stgpool and run the reclamation.

Harry

 

[ramzeso]

Hello,

this is all about how TSM measures and reports used space plus the effect of expiration. If you have 100GB volume and write 100GB data to it (end-of-file/volume is reached), then the volume is reported as FULL (EOF - cannot be appended as it is sequential) and 100% full.
If first 50GB expire (obsolete data), then it is still FULL, but only 50% of capacity used/reported.
What you need to do is to add scratch volumes/files to your stgpool and run the reclamation.

Harry

Ok thx, I increased number of scratch volumes from 10 to 20 and i run reclamation and

 

[Harry_Redl]

Hi,

it depends on the reclamation threshold set, pct_reclaim on volumes and other things - for individual volumes you can use "move data e:\tsmdata\dedup\00005a58.bfs"

Harry

 

[ramzeso]

Ok, thx. As a conclusion.

1) Set more scratch volumes
2) Move data manually or run reclamation


Now it worked, but still this volume is set as "FULL". Pct before move data was 8.5, now is 2.4 and still FULL

 

[Harry_Redl]

Hi,

when all data is moved off the (file) volume (this can take some time), the volume should:
a) get deleted (if it is scratch and in READWRITE state - if it is in other state - like READONLY - update it to access RW)
b) have status "EMPTY" - if it is not scratch

Harry

 

[ramzeso]

Hi,

when all data is moved off the (file) volume (this can take some time), the volume should:
a) get deleted (if it is scratch and in READWRITE state - if it is in other state - like READONLY - update it to access RW)
b) have status "EMPTY" - if it is not scratch

Harry

After doing move data,


It is scratch, like u said, so it should have been deleted...

 

[Harry_Redl]

Hi,

yes, but it still contains 2.3% of data - try the "move data" again and check (in the activity log) the progress - what went wrong so the volume is not empty.

Harry

 

[ramzeso]

Hi,

yes, but it still contains 2.3% of data - try the "move data" again and check (in the activity log) the progress - what went wrong so the volume is not empty.

Harry

Like u said, there is problem, I saw on forum similar problem but w/o solution.

 

[Harry_Redl]

Hi,

can be normal - if your DEDUPREQUIRESBACKUP server option is set to YES (which is the default), then the data not already copied to copypool cannot be moved off the volume. So try to run the "backup stg ....." command and then "move data" again.
Other option is to set the option DEDUPREQUIRESBACKUP to NO - but this is not recommended (unless you really know what you are doing).
Is your server upgraded from previous version? (like 6.1?) There used to be a bug with this - but I cannot find the details now.
If the above does not help, open the PMR.

Harry

 

[ramzeso]

Hi,

can be normal - if your DEDUPREQUIRESBACKUP server option is set to YES (which is the default), then the data not already copied to copypool cannot be moved off the volume. So try to run the "backup stg ....." command and then "move data" again.
Other option is to set the option DEDUPREQUIRESBACKUP to NO - but this is not recommended (unless you really know what you are doing).
Is your server upgraded from previous version? (like 6.1?) There used to be a bug with this - but I cannot find the details now.
If the above does not help, open the PMR.

Harry

My server is 6.3 version.
A "coopypool" u mean next pool to primary?
Can u pls explain what does it mean scratch volume? I need simple explanation.

 

[Harry_Redl]

Hi,

I can see your TSM server is 6.3.1.0 (from the first screenshot) - but if it was 6.1 (for example), the data was already there and then it was updated to 6.3, you MAY hit the bug.
Copypool - TSM has two basic types of stgpools - primary (data are stored here when doing backup) and copy - data are COPIED here from PRIMARY storage pools - this is mainly for prevent the data loss when primary volume gets damaged.
Primary pools can be arranged hierarchically (using "next storage pool") - data are moved between the pools by "migration" and "move data".
Copypools cannot have "next" stgpool - data are stored there using "backup stgpool". You cannot move the data from one copypool to another.
Scratch volume - for file volumes - file which is created when needed (and allowed) and deleted when empty. For tapes - tape not being directly assigned to the storage pool. After the tape becomes empty again, it returns to "scratch pool" and can be used by any other storagepool or DB backup, export ... whatever.

Harry

 

[ramzeso]

Correct me if I am wrong. See screen below

TSM is doing backup. Lets say we doin sched incremental bck for desktops. When launching all data is going to stg "DESKTOP" (DESKTOP got some volumes). I have got set in maintance script

BACKUP DB DEVCLASS=DBBACKUP WAIT=YES TYPE=FULL
BACKUP DB DEVCLASS=ULTRIUM5 WAIT=YES TYPE=FULL
del volhist todate=today-2 t=dbb
PARALLEL
MIGRATE STGPOOL DESKTOP WAIT=YES LOWMIG=20 DURATION=300
MIGRATE STGPOOL ORACLE WAIT=YES LOWMIG=20 DURATION=300
MIGRATE STGPOOL SERVERS WAIT=YES LOWMIG=20 DURATION=300
MIGRATE STGPOOL SQLPOOL WAIT=YES LOWMIG=20 DURATION=300
SERIAL
EXPIRE INVENTORY SKIPDIRS=NO WAIT=YES RESOURCE=4
SERIAL

So it means when data on DESKTOP stg pool is bigger then 20% It migrates to next stg pool "DESKTOPTAPE" which is tape.

So basically data never stay (max 20% capacity) on DESKTOP stg, is being migrate to DESKTOPTAPE

I could find any script where is command backup stgpool and where its going. Is any way to list my copypools?

PS DESKTOPTAPE type is Primary .... :/ I am really confused

I think i dont have any copypools

q stg pooltype=copy is empty

 

[Harry_Redl]

Hi,

yes, you do not have any copypool (in "q stg" they would have all migration percentages blank) - so it is pointless to have the DEDUPREQUIRESBACKUP set to YES (if it is so - see "q opt").
Are you sure you want to run deduplication with setup like this? As the data is stored in the DEDUP pool, deduplicated (if it happens) and rehydrated again when migrated to tape. Not to mention you only have like 200GB stgpool for dedup .... looks weird from here (but I do not know the background ....).

Harry

 

[ramzeso]

Ok, but what the difference between
1) migration data from primarypool to next stg (in my case tapes)
2) bck stgpool from primary to next ts (in my case tapes)

Most of config can be weird, I am totally newbie in TSM, in my company TSM was implemented by other company, now I am the guy who tries to fix it.

 

[Harry_Redl]

Hi,

1) MOVES data from one stgpool to another (you still have only one copy)
2) COPIES data (one stays in primary, second is created in copy) - bck stgpool is NOT an operation between primary pools!!!!

(my original suggestion to run "backup stg ..." is irrelevant now - as I did not see you only have primary pools before)

Harry

 

[ramzeso]

I think is time to this from beginning. I will create copypools .