move drmedia when tapes are mounted....

[gary_brierley]

Hi,



I'm currently implementing TSM DRM and have started to automate the offsite DR tasks. Unfortunately I've hit a bit of a barrier when I come to move the drmedia from the library.



Once I've completed all my stgpool backups, I then want to remove my drmedia from the library and send offsite - I do this with the command 'move drmedia * wherestate=mountable' (which ejects the media from the tape library). The problem occurs when there are any 'mountable' drmedia mounted in the drive - the move command misses these tapes! I currently have the mount retention set to 20mins.



Is there a 'dismount all mounted volumes' command, or something similar? I'm sure people will have come across this before, I just hope you found a solution....



Yours Hopefully, Gary

 

[klebesco]

Gary,



This use to happen to me all the time...



I certainly don't know how your shop is configured but I can tell you my situation.



We have an Magstar ATL and about 1800 vols assigned to TSM.



I was running the move DR job at 11 a.m. for the convenience of the courier and

we had mount retention set at 5 min which I believe is the default. We were doing migration and client backups at the same time (all night) and occasionally the mount retention would save the robot from having to do a few mounts.



I did however notice that my backups (which at the time were primary tape pool to vault tape pool seemed to go on forever). Our robot takes about 1 minute on average to mount a tape and each tape take up to another minute to rewind and unload. when you add in the 5 min retention... not much was happening, to make a long story short I experimented with the mount retention setting and a value of 0 made my backups run fastest. The bad side of this was that when we were doing migration we would suffer a few more mounts.



I still had the move DR issue with tapes I wanted to go to the vault still mounted (In my case because sometimes my backups were still not done).



Reschuling the backups to start earlier solved the problem.



hope this helps



Kurt





:grin: :grin: :grin: :-o :-o :evil:

 

[shcart]

If you are doing all of this from within TSM itself then your stuck with what the scripts can do. However we have coded our whole DR ejects process from ksh coding and a bit of awk. I have a wait loop set up so that any idle tapes will be found and a dismount volume attempted. I also report any busy tapes in a seperate section of the report produced



The script was written at a time when wait for backup stgpool to complete = carry on and do what you like



The script follows:



#!/usr/bin/ksh

#

# SHC 06/14/2001

# Corrected to correctly wait for busy and dismounting tapes.

#

# SHC 06/29/2001

# Added housekeeping to the local DRMDIR prepfiles remove after 30 days

#

# SHC 09/04/2001

# Amendments to reduce verbiage a little

#

# SHC 12/12/2001

# CUST arcus code modified to CUST

# Added tape courierretrieve processing and reporting

#

# SHC 12/14/2001

# Added processing for the 7YEAR-COPYPOOL

#

# SHC 02/14/2002

# Added processing for the SYSBACK-COPYPOOL

#

# SHC 04/02/2002

# Added Parameter for backup number of processes in dobackup function

#

# SHC 05/20/2002

# Added 1041.. thru 1044.. for offsite range

#

# SHC 05/29/2002

# Added Archive TApe processing

#

# SHC 06/14/2002

# Modified Tape archive processing to correct limitation with SQL

# Replaced SQL 'And NOT LIKE' with awk changes as these work every time.

#

# SHC 10/29/2002

# Modified Tape archive processing to report archived tapes as vaulted

#

# SHC 01/20/2003

# Modified to remove CWS0004A from the preptgt list

#

# SHC 01/21/2003

# Modified to add the Lanfree pool

#

# SHC 01/24/2003

# Modified to correct formatting issue with LAnfree processing

# TSM Wrapped the Storagepool Name

#

# SHC 01/30/2003

# Modified to correct formatting issue with LAnfree processing

# TSM Wrapped the Storagepool Name

#

# SHC 02/18/2003

# Mofified to add Run Date Message



# AMS 05/14/2003

# Change Access of archived tapes from UNAVAILABLE to OFFSITE.



# JG 5/22/2003

# Comment out LANFREE COPY POOL BACKUP until it goes into production



# SHC 05/22/2003

# Change Access of archived tapes from OFFSITE to UNAVAILABLE.





# #

# # ## ##### # ## ##### # ###### ####

# # # # # # # # # # # # # #

# # # # # # # # # ##### # ##### ####

# # ###### ##### # ###### # # # # #

# # # # # # # # # # # # # # #

# # # # # # # # ##### ###### ###### ####



SERVER_NAME=`hostname`

REPORT_NAME=${SERVER_NAME}.OFFSITE_LIST

WORK_DIR=/tsmbase/tsmmgr/work

REPORT_DIR=/tsmbase/tsmmgr/reports

CDIRECT=/tsmbase/tsmmgr/cdirect

TSM_QUERY="dsmadmc -id=USERID -password=PASSW "

DRMDIR=/tsmdrm01

PREPTGT="CWS0001a CWS0002A"



recipients=`cat /tsmbase/tsmmgr/bin/maillist.txt`



#######

# # # # # #### ##### # #### # # ####

# # # ## # # # # # # # ## # #

##### # # # # # # # # # # # # # ####

# # # # # # # # # # # # # # #

# # # # ## # # # # # # # ## # #

# #### # # #### # # #### # # ####





waitproc()

{

date +"Wait $2 Begin :- %H:%M:%S"

echo "\n"

$TSM_QUERY q proc | grep "^[ 0-9][, 0-9][ 0-9][ 0-9][, 0-9][ 0-9][0-9]"

while [ `${TSM_QUERY} 'QUERY PROCESS' | grep -c "$3"` -ne 0 ]

do

sleep $1

done

date +"Wait Done :- %H:%M:%S"

echo "\n\n"

}



dobackup()

{

printf "Backing Up Storage Pool %s CC=" $1

$TSM_QUERY "BACKUP STGPOOL $1 $2 MAXPR=$3 WAIT=NO" | awk '/^ANS8002I/ { print $6 }'

echo ""

}





# # ######

## ## ## # # # # # #### ##### # #

# # # # # # # ## # # # # # # # # #

# # # # # # # # # ###### # # # # #

# # ###### # # # # # # # # # # #

# # # # # # ## # # # # # # #

# # # # # # # ###### #### ##### #



#

# Backup all of the storage pools to flush out the days accumulated files

# to the appropriate offsite tapes

#

# Start of the output redirection grouping

#



{



banner WDC2465A OUTPUT



echo "\n\nEject-Run-Date `date +"%m/%d/%Y"`\n"



echo "\n>>>>> RC=0, RC=11 or RC=14 are acceptable completion codes <<<<<\n"



dobackup DBAPI-DCP-DISK DCP-COPYPOOL 4

dobackup DBAPI-DCP-TAPE DCP-COPYPOOL 4

dobackup OTHER-DCP-DISK DCP-COPYPOOL 3

dobackup OTHER-DCP-TAPE DCP-COPYPOOL 3

dobackup SP-DCP-DISK DCP-COPYPOOL 3

dobackup SP-DCP-TAPE DCP-COPYPOOL 3

# dobackup LANFREE_DCP_TAPEPOOL LANFREE_DCP_COPYPOOL 2



#

# May as well backup the offsite pool as well as it is currently managed by DRM

#



dobackup DBAPI-OFFSITE-DISK OFFSITE-COPYPOOL 3

dobackup DBAPI-OFFSITE-TAPE OFFSITE-COPYPOOL 2

dobackup OTHER-OFFSITE-DISK OFFSITE-COPYPOOL 2

dobackup OTHER-OFFSITE-TAPE OFFSITE-COPYPOOL 2

dobackup SP-OFFSITE-DISK OFFSITE-COPYPOOL 2

dobackup SP-OFFSITE-TAPE OFFSITE-COPYPOOL 2

dobackup 7YEAR-DISK 7YEAR-COPYPOOL 1

dobackup 7YEAR-TAPE 7YEAR-COPYPOOL 1

dobackup SYSBACK-DISK SYSBACK-COPYPOOL 1

dobackup SYSBACK-TAPE SYSBACK-COPYPOOL 1



#

# Now lets loop till all those backups complete

#



waitproc 30 Stgpool_Backups 'Backup Storage Pool'



#

# Now backup the database fully

# We can wait for this because data moves here

#



$TSM_QUERY "BACKUP DB DEVCLASS=LIBTAPE TYPE=FULL WAIT=YES" | grep "^AN[RS]" | grep -v "4554I"

echo ""



#

# Backup the Volume History

#



$TSM_QUERY "BACKUP DEVCONF FILE=/tsmdrm01/bu_configs/devconf.bak" | grep "^AN[RS]"

echo ""



#

# and wait for completion

#



waitproc 10 DevConf_Backup 'Backup Storage Pool'



#

#Backup the devices

#



$TSM_QUERY "BACKUP VOLHIST FILE=/tsmdrm01/bu_configs/volhist.bak" | grep "^AN[RS]"

echo ""



#

# and wait for completion

#



waitproc 10 VolHist_Backup 'Backup Storage Pool'



#

# Well thats the backups all done now we can start working on the tapes

#

# End of the output redirection grouping

#



} > ${REPORT_DIR}/${REPORT_NAME} 2>&1



#

# create the working picking list and get it ready for use

#



if [ -f $WORK_DIR/PREPTAPE ] ; then

rm $WORK_DIR/PREPTAPE

fi



#

# create the mounted tape list and get it ready for use

# then dismount all except the IN USE tapes

#



Sanity=0



while [ $Sanity -lt 15 ] ; do

BusyList="BusyDUMY"

Tapewait=1



$TSM_QUERY 'q mount' | awk '/^ANR8329|^ANR8330|^ANR8331/{print $4 " " $NF}' | tr -d "\." > $WORK_DIR/MOUNTEDTAPE



for IDLETAPE in `awk '/IDLE$/ { print $1" " }' < $WORK_DIR/MOUNTEDTAPE`

do

#

# Idle tape processing

# Try to get them dismounted

# Extend wait to at least 60 seconds

# the 3494 isn't Fast

#



$TSM_QUERY "DISMOUNT VOLUME $IDLETAPE"



Tapewait=60

done



for BUSYTAPE in `awk '/USE$/ { print $1" " }' < $WORK_DIR/MOUNTEDTAPE`

do

#

# Not much we can do other than record the busy tapes

# for later reporting

# Extend the wait in the hope that they become free

# before we finish

#



BusyList="${BUSYTAPE}|${BusyList}"



Tapewait=60

done







for DISMTAPE in `awk '/DISMOUNTING$/ { print $1" " }' < $WORK_DIR/MOUNTEDTAPE`

do

#

# Dismounting tapes

# just extend the wait so they will be dismounted

# Hopefully on the next loop

#



Tapewait=60

done



#

# Increment the Sanity variable

# We dont want to loop forever here

#



let Sanity=Sanity+1



#

# Now snooze to let things happen

#



sleep $Tapewait

done



#

# Blank out out scan files

#



cp /dev/null ${CDIRECT}/PTSM3.fil

cp /dev/null ${CDIRECT}/RTSM3.fil

cp /dev/null ${CDIRECT}/ITSM3.fil

cp /dev/null ${CDIRECT}/EXPE3.fil

cp /dev/null ${CDIRECT}/BUSY3.fil

if [ "${BusyList}" = "" ] ; then

BusyList="BusyDUMY"

fi



#

# Record format for the Scan file

# Cols Content

# 1 Blank

# 2-5 'CUST'

# 6 Blank

# 7-8 'CT' = Cartridge

# 9 Blank

# 10-15 'Rnnnnn' Volume Name

# 16-64 Blank

# 65 'D' = DCP or 'A' =Archive

# 66 T = TSM [Groups 1,2 or 3.]

# 67 '1-9' RECALL Priority

# DCP-COPYPOOL=1, OFFSITE-COPYPOOL=5 , 7YEAR-COPYPOOL=9

# Tapes that are Mounted & IN USE are recorded in the previous loop

# Tapes in COURIERRETRIEVE are expected back from an earlier run

# These are output as BUSYRnnnn

#

#$TSM_QUERY "select volume_name,state,stgpool_name,'DUMY', date(upd_date) from drmedia" | awk '

#!/^R[0-9][0-9][0-9][0-9][0-9]|104[1-4][0-9][0-9]/ {next} ;

#/'$BusyList'/ {printf"BUSY %6s\n",$1 ; next } ;

#$2=="MOUNTABLE" {printf"MCUST CT %6s",$1 } ;

#$2=="VAULTRETRIEVE" {printf"RCUST CT %6s",$1 } ;

#$2=="VAULT" {printf "VCUST CT %6s",$1 } ;

#$2=="COURIERRETRIEVE" {printf "EXPECT %6s",$1 } ;

#$3=="DUMY" {printf "%-49sDT1 \n"," DB-BACKUP " ;next } ;

#$3=="DCP-COPYPOOL" {printf "%-49sDT1 \n"," DCP-COPYPOOL " ;next } ;

#$3=="SYSBACK-COPYPOOL" {printf "%-49sDT1 \n"," SYSBACK-COPYPOOL " ;next } ;

#$3=="LANFREE_DCP_COPYP-" {printf "%-49sDT1 \n"," LANFREE_DCP_COPYPOOL " ;next } ;

#$3=="7YEAR-COPYPOOL" {printf "%-49sDT9 \n"," 7YEAR-COPYPOOL " ;next } ;

#$3=="OFFSITE-COPYPOOL" {printf "%-49sDT5 \n"," OFFSITE-COPYPOOL " ;next } ' | sort > $WORK_DIR/PREPTAPE



$TSM_QUERY "select volume_name,state,stgpool_name,'DUMY', date(upd_date) from drmedia" | awk '

!/^R[0-9][0-9][0-9][0-9][0-9]|104[1-4][0-9][0-9]/ {next} ;

/'$BusyList'/ {printf"BUSY %6s\n",$1 ; next } ;

$2=="MOUNTABLE" {printf"MCUST CT %6s",$1 } ;

$2=="VAULTRETRIEVE" {printf"RCUST CT %6s",$1 } ;

$2=="VAULT" {printf "VCUST CT %6s",$1 } ;

$2=="COURIERRETRIEVE" {printf "EXPECT %6s",$1 } ;

$3=="DUMY" {printf "%-30s%-19sDT1 \n"," DB-BACKUP ", $NF ;next } ;

$3=="DCP-COPYPOOL" {printf "%-30s%-19sDT1 \n"," DCP-COPYPOOL " , $NF;next } ;

$3=="SYSBACK-COPYPOOL" {printf "%-30s%-19sDT1 \n"," SYSBACK-COPYPOOL " , $NF;next } ;

$3=="LANFREE_DCP_COPYP-" {printf "%-30s%-19sDT1 \n"," LANFREE_DCP_COPYPOOL " , $NF;next } ;

$3=="7YEAR-COPYPOOL" {printf "%-30s%-19sDT9 \n"," 7YEAR-COPYPOOL " , $NF;next } ;

$3=="OFFSITE-COPYPOOL" {printf "%-30s%-19sDT5 \n"," OFFSITE-COPYPOOL " , $NF;next } ' | sort > $WORK_DIR/PREPTAPE





$TSM_QUERY "select volume_name,status,stgpool_name,location,date(LAST_WRITE_DATE) from volumes where STGPOOL_NAME like '%ARCHIVE%' " | awk '

!/^R[0-9][0-9][0-9][0-9][0-9]|104[1-4][0-9][0-9]/ {next} ;

/Arcus/ {next} ;

/FULL|FILLING/ {printf"ACUST CT %6s%-30s%-19sDT0 \n",$1," ARCHIVAL_TAPE " ,$NF ;next } ' | sort >> $WORK_DIR/PREPTAPE



$TSM_QUERY "select volume_name,status,stgpool_name,location,date(LAST_WRITE_DATE) from volumes where STGPOOL_NAME like '%ARCHIVE%' " | awk '

!/^R[0-9][0-9][0-9][0-9][0-9]|104[1-4][0-9][0-9]/ {next} ;

!/Arcus/ {next} ;

/FULL|FILLING/ {printf"VCUST CT %6s%-30s%-19sDT0 \n",$1," ARCHIVAL_TAPE ",$NF ;next } ' | sort >> $WORK_DIR/PREPTAPE



#

# The PREPTAPE file s a scanfile record with a prefix letter on each record as follows

# M = Mountable tapes to be checked out

# R = Recalled tapes coming back on site

# V = Vault tapes at arcus

#

# this prefix is substituted by a space later and is only used to sort the tapes in the previous step

# The list is also generated to checkout the individual volumes

#



grep "^MCUST" $WORK_DIR/PREPTAPE | sed "s/MCUST/ CUST/" > ${CDIRECT}/PTSM3.fil

grep "^ACUST" $WORK_DIR/PREPTAPE | sed "s/ACUST/ CUST/" >> ${CDIRECT}/PTSM3.fil

grep "^RCUST" $WORK_DIR/PREPTAPE | sed "s/RCUST/ CUST/" > ${CDIRECT}/RTSM3.fil

grep "^VCUST" $WORK_DIR/PREPTAPE | sed "s/VCUST/ CUST/" > ${CDIRECT}/ITSM3.fil

grep "^EXPE" $WORK_DIR/PREPTAPE > ${CDIRECT}/EXPE3.fil

grep "^BUSY" $WORK_DIR/PREPTAPE > ${CDIRECT}/BUSY3.fil



grep "^MCUST" $WORK_DIR/PREPTAPE | awk '

/DT1|DT3|DT5|DT9/ {printf "MOVE DRMEDIA %s WHERESTATE=MOUNTABLE TOSTATE=VAULT WAIT=YES\n",$3 ; next } ;

/xxxxxx/ {printf "CHECKOUT LIBVOL WM1LIB11 %s\n",$3 ; next } ;

' > ${WORK_DIR}/CHECKOUTS.wrk



#

# We need to move the vault retrieve tapes onsite too so to make life easier

# we'll add them onto the checkout routine file

# Changed to courier retrieve

#



grep "^RCUST" $WORK_DIR/PREPTAPE | awk '

/DT1|DT3|DT5|DT9/ {printf "MOVE DRMEDIA %s WHERESTATE=VAULTRETRIEVE TOSTATE=COURIERRETRIEVE WAIT=YES\n",$3 ; next }

' >> ${WORK_DIR}/CHECKOUTS.wrk



#

# We need to check out the archive tapes a little differently

#



grep "^ACUST" $WORK_DIR/PREPTAPE | awk '

/DT0/ {printf "UPDATE VOLUME %s ACCESS=UNAVAILABLE LOCATION=Arcus\\\(781-273-9500-Cust#CUST\\\)\n",$3

printf "CHECKOUT LIBVOLUME WM1LIB11 %s\n",$3 ; next }

' >> ${WORK_DIR}/CHECKOUTS.wrk



#

# Start of the output redirection grouping

#



{

if [ -s ${CDIRECT}/PTSM3.fil ] ; then

while read command_line

do

#echo "would be doing ${command_line}"

$TSM_QUERY ${command_line} | awk '

/^ANS8000I/ {printf "%s %s %s", $4, $5, $6 } ;

/^ANS8002I/ {printf " RC=%s\n", $6 }

' | tr -d "'"

done < ${WORK_DIR}/CHECKOUTS.wrk

fi

#

# Keep this inside the report loop but away from the summary

#

banner "Media To" Offsite

echo "\n\nThe following tapes are being sent offsite\n\n"

if [ -s ${CDIRECT}/PTSM3.fil ] ; then

cat ${CDIRECT}/PTSM3.fil

else

echo "List of tapes being sent offsite is empty!"

fi

echo "\n\nEnd Of List - Tapes Being Sent Offsite # tapes being despatched = `wc -l ${CDIRECT}/PTSM3.fil` \n\n"



#

# There is no need to report busy tapes if there are none

#



if [ -s ${CDIRECT}/BUSY3.fil ] ; then

banner "Media Busy"

echo "\n\nThe following tapes are mounted IN USE & cannot be sent offsite\n\n"

cat ${CDIRECT}/BUSY3.fil

echo "\n\nEnd Of List - Busy Tapes # tapes Busy Status = `wc -l ${CDIRECT}/BUSY3.fil` \n\n"

else

echo "\n\nThere are No Busy Tapes to Report \n\n"

fi



#

# List the recalled tapes

#



banner "Media From" Offsite

echo "\n\nThe following tapes are being returned from offsite\n\n"

if [[ -s ${CDIRECT}/RTSM3.fil ]] ; then

cat ${CDIRECT}/RTSM3.fil

else

echo "List of tapes being recalled from offsite is empty!"

fi

echo "\n\nEnd Of List - Tapes Being Returned From Offsite # tapes being recalled = `wc -l ${CDIRECT}/RTSM3.fil`\n\n"



#

# List the Expected tapes

#



banner Expected Returns

echo "\n\nThe following tapes should already have been returned from offsite\n\n"

if [[ -s ${CDIRECT}/EXPE3.fil ]] ; then

cat ${CDIRECT}/EXPE3.fil

else

echo "List of tapes expected already to be returned from offsite is empty!"

fi

echo "\n\nEnd Of List - Tapes Already Expected From Offsite = `wc -l ${CDIRECT}/EXPE3.fil`\n\n"



#

#

#



banner Vaulted Media

echo "\n\nThe following tapes are remaining in the vault offsite\n\n"

if [[ -s ${CDIRECT}/ITSM3.fil ]] ; then

cat ${CDIRECT}/ITSM3.fil

else

echo "List of tapes in vault is empty!!!"

fi

echo "\n\nEnd Of List - Tapes Remaining In Vault # tapes being held offsite = `wc -l ${CDIRECT}/ITSM3.fil`\n\n"



#

# Now all the tapes are out we can run the DRM Prepare

#



banner "DRM PREP"

echo "\n\nOutput from the DRM prepare command\n\n"

$TSM_QUERY "PREPARE WAIT=YES"

echo "\n\nEnd of DRM prepare command output\n\n"



#

# Lets transfer the scan files to the NT Server for arcus

#



/home/cdadmin/unix2nt.ksh



#

# Lets copy the prepare file over to all 3 CWS Nodes

#



PREPFILE=`ls -rt ${DRMDIR}/recoveryplans/prepare.* | tail -1`



for PREPNODE in ${PREPTGT}

do

rcp -p ${PREPFILE} ${PREPNODE}:/tsmbase/sysmgr/DCP/TSM/. && echo "Copied ${PREPFILE} to ${PREPNODE}"

echo "Housekeeping has removed the following file(s) on node ${PREPNODE}"

rsh ${PREPNODE} " find /tsmbase/sysmgr/DCP/TSM/prepare.* -mtime +7 -print -exec rm {} \; "

# rsh ${PREPNODE} "ls /tsmbase/sysmgr/DCP/TSM"

done



#

# End of the output redirection grouping

#



} >> ${REPORT_DIR}/${REPORT_NAME} 2>&1



#

# Tidy up out local prepare files keep 30 days for reference

#



find ${DRMDIR}/recoveryplans/prepare.* -mtime +30 -exec rm {} \;



#

# Now lets make sure we keep a few ( currently 6 ) old versions

#



for newver in 6 5 4 3 2

do

(( oldver = newver - 1 ))

if [ -f ${REPORT_DIR}/${REPORT_NAME}.${newver} ] ; then

rm ${REPORT_DIR}/${REPORT_NAME}.${newver}

fi

if [ -f ${REPORT_DIR}/${REPORT_NAME}.${oldver} ] ; then

mv ${REPORT_DIR}/${REPORT_NAME}.${oldver} ${REPORT_DIR}/${REPORT_NAME}.${newver}

fi

done



#

# make sure we get a copy of the current version in the archive cycle

#



cp -p ${REPORT_DIR}/${REPORT_NAME} ${REPORT_DIR}/${REPORT_NAME}.1



#

# now lets finaly mail this report away

#



mailx -s "TSM ${SERVER_NAME} offsite tapes" "$recipients" < ${REPORT_DIR}/${REPORT_NAME}

 

[shcart]

the hashes mean something when viewed in fixed font and be careful the mailer used wrapped lines it shouldnt have