Move data script for batch job

R

rpandey

ADSM.ORG Member
Joined
Feb 2, 2009
Messages
93
Reaction score
2
Points
0
Location
Australia
Hi
I found below script for the move data. This script is for move data from one volume at time. I need same sort of script but able to move data from multiple volumes at time, sleeps then checks if the existing move is done prior to starting another batch.

Can expert person help me to fix this script?


#!/bin/bash
dsmadmc -se=StanzaName -id=admin -pa=XXXXXX -tab -dataonly=yes "select count(volume_name) from volumes where stgpool_name='SrcStgPool' and STATUS='FULL'"
for volume in $(dsmadmc -se=StanzaName -id=admin -pa=XXXXXX -tab -dataonly=yes "select 'MARKER',volume_name from volumes where stgpool_name='SrcStgPool' and STATUS='FULL'" |grep '^MARKER' | cut -b :cool:; do
echo ${volume}
dsmadmc -se=StanzaName -id=admin -pa=XXXXXX move data ${volume} stg=DestStgPool wait=yes
done
 
Hard to help without being able to test it, but I imagine it would be something like this:

Bash: 
#!/bin/bash

stanza="StanzaName";

tsm_id="admin";

tsm_pa="XXXX";



src_stgpool="SrcStgPool";

dst_stgpool="DestStgPool";



max_processes=4;

sleep_interval=300;

progress_file="/tmp/TSM_move_data_progress.txt";





dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} -tab -dataonly=yes "select count(volume_name) from volumes where stgpool_name='${src_stgpool}' and STATUS='FULL'";

for volume in $(dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} -tab -dataonly=yes "select 'MARKER',volume_name from volumes where stgpool_name='${src_stgpool}' and STATUS='FULL'" |grep '^MARKER' | cut -b ; do

    echo ${volume};

    while true; do

        if [ $(grep -c "${stanza} ${src_stgpool} ${dst_stgpool}" ${progress_file}) -lt ${max_processes} ]; then

            dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} move data ${volume} stg=${dst_stgpool} wait=no;

            echo "${stanza} ${src_stgpool} ${dst_stgpool} ${volume}" >> ${progress_file};
            continue;

        else

            # see if anything is done

            while read L; do

                check_vol=$(echo -n "${L}" |awk '{print $4}');

                # This statement you have to test manually and see that you get only 1 returned count

                check_statement="SELECT count(*) FROM processes WHERE process="Move Data" AND STATUS LIKE '%${check_vol}%'";

                if [ $(dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} -tab -dataonly=yes "${check_statement}") == 0 ]; then

                    # remove it from the file

                    echo "Process for volume ${check_vol} seems to be done - removing it from file..";

                    sed -i "/^${stanza} ${src_stgpool} ${dst_stgpool} ${check_vol}/d" ${progress_file};

                fi

            done<<<$(grep "^${stanza} " ${progress_file})

            echo "sleeping ${sleep_interval} ";

            sleep ${sleep_interval};

        fi

    done

done


Not at all tested on my part, you will have to test and modify accordingly - but the logic should be something like that I think.

This part you have to test (can be tested like so):
check_statement="SELECT count(*) FROM processes WHERE process="Move Data" AND STATUS LIKE '%${check_vol}%'";
if [ $(dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} -tab -dataonly=yes "${check_statement}") == 0 ]; then echo "Process not found"; else echo "Process found"; fi
Hope it helps.

EDIT: trying to fix indentation
 
I can't seem to edit the post anymore, but I think also the "while true" line should be above the "for volume in" line. Again you will have to test and modify - but I guess it is a start :)
 
Hi
Below error occurred.

./movescript.ksh: line 50: Data AND STATUS LIKE '%/tsmdata/stg/dom_monthly/000BD12E.BFS%': No such file or directory
./movescript.ksh: line 52: [: ==: unary operator expected

line50: check_statement="SELECT count(*) FROM processes WHERE process="Move Data" AND STATUS LIKE '%${check_vol}%'";

line52: if [ $(dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} -tab -dataonly=yes "${check_statement}") == 0 ]; then
 
Hi,

Stupid mistake, but yes this should fix it (had unescaped double quotes within double quotes, so I changed it to single quotes):
check_statement="SELECT count(*) FROM processes WHERE process='Move Data' AND STATUS LIKE '%${check_vol}%'";

Line 52 I think is fine, that is if quotes are not messed up in ${check_statement} variable.

I noticed you have the extension .ksh as well; is it korn or bash shell you are using? If it is korn then it may require additional tuning, again not at all tested on my part.
 
Try this query before you do anything though:

SELECT count(*) FROM processes WHERE process="Move Data" AND STATUS LIKE '%/tsmdata/stg/dom_monthly/000BD12E.BFS%'

Assuming you have a process running for that one.
 
Hi
Here you go, as per your suggestion i have change "Move Data" to 'Move Data' with active volume name.

AALTSMWPH008> SELECT count(*) FROM processes WHERE process="Move Data" AND STATUS LIKE '%/tsmdata/stg/dom_monthly/000BD12E.BFS%'

ANR0162W Supplemental database diagnostic information: -1:42S22:-206 ([IBM][CLI Driver][DB2/LINUXX8664] SQL0206N "Move Data" is not valid in the context where it is used. SQLSTATE=42703~).
ANR2940E The command 'SELECT count ( * ) FROM processes WHERE process = "Move Data" AND STATUS LIKE '%/tsmdata/stg/dom_monthly/000BD12E.BFS%' ' contains an unknown SQL column name.


Return code 3.

Protect: AALTSMWPH008>SELECT count(*) FROM processes WHERE process='Move Data' AND STATUS LIKE '%/tsmdata/stg/dom_monthly/000BE725.BFS%'

Unnamed[1]
------------
1

 
HI
Here is what i have. There is same volume for four time. Those volume names are not deleting

1688972392182.png

1688972429793.png
 
Hi @scr1pt
Now i am getting below error. I could not fix this error.

/tsmdata/stg/aal_prod_db/00015B11.BFS
./move_05.sh: line 54: [: too many arguments
sleeping 300

Line 54: if [ $(dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} -tab -dataonly=yes "${check_statement}") == 0 ]; then
 
rpandey said:
Hi @scr1pt
Now i am getting below error. I could not fix this error.

/tsmdata/stg/aal_prod_db/00015B11.BFS
./move_05.sh: line 54: [: too many arguments
sleeping 300

Line 54: if [ $(dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} -tab -dataonly=yes "${check_statement}") == 0 ]; then
Click to expand...
Try maybe:
CHECK_TEST=$(dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} -tab -dataonly=yes "${check_statement}")
echo "Test is '${CHECK_TEST}'
What does it produce?

If it shows "Test is '0'" or "Test is '1'" then I guess changing to:
CHECK_TEST=$(dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} -tab -dataonly=yes "${check_statement}")
if [ "${CHECK_TEST}" == 0 ]; then

Might work.
 
Thanks for your update.
Now not deleting content from 'progress_file' . i tried with single and double quote. see below output

sed -i "/^${stanza} ${src_stgpool} ${dst_stgpool} ${check_vol}/d" ${progress_file};
auaal1382:/home/CORP/jrsp/ramesh $ ./move_05_final.sh
481
/tsmdata/stg/aal_prod_db/00016F23.BFS
Process for volume /tsmdata/stg/aal_prod_db/000254A7.BFS seems to be done - removing it from file..
sed: can't find label for jump to `smdata/stg/aal_prod_db/000254A7.BFS/d'
sleeping 300

sed -i '/^${stanza} ${src_stgpool} ${dst_stgpool} ${check_vol}/d' ${progress_file};
auaal1382:/home/CORP/jrsp/ramesh $ ./move_05_final.sh
481
/tsmdata/stg/aal_prod_db/00016F23.BFS
Process for volume /tsmdata/stg/aal_prod_db/000254A7.BFS seems to be done - removing it from file..
sleeping 300
 
I think this might be caused because of forward slashes in the string, let's try to escape them.

Try this:
sed_line="${stanza} ${src_stgpool} ${dst_stgpool} ${check_vol}"
sed_line=$(echo ${sed_line} |perl -pe "s/\//\\\\\//g")
sed -i "/^${sed_line}/d" ${progress_file};
 
Here is final working version -

#!/bin/bash

stanza="StanzaName";

tsm_id="admin";

tsm_pa="XXXX";



src_stgpool="SrcStgPool";

dst_stgpool="DestStgPool";

max_processes=4;

sleep_interval=300;

progress_file="/tmp/TSM_move_data_progress.txt";

dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} -tab -dataonly=yes "select count(volume_name) from volumes where stgpool_name='${src_stgpool}' and STATUS='FULL'";

while true; do

for volume in $(dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} -tab -dataonly=yes "select 'MARKER',volume_name from volumes where stgpool_name='${src_stgpool}' and STATUS='FULL'" |grep '^MARKER' | cut -b 12-); do

echo ${volume};

if [ $(grep -c "${stanza} ${src_stgpool} ${dst_stgpool}" ${progress_file}) -lt ${max_processes} ]; then

dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} move data ${volume} stg=${dst_stgpool} wait=no;

echo "${stanza} ${src_stgpool} ${dst_stgpool} ${volume}" >> ${progress_file};
continue;

else

# see if anything is done

while read L; do

check_vol=$(echo -n "${L}" |awk '{print $4}');

# This statement you have to test manually and see that you get only 1 returned count

check_statement="SELECT count(*) FROM processes WHERE Process='Move Data' AND STATUS LIKE '%${check_vol}%'";

CHECK_TEST=$(dsmadmc -se=${stanza} -id=${tsm_id} -pa=${tsm_id} -tab -dataonly=yes "${check_statement}")
if [ "${CHECK_TEST}" == 0 ]; then

# remove it from the file

echo "Process for volume ${check_vol} seems to be done - removing it from file..";

sed_line="${stanza} ${src_stgpool} ${dst_stgpool} ${check_vol}"
sed_line=$(echo ${sed_line} |perl -pe "s/\//\\\\\//g")
sed -i "/^${sed_line}/d" ${progress_file};

fi

done<<<$(grep "^${stanza} " ${progress_file})

echo "sleeping ${sleep_interval} ";

sleep ${sleep_interval};

fi

done

done
 
rpandey said:
Hi
I found below script for the move data. This script is for move data from one volume at time. I need same sort of script but able to move data from multiple volumes at time, sleeps then checks if the existing move is done prior to starting another batch.

Can expert person help me to fix this script?


#!/bin/bash
dsmadmc -se=StanzaName -id=admin -pa=XXXXXX -tab -dataonly=yes "select count(volume_name) from volumes where stgpool_name='SrcStgPool' and STATUS='FULL'"
for volume in $(dsmadmc -se=StanzaName -id=admin -pa=XXXXXX -tab -dataonly=yes "select 'MARKER',volume_name from volumes where stgpool_name='SrcStgPool' and STATUS='FULL'" |grep '^MARKER' | cut -b :cool:; do
echo ${volume}
dsmadmc -se=StanzaName -id=admin -pa=XXXXXX move data ${volume} stg=DestStgPool wait=yes
done
Click to expand...
add it to crontab
 
You must log in or register to reply here.

Data Privacy Impact Assessment

Compliance Assurance in Every Byte: Discover the Impact of Our DPIA Services.

Sponsor ADSM.ORG

If you are reading this, so are your potential customer. Advertise at ADSM.ORG right now.

Navigation Menu

Forums
   IBM TSM
   EMC Backup and Recovery
   Symantec NetBackup  
   SAN  
   NAS
   Platforms / OS
Mailing List Archives
   ADSM-L
User Groups

NordVPN 3 Months FREE

Help Support ADSM.ORG and Get NordVPN with 64% Off and 3 Months FREE.

NordVPN with 64% off + 3 months FREE

Forum statistics

Threads
31,974
Messages
135,448
Members
21,953
Latest member
jrogers
Back
Top