Networker

Re: [Networker] nsrclone

2003-12-23 13:19:59
Subject: Re: [Networker] nsrclone
From: Tarik El Mansouri <etarik AT YAHOO DOT COM>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Tue, 23 Dec 2003 19:19:52 +0100
I suppose you are using  something like :

echo "$ssids" | xargs -n 1 echo  | nsrclone -b CLONEPOOL -S -f -

The first thing you can do is to order the ssids (my mediafile/mediarec) you 
want to clone per tape. The list of the ssid can be generated by an mminfo 
commands ( -q copies=1 ) with arguments of your choice.
Then the cloning will be clone with 2 tape drives (one reading, one writing) 
and the cloned data will be multiplexed. With this option, you can run 3 
nsrclone sessions in parallel (6 busy drives), and the last one can be used for 
restores...

If you are using cloning ssid per ssid, it will not be very fast (something 
like : echo $ssids | xargs -n1 nsrclone -b CLONEPOOL -S ). In this case the 
cloned data is not multiplexed (interlaced).

You can have a look to the script attached, it may help you.

Regards.


Darren Dunham <ddunham AT TAOS DOT COM> wrote:
>
> Hi,
>
> How many instances of nsrclone can I run simultaneously ?

I don't know of a specific limit, but more than 1 certainly.

> We have 7 tapedrives and I would like to speedup cloning by grouping ssids
> and feeding them to several nsrclones, but when I run a second nsrclone
> command I get this message:
>
> NSR server backupservername: busy
> nsrclone: waiting 30 seconds then retrying

Find two savesets that each exist only on a single (different) volume.
Invoke nsrclone twice, once on each saveset. Do you have the same
problem?

I've certainly had multiple nsrclones running simultaneously.

--
Darren Dunham ddunham AT taos DOT com
Unix System Administrator Taos - The SysAdmin Company
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=




---------------------------------
Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français !
Testez le nouveau Yahoo! Mail

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
#!/bin/ksh
#
# Script used to execute an offline cloning to the storage node tahiti
#
# HISTORY
# -------
# Date        Author              Comment
# ----------  -----------------   ---------------
# 16/09/2003  Tarik EL MANSOURI   Initial version
#

#-------------------------------
#       Variables
#-------------------------------


# Source environnemnt
MENUPATH=/nsr/scripts
export MENUPATH
[ -z ${MENUPATH:=} ] && echo MENUPATH not set && return 1
. $MENUPATH/cfg/main.ini

JUKEBOX1=L180-01
export JUKEBOX1

LOGFILE=/nsr/scripts/log/offline_cloning.log
export LOGFILE
#-------------------------------
#       Main
#-------------------------------

# Test and update client to use CLONE_SN storage node
echo "-- $(date '+%d/%m/%y %H:%M:%S') Checking clone storage node settings for 
all clients"
nsradmin_print -t client -s name -x -u | while read i
do  [ "$(nsradmin_print -t client -s "clone storage nodes" -n $i -x -u | xargs 
echo)" != "$CLONE_SN" ] && {
       echo "    . Updating client $i to use clone storage node $CLONE_SN"
       nsradmin_update -t client -n $i -U "clone storage nodes: $CLONE_SN"
    }
done

# Test if all the backups are finished
echo "-- $(date '+%d/%m/%y %H:%M:%S') Checking that all backup are finished"
while :
do
  [ "$(nsradmin_print -t group -s status -x -u | xargs echo)" == "idle" ] && 
break
  echo "    . A backup group is still running, waiting ..."
  sleep 10
done

# Combine all groups together, as several groups can use the same tape
# And sort the ssids by tape and clone pool
echo "-- $(date '+%d/%m/%y %H:%M:%S') Getting save sets to clone for groups 
$(echo $CLONED_GROUPS)"
(
  for group in $CLONED_GROUPS
  do
    # Select the complete ssids that are not cloned
    mminfo -s $EBS_SERVER -a -o t -t "${CLONEPERIOD:=1 days ago}" -q 
'copies=1,!incomplete,group="'$group'",location="'$JUKEBOX1'"' -r volume,ssid 
2>/dev/null |
      awk 'NR > 1' |
      while read volume ssid
      do
        # Get the clone pool of the group
        clonepool=$(nsradmin_print -t group -n $group -s 'clone pool' -x -u)
        # Print the volume name, the clone pool name and the ssid
        [ ! -z ${clonepool:=} ] && [ "$clonepool" != "Default Clone" ] && echo 
"$volume $clonepool $ssid"
      done
  done
) |
  sort -k 1,2 |
  /usr/xpg4/bin/awk 'BEGIN {vol=""}
                     $1 == vol {printf " %s", $3}
                     $1 != vol && vol == "" { vol=$1 ; printf "%s %s %s", vol, 
$2, $3 }
                     $1 != vol && vol != "" { vol=$1 ; printf "\n%s %s %s", 
vol, $2, $3 }
                     END {print ""}'  |
  while read volume clonepool ssids
  do
     [ ! -z $clonepool ] && [ ! -z $ssids ] && {
       # Check the number of instances of cloning
       while :
       do  [ $(pgrep -x nsrclone  | wc -l) -lt $CLONE_PARALLELISM ] && break
           sleep 10
       done

       # Check if the volume is in the jukebox and mounts the tape
       mmlocate -s $EBS_SERVER -l -n $volume 2>/dev/null |
         awk 'BEGIN {rc=1} NR>1 && $1 == "'"$volume"'" && $2 == "'"$JUKEBOX1"'" 
{rc=0} END {exit rc}'  && {
          # Check that the tape is not in a shared storage node device
          nsrmm -s $EBS_SERVER | grep -w $volume | grep "rd=" >/dev/null 2>&1 
&& _nsrjb -j $JUKEBOX1 -u  $volume
          echo "    . Mounting the volume $volume in the jukebox $JUKEBOX1"
          _nsrjb -j $JUKEBOX1 -l $volume >/dev/null 2>&1

          echo "    . Starting cloning of the volume $volume to pool $clonepool 
for the following ssids"
          echo $ssids | xargs -n1 echo "       "
          echo $ssids | xargs -n1 echo |  nsrclone -s $EBS_SERVER -b $clonepool 
-S -f -  2>&1 &
       }
     }
  done


# Wait the last nsrclone to finish
while :
do [ $(pgrep -x nsrclone  | wc -l) -eq 0 ] && break
   sleep 10
done

# Check if the cloning has succeeded on all the ssids, and report failed ssids

echo "-- $(date '+%d/%m/%y %H:%M:%S') Checking that all the save sets have been 
successfully cloned."
$MENUPATH/sh/cloning_report.sh | mail  $MAIL_TO_ADMIN
echo "    Mail sent to $MAIL_TO_ADMIN"

[ -f $LOGFILE ] && mv $LOGFILE $LOGFILE.$(date '+%d')

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
<Prev in Thread] Current Thread [Next in Thread>