Veritas-bu

[Veritas-bu] Backing up to disk (and then tape ??)

2003-09-15 11:59:04
Subject: [Veritas-bu] Backing up to disk (and then tape ??)
From: Mark.Donaldson AT experianems DOT com (Donaldson, Mark)
Date: Mon, 15 Sep 2003 09:59:04 -0600
Here's my script for doing this.  It's pretty long but it has error
handling, extensive logging, etc.  It can use a configuration file that'll
lookup policy/sched pairs and cross reference a tape pool from that (since a
disk STU doesn't use pools).  The conf file looks simply like:

#  Class/policy  Sched    Volume_Pool
Oracle_Arcs     Archive   Ora_Arc
Archives              *   Arc
Pkgs              1year   Pkg

(white space separated, comments allowed)  You can use a "*" as a schedule
name to do just a policy with all its schedules.

It'll group images together into sets (GLOB variable) to keep the individual
bpduplicate jobs from running too long.  It'll also do multiple sets
simultaneously (SIMUL variabe) if you so desire.

It use bpimagelist to query the disk STU's imagelist, groups images by
destination tape pool, and then uses bpduplicate to create a second copy of
sets of those images.  If the second copy of an images is verified after the
duplication process, then it uses bpexpdate to expire the disk copy.  The
promotion of the tape copy to the primary restore image is automatic when
the disk image is expired.

Look it over, cherry-pick or use it as you will.

-M

#!/bin/ksh

PROGNAME=`basename $0`

#Default volume pool if config file doesn't have a match
DEFAULTPOOL="HSM"

#Number of simultaneous bpduplications to run (per pool)
SIMUL=2

#Filesystem for disk STU
DISK="/images"

#Destination Storage unit
STU="L3500_4"

#Number of images per duplication group per pool
GLOB=5

#email address for reports
MAILADDR=YOU AT YOURDOMAIN DOT COM

#Configuration file - lookup pools for class/sched pairs
CONFFILE=/usr/openv/local/$PROGNAME.conf

PATH=$PATH:/usr/openv/netbackup/bin/admincmd

LOGDIR=/usr/openv/netbackup/logs
LOGFILE=$LOGDIR/$PROGNAME.log.1.$$
DETAILS=$LOGDIR/$PROGNAME.log.2.$$
KEEPLOG=$LOGDIR/$PROGNAME.log.`date +%a`

LOCKFILE=/tmp/$PROGNAME.LCK

BIDFILE=/tmp/$PROGNAME.bidfile.$$
POOLFILE=/tmp/$PROGNAME.poolfile.$$
SPLITFILE=/tmp/$PROGNAME.set.

###################
## Function Defs ##
###################

verifytape () {
  # Function: Supply bpimageid, returns 0=not-on-tape, else n=copy on tape
  image=$1
  pattern=`echo "$DISK" | sed 's,\/,\\\/,g'`
  copy=`bpimagelist -backupid $image|\
        awk 'BEGIN {copy=0}
             {if ($1=="FRAG" && $9!~/^'$pattern'/) {copy=$2}}
             END {print copy}'`
  echo $copy
  return $copy
}

diskcopy () {
  # Function: Supply bpimageid, returns 0=not-on-disk, else n=copy on disk
  image=$1
  pattern=`echo "$DISK" | sed 's,\/,\\\/,g'`
  copy=`bpimagelist -backupid $image|\
        awk 'BEGIN {copy=0}
             {if ($1=="FRAG" && $9~/^'$pattern'/) {copy=$2}}
             END {print copy}'`
  echo $copy
  return $copy
}

dupit () {
  #Function to duplicate & verify a list of images given a bidfile
  dupfile=$1

  echo "## Start: `basename $dupfile`"
  # The copy below is to correct for the bpduplicate "bug" where
  # the bidfile is deleted after use by bpduplicate.
  cp ${dupfile} ${dupfile}.2

  sed 's/^/##    /' $dupfile
  echo "## Calling bpduplicate for `basename $dupfile`."
  bpduplicate -L $DETAILS -dp $POOL -dstunit $STU -Bidfile $dupfile
2>>$DETAILS
  RC=$?
  if [ $RC -ne 0 ]
  then
    MRC=1
    echo "## WRN: bpduplicate error: $RC."
  fi
  mv ${dupfile}.2 ${dupfile}
  echo "## Expire: `basename $dupfile`"
  for bpid in `cat $dupfile`
  do
    #Update the value
    retval=`verifytape $bpid`

    if [ $retval -eq 0 ]
    then
      #Duplication failed - no tape version
      echo "## WRN: Duplicate of $bpid not found on tape." | tee -a $DETAILS
      MRC=1
    else
      copynum=`diskcopy $bpid`
      if [ $copynum -ne 0 ]
      then
        echo "## Expiring copy number $copynum of $bpid." >>$DETAILS
        bpexpdate -d 0 -backupid $bpid -force -copy $copynum 2>>$DETAILS
        if [ $? -ne 0 ]
        then
          echo "## ERR: bpexpdate had non-zero return code for \"$bpid\"." |
tee -a $DETAILS
          MRC=1
        fi
      else
        echo "## ERR: lookup of disk copy number failed for $bpid." | tee -a
$DETAILS
        MRC=1
      fi
    fi
  done
  [ -f $dupfile ] && rm -f $dupfile
  echo "## Complete: `basename $dupfile`"
  return $MRC
}

cleanup () {
  #Function to clean up after trap or end of script
  echo "## Cleanup"
  [ -f $LOCKFILE ] && rm $LOCKFILE
  echo "## $PROGNAME Complete: `date`"

  #Delete old cumulative log file
  find $LOGDIR -name "$PROGNAME.log.[SMTWF]??" -mtime +5 -exec rm {} \;

  #Append to current cumulative log file
  [ -f $LOGFILE ] && cat $LOGFILE >>$KEEPLOG
  [ -f $DETAILS ] && cat $DETAILS >>$KEEPLOG

  if [ $MRC -ne 0 ]
  then
    subject="$PROGNAME: Errors"
  else
    subject="$PROGNAME: Output"
  fi

  cat $LOGFILE $DETAILS | mailx -s "$subject" $MAILADDR

  # Logfile Cleanup
  [ -f $LOGFILE ]  && rm $LOGFILE
  [ -f $DETAILS ]  && rm $DETAILS
  [ -f $BIDFILE ]  && rm $BIDFILE
  [ -f $POOLFILE ] && rm $POOLFILE
  [ -f ${SPLITFILE}?? ] && rm ${SPLITFILE}??
}

##########
## MAIN ##
##########

if [ -n "$1" ]
then
  #Something on command line - give usage statement
  echo "Usage: $PROGNAME"
  echo "\nThis script will duplicate images from a disk storage area to a "
  echo "set of tapes.  Usage should be regularily via cron as root."
else
  MRC=0

  #Init log files
  exec 1>$LOGFILE 2>&1
  echo "###### $PROGNAME Start: `date`\n" >$LOGFILE
  echo "\n###### $PROGNAME Detail Log: `date`\n" >$DETAILS

  if [ -f $LOCKFILE ]
  then
    echo "## LOCKFILE found - exiting : `date`" | tee -a $DETAILS
    mailx -s "$PROGNAME: Lockfile Warning" $MAILADDR <$LOGFILE
    MRC=1
  else
    date > $LOCKFILE
    trap "MRC=1;echo 'TRAPPED INTERRUPT-please wait for cleanup.';echo
'Background tasks may complete without futher logging' ; cleanup ; exit
$MRC" 1 2 15

    #program misbehaves if no conf file so make one if missing
    [ ! -f $CONFFILE ] && touch $CONFFILE

    echo "## Getting imagelist for media \"$DISK\"."
    images=`bpimmedia -mediaid $DISK | awk '$1=="IMAGE" {print $4}'`

    #For each backupid on the disk storage unit (NB Database lookup...)
    for bpid in $images
    do
      echo "## Beginning processing of bpid: $bpid" >>$DETAILS
      retval=`verifytape $bpid`
      if [ $retval -eq 0 ]
      then
        #Image not on tape yet
        #Lookup Volume pool info for class & schedule
        info=`bpimagelist -backupid $bpid | awk '$1=="IMAGE" {print
$7":"$11}'`
        class=`echo $info | cut -f1 -d:`
        sched=`echo $info | cut -f2 -d:`

        #Lookup pool for given policy & sched or policy & wildcard for sched
        POOL=`awk '{if ($1=="'$class'" && $2=="'$sched'") {print $3} else
{if ($1=="'$class'" && $2=="*" ) {print $3}}}' $CONFFILE|tail -1`
        if [ -z "$POOL" ]
        then
          #Failed lookup of pool from config file
          echo "## WRN: could not lookup a pool for this image, using
\"$DEFAULTPOOL\"." | tee -a $DETAILS
          POOL=$DEFAULTPOOL
          MRC=1
        fi
        echo "## INF: Assigning $bpid to pool \"$POOL\"." >>$DETAILS
        echo "$POOL $bpid" >>$POOLFILE
      else
        # Image is already on tape - must be a stray image from previous
        # duplication attempt that didn't complete expire process - delete
it.
        #
        # Remove this section if you don't want duplicated images deleted
        #
        copynum=`diskcopy $bpid`
        if [ $copynum -ne 0 ]
        then
          echo "## INF: Expiring stranded copy of $bpid." |tee -a $DETAILS
          bpexpdate -d 0 -backupid $bpid -force -copy $copynum 2>>$DETAILS
          if [ $? -ne 0 ]
          then
            echo "## ERR: bpexpdate had non-zero return code for \"$bpid\"."
| tee -a $DETAILS
            MRC=1
          fi
        else
          echo "## ERR: lookup of disk copy number failed for $bpid." | tee
-a $DETAILS
          MRC=1
        fi
      fi
    done
    # All backupids assigned to volume pools - start the duplications
    for POOL in `awk '{print $1}' $POOLFILE | sort -u`
    do
      lkup=`vmpool -listall -b | awk 'NR>2 && $1=="'$POOL'" {print $1}'`
      if [ -z "$lkup" ]
      then
        # Pool name doesn't exist so I will create it.
        # This is potentially bad but I want to avoid disk overflow.
        vmpool -add $POOL "Added by $PROGNAME" ANYHOST -1 -2
        echo "## WRN: Pool \"$POOL\" did not exist. Created automatically."
        MRC=1
      fi
      #Sort out only those images for this pool
      awk '$1=="'$POOL'" {print $2}' $POOLFILE | sort >$BIDFILE
      echo "\n\n### Duplicating pool \"$POOL\" with `wc -l $BIDFILE|awk
'{print $1}'` images: `date`."
      echo "## Starting pool \"$POOL\"."  >>$DETAILS

      #split the big image file into manageable portions
      split -l $GLOB $BIDFILE $SPLITFILE
      count=0
      for smallbid in `ls $SPLITFILE??`
      do
        echo "## INF: Pool \"$POOL\", set $count, fileset `basename
$smallbid`." | tee -a $DETAILS
        dupit $smallbid &
        sleep 5
        while [ `jobs | wc -l` -ge $SIMUL ]
        do
          sleep 30
        done
        count=`expr $count + 1`
      done
      #Wait for remaing jobs to complete
      while [ `jobs | wc -l` -gt 0 ]
      do
        sleep 30
      done
      echo "## INF: Done with pool \"$POOL\": `date`."
    #Go back for next pool name
    done
    cleanup
  fi
fi
exit $MRC


-----Original Message-----
From: Richard Hellier [mailto:rlh AT lsil DOT com]
Sent: Monday, September 15, 2003 8:35 AM
To: (veritas-bu AT mailman.eng.auburn DOT edu)
Subject: [Veritas-bu] Backing up to disk (and then tape ??)


Folks,
        In various recent threads, some people have mentioned using
"backup to disk" (and then (e.g.) dup'ing to tape) as a way of
accelerating backups, getting reduced backup windows etc.

        If you are doing this, could I ask you, please, to let me know
some details of what sort of equipment you're using, any gotchas with
this
approach?


        I'm asking because our current 1.8 Tbyte data volume looks set to 
grow to 5-6 Tbyte and I don't think our current approach (ATL P3000 +
5 DLT drives) will scale to that sort of data volume.