Networker

Re: [Networker] Extended retention time on EDL

2009-09-03 15:48:45
Subject: Re: [Networker] Extended retention time on EDL
From: Anacreo <anacreo AT GMAIL DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Thu, 3 Sep 2009 14:42:41 -0500
Here is the script I'm using to find those recycleable EDL tapes... I've not
had to complete the NDMP portion so ignore that dead end bit of code, 'if [
$POOL -eq "DailyEDL" ]; then', prevents the NDMP pool from actually being
processed.

DailyEDL is your EDL pool, NDMPEDL is your NDMP data on EDL...  The number
200000000000, is any portion of a save set that is part of a whole save set
over 200gb, don't bother moving... this should be a variable at the top
perhaps.

  The output will look something like:

Your "cron" job on uschi1leg01

/nsr/admin/SCRIPTS/cloning/find_recycle_candidates.sh



produced the following output:



VA000025 7.72903 DailyEDL

6365:nsrstage: Space can only be recovered from adv_file and file type
devices.
Successfully deleted original clone 1247990613 of save set 325243544 from
media database.

Successfully deleted original clone 1248051997 of save set 1432601140 from
media database.

Recovering space from volume 727869823 failed with the error 'Space can only
be recovered from adv_file and file type devices.'.

VA000049 0.023451 DailyEDL

6365:nsrstage: Space can only be recovered from adv_file and file type
devices.

Successfully deleted original clone 1248484038 of save set 2372557207 from
media database.

Recovering space from volume 963189701 failed with the error 'Space can only
be recovered from adv_file and file type devices.'.
...

Good luck, let me know if you have any issues with this.

Here is the program:
find_recycle_candidates.sh:
#!/bin/sh

# Prevent code from running twice
progname=`basename $0`
pidcount=`pgrep $progname| wc -l`
if [ $pidcount -gt 1 ]; then
  echo "$progname is already running." 1>&2;
  exit 1
fi

# Identify tapes to check
TAPES=`mminfo -r volume -q 'pool=DailyEDL,pool=NDMPEDL,ssrecycle,volretent>next
week,full'`

# Check if TAPES exist...
if [ ! "$TAPES" ]; then
  exit
fi

space_used ()
{
  USED=`mminfo -r fragsize'(15)' -q volume=$1,\!ssrecycle | awk -e '{total =
total + $1} END {print total/1024/1024/1024}
'`
}

get_pool ()
{
  POOL=`mminfo -r pool -q volume=$1`
}

get_non_expired_ssid ()
{
  NONEXPIREDSSID=`mminfo -r ssid,cloneid -xc/ -q volume=$1,!ssrecycle | sed
-e 1d`
}

get_totalsize ()
{
  TOTALSIZE=`mminfo -r totalsize -q ssid=$1`
}

# Scan through each tape and get the size
for TAPE in $TAPES; do
  space_used $TAPE
  if [ $USED -lt 20 ]; then
    get_pool $TAPE
    echo $TAPE $USED $POOL
    if [ $POOL -eq "DailyEDL" ]; then
      get_non_expired_ssid $TAPE
      if [ "$NONEXPIREDSSID" ]; then
        for SSID in $NONEXPIREDSSID; do
          get_totalsize $SSID
          if [ "$TOTALSIZE" -gt 200000000000 ]; then
            echo "\tSkipping $TAPE because $SSID \($TOTALSIZE\) is over the
limit."
            continue
          fi
        done
        nsrstage -b $POOL -m -S $NONEXPIREDSSID
      fi
    fi
  fi
done


On Wed, Sep 2, 2009 at 8:59 AM, Anacreo <anacreo AT gmail DOT com> wrote:

> I set my long retention in NW like normal 13 months or 4 weeks.  All
> my backups land on the EDL...  I run a script on Mondays to *push* all
> data older then 4 weeks off of the EDL onto tape.  For non-NDMP I can
> just do nsrstage...  For NDMP I do similar but have to clone and
> delete the original cloneid because nsrstage doesn"t work for NDMP.
>
> Also because the EDL is handled like tape I have an aggressive script
> that runs weekly to recycle tapes with less then 20gigs of data. If
> the total ssid size on that tape is less then 100g, which my EDL tape
> size.
>
> I've shared out these scripts before and will do it again upon request.
>
> Alec
>
> On 9/2/09, Chester Martin <cmartin AT spp DOT org> wrote:
> > We don't store any data on edl longer than 2 weeks, so my browse and
> > retention times are set to 2 weeks.  But we clone to physical tapes
> > daily.  The pool that it's cloned to has the retention times set to much
> > longer than 2 weeks.
> >
> > Unfortunately I can't increase the browse time to the same as the
> > retention time when it's cloned to physical tape (7.4.4), but I think I
> > can do it with 7.5.1, which I'll be upgrading to in the next few weeks.
> >
> > -----Original Message-----
> > From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT 
> > EDU] On
> > Behalf Of JKK
> > Sent: Tuesday, September 01, 2009 8:20 PM
> > To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> > Subject: [Networker] Extended retention time on EDL
> >
> > Hi.
> >
> > Those of you who are writing backups to EDL:
> >
> > What do you do about backups that need retention times longer that 2-3
> > months? Are you storing backups for longer than 3 months on EDL?
> >
> > What are alternate ways to store backups for years? clone to Tapes,
> > Archive
> > to disk?
> >
> > It would be interesting to see what methods have been tried and tested
> > for
> > years on this topic.
> >
> > Johannes
> >
> > To sign off this list, send email to listserv AT listserv.temple DOT edu and
> > type "signoff networker" in the body of the email. Please write to
> > networker-request AT listserv.temple DOT edu if you have any problems with 
> > this
> > list. You can access the archives at
> > http://listserv.temple.edu/archives/networker.html or
> > via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
> >
> > To sign off this list, send email to listserv AT listserv.temple DOT edu and
> type
> > "signoff networker" in the body of the email. Please write to
> > networker-request AT listserv.temple DOT edu if you have any problems with 
> > this
> > list. You can access the archives at
> > http://listserv.temple.edu/archives/networker.html or
> > via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
> >
>
> --
> Sent from my mobile device
>

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

<Prev in Thread] Current Thread [Next in Thread>