Networker

Re: [Networker] Re-using tape?

2009-12-28 19:39:01
Subject: Re: [Networker] Re-using tape?
From: Anacreo <anacreo AT GMAIL DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Mon, 28 Dec 2009 18:36:33 -0600
Actually some of the above statements are a bit misleading... if all save
sets are marked recycle and the tape is NOT marked as full, the tape will
not be recycled as it would remain in append mode.  The requirements are as
far as I can tell that the tape must be marked as full, all save sets must
be any mix of the following: no save sets, recycleable, aborted (I believe +
a few days), and/or incomplete.  I'm not sure if suspect save sets are ever
recycled automatically, but maybe so.

So if you have a save set you can get it to expire by changing its expire
time, nsrstage to move it off, clone it and delete it, or simply delete it
if you don't want the data, and then run nsrim to have NetWorker expire the
eligible tapes.

When tapes are to be labelled there are a few rules that are taken into
account...
If poola needs tapes it will look at: (not sure on this order if someone
knows better)
  1. Unlabeled tapes inside of jukebox with auto media management enabled
  2. (I suspect) Recyclable tapes within the media pool that needs tapes.
  3. Online recycleable tapes across any pool that allows "Recycle to other
pools", and if "Recycle from other pools" is enabled for the pool that
requires tapes.
      Does anyone know for sure if they are picking tapes with the oldest
last access time, or preferring tapes from the current pool, etc?

I have a script that I use quite often to juggle around data as some clients
end up leaving behind "residue" on tapes, I believe due to corrupted indexes
NetWorker believes there is a dependency and fails to expire my save sets...
 (I should really open a case on these clients...)

I have an annoying problem of some of my offsite tapes getting left behind
for a very long time with very little data on it... So all my tapes come
back in on Wednesday, and on Friday's I have this program
run, find_recycle_candidates.sh, and basically searches for tapes with less
then 20g of unexpired data in the SL500 and shuffles it onto a new tape in
the same pool (our purge policy is 4 weeks and tapes remain offsite for 8
weeks, so everything should be expired except for our 13 month retention
data).   Below that I have a script that I call from a web page that
suggests tapes that can be inserted into the SL500 that would be eligible
for the previous script.  Please let me know if you find the data helpful,
it gets a bit trickier for NDMP data as the nsrstage command won't fork for
that, but its still doable using clone and delete.

!!! Something important to note is that when you clone save sets the ENTIRE
save set must be available to be cloned not just the fragment.  My script
could do a better job in this department of ensuring the entire save set is
online...  There is a stop measure in place to make sure that the entire
save set is not more then 200g of space.  !!!

------  find_recycle_candidates.sh -----
#!/bin/sh

STARTPOOL=SynOffsite
# Identify tapes to check
TAPES=`mminfo -r volume -q
"pool=$STARTPOOL,ssrecycle,volretent>today,full,location=SL500`

# Check if TAPES exist...
if [ ! "$TAPES" ]; then
  exit
fi

space_used ()
{
  USED=`mminfo -r fragsize'(15)' -q volume=$1,\!ssrecycle | awk -e '{total =
total + $1} END {print total/1024/1024/1024}'`
}

get_pool ()
{
  POOL=`mminfo -r pool -q volume=$1`
}

get_non_expired_ssid ()
{
  NONEXPIREDSSID=`mminfo -r ssid,cloneid -xc/ -q volume=$1,!ssrecycle | sed
-e 1d`
}

get_totalsize ()
{
  TOTALSIZE=`mminfo -r totalsize -q ssid=$1`
}

# Scan through each tape and get the size
for TAPE in $TAPES; do
  space_used $TAPE
  if [ $USED -lt 20 ]; then
    get_pool $TAPE
    echo $TAPE $USED $POOL
    if [ $POOL -eq "$STARTPOOL" ]; then
      echo "Destaging from $POOL to $POOL"
      get_non_expired_ssid $TAPE
      if [ "$NONEXPIREDSSID" ]; then
        for SSID in $NONEXPIREDSSID; do
          get_totalsize $SSID
          if [ "$TOTALSIZE" -gt 200000000000 ]; then
            echo "\tSkipping $TAPE because $SSID ($TOTALSIZE) is over the
limit."
            continue
          fi
        done
        nsrstage -b $POOL -m -S $NONEXPIREDSSID
      fi
    fi
  fi
done







------  suggest_synoffsite.sh -----
#!/bin/sh

STARTPOOL=SynOffsite
# Identify tapes to check
TAPES=`mminfo -r volume -q
"pool=$STARTPOOL,ssrecycle,volretent>today,full,location=""`

# Check if TAPES exist...
if [ ! "$TAPES" ]; then
  exit
fi

space_used ()
{
  USED=`mminfo -r fragsize'(15)' -q volume=$1,\!ssrecycle | awk -e '{total =
total + $1} END {print total/1024/1024/1024}'`
}

get_pool ()
{
  POOL=`mminfo -r pool -q volume=$1`
}

get_non_expired_ssid ()
{
  NONEXPIREDSSID=`mminfo -r ssid,cloneid -xc/ -q volume=$1,!ssrecycle | sed
-e 1d`
}

get_totalsize ()
{
  TOTALSIZE=`mminfo -r totalsize -q ssid=$1`
}

# Scan through each tape and get the size
for TAPE in $TAPES; do
  space_used $TAPE
  if [ $USED -lt 20 ]; then
    get_pool $TAPE
    # echo $TAPE $USED $POOL
    if [ $POOL -eq "$STARTPOOL" ]; then
      # echo "Destaging from $POOL to $POOL"
      get_non_expired_ssid $TAPE
      if [ "$NONEXPIREDSSID" ]; then
        for SSID in $NONEXPIREDSSID; do
          get_totalsize $SSID
          if [ "$TOTALSIZE" -gt 200000000000 ]; then
            echo "\tSkipping $TAPE because $SSID ($TOTALSIZE) is over the
limit."
            continue
          fi
        done
        # nsrstage -b $POOL -m -S $NONEXPIREDSSID
        echo "$TAPE"
      fi
    fi
  fi
done


On Mon, Dec 28, 2009 at 6:49 AM, Eric Fabulus <efabulus AT hotmail DOT com> 
wrote:

> Tapes are marked 100 % use because modern autochanger use barcode reader
> for exemple to know the type of the tape used (lto3 for exemple) and so the
> software do know the tape nominal capacity. However, Networker use
> compression. So when it says 100 % there maybe still be 50 % of the tape
> available.
>
> > Date: Tue, 22 Dec 2009 10:26:14 -0500
> > From: networker-forum AT BACKUPCENTRAL DOT COM
> > Subject: [Networker] Re-using tape?
> > To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> >
> > Tapes are re-used by re-labeling them into a media pool manually or
> automatically by the software. tapes are re-used only when all the savesets
> it contains are marked recyclable.
> >
> > a tape that's marked 100% will still have some space left in it and the
> software will pick up from there during the next backup until that tape is
> filled. at this point, the tape will be marked "full" and the "appendable"
> will disappear.
> >
> > it doesn't matter whether the tape is at 10 % or 50 % or full, the tape
> is re-used automatically (if so configured) once its savesets are
> recyclable. Manually re-using the tape should be done with caution.
> >
> >
> > opopanax
> >
> >
> >
> > Garcia wrote:
> > > Totally new to Legato Networker (7.4.4) and need to know how to make
> some tapes re-usable for backups. Tapes appear marked as appendable but the
> % Used still 100%, Is the backup job going to re-use these tapes or do I
> have to change something here in order to re-use them?? Step by step will be
> nice for this newbie . .  Hoping this is an easy one for all the experts in
> this cool forum.
> > > Thanks for your help. [Question]
> >
> >
> > +----------------------------------------------------------------------
> > |This was sent by carnegie.deli AT gmail DOT com via Backup Central.
> > |Forward SPAM to abuse AT backupcentral DOT com.
> > +----------------------------------------------------------------------
> >
> > To sign off this list, send email to listserv AT listserv.temple DOT edu and
> type "signoff networker" in the body of the email. Please write to
> networker-request AT listserv.temple DOT edu if you have any problems with 
> this
> list. You can access the archives at
> http://listserv.temple.edu/archives/networker.html or
> > via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>
> _________________________________________________________________
> Nouveau Windows 7 : Trouvez le PC qui vous convient. En savoir plus.
> http://windows.microsoft.com/shop
> To sign off this list, send email to listserv AT listserv.temple DOT edu and 
> type
> "signoff networker" in the body of the email. Please write to
> networker-request AT listserv.temple DOT edu if you have any problems with 
> this
> list. You can access the archives at
> http://listserv.temple.edu/archives/networker.html or
> via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

<Prev in Thread] Current Thread [Next in Thread>