Veritas-bu

[Veritas-bu] list media by last written date

2003-01-23 14:11:14
Subject: [Veritas-bu] list media by last written date
From: Mark.Donaldson AT experianems DOT com (Donaldson, Mark)
Date: Thu, 23 Jan 2003 12:11:14 -0700
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C2C313.32B98850
Content-Type: text/plain

the one line below should read:

<12 hours until it expires:  Why pull something that's about to become
available for reuse? 

(...*less* than 12 hours...)

-M

-----Original Message-----
From: Donaldson, Mark [mailto:Mark.Donaldson AT experianems DOT com]

For daily tape rotations (pulling full tapes from the library) I've got a
script with a ruleset like this: more than 1-week since last written AND
more then 1 day since last read AND more than 12 hours until it expires AND
in library AND (full OR frozen OR suspended) AND NOT Scratch. 
It's written around vmquery & bpmedialist output. 

The rule choices are for these reasons: 
>1 week since written: Since I run a weekly full/incr cycle, the image has
probably been superceded by a more recent image - this keeps me from having
to insert tapes back in for ad-hoc restores since they tend to want the
latest image.

>1 day since last read: If it's been read very recently, I consider this an
"active" tape and assume it may be used again soon, therefore I don't pull
it.

>12 hours until it expires:  Why pull something that's about to become
available for reuse? 
in library: obvious - don't rotate out what's already out 

NOT Scratch: also obvious - don't pull your scratch tapes 

(full OR frozen OR suspended):  These are tapes that will not be appended
to.  My rule is to not pull partially full tapes that may still yet have
images added to them.

It takes two sets of queries to generate the list, one for the "in-library",
the other for all the expiration stuff.  

This uses a little C program that I was given called "seconds_since_epoch"
to generate the current unix time integer but I think Netbackup has a little
sub function of a major command for this, too.  I just don't remember what
it is.

Here's the c code to compile if you wish: 
$ cat seconds_since_epoch.c 
/*      This is an annoying little program that does nothing 
        more than print the current time since epoch as an 
        unsigned integer.  The value is used in comparisons 
        between the current time and when the tapes will expire. 
*/ 
#include <sys/types.h> 
#include <time.h> 
#include <stdio.h> 
int main() 
        { 
        time_t value=0; 
        printf("%u\n", time(&value)); 
        return(0); 
        } 
/* END */ 


Here's the selection script.  Change the top three variables to match
whatever time selection settings you want. 

$ cat ident_removable 
#!/bin/ksh 
#Default Hours since a backup was written to the tape 
HOURSWRITE=168 

#Hours within the tape will expire and be reusable 
HOURSEXP=12 

#Hours within the tape has not been read 
HOURSREAD=24 

PROGNAME=`basename $0` 
TMPFILE1=/tmp/$PROGNAME.1.$$ 
TMPFILE2=/tmp/$PROGNAME.2.$$ 

PATH=$PATH:/usr/openv/netbackup/bin/admincmd:/usr/openv/volmgr/bin:/usr/open
v/local 

#call my little C program to get current time in epoch seconds 
now=`seconds_since_epoch` 

# Get scrach pool number 
spname=`vmpool -listscratch | tail -1` 
 spnum=`vmpool -listall | \ 
         awk '{if ($2=="number:") {pnum=$3} 
               if ($2=="name:" && $3=="'$spname'") {print pnum}}'` 

#get list of media & filter 
bpmedialist -l -mlist | \ 
  awk 'BEGIN {  now='$now' 
           scpool='$spnum' 
            write='$HOURSWRITE' * 3600 
           expire='$HOURSEXP' * 3600 
             read='$HOURSREAD' * 3600 
 } 
     {if ( $13!=$scpool      && \ 
           now>=$6+write     && \ 
           now<=$7-expire    && \ 
           now>=$8+read      && \ 
          ( int($15/8)%2==1  || \ 
                $15%2==1     || \ 
            int($15/2)%2==1 )) {print $1}}' > $TMPFILE1 

# TMPFILE1 contains a list of matching tapes regardless 
# of location so now sort to only in-library stuff 

#Use vmquery to find all tapes in library 
vmquery -a -w | awk '$8!="-" {print $8"\t"$1}' >$TMPFILE2 

#Print header 
echo "Robot\tTape\n-----\t------" 
for tape in `cat $TMPFILE1` 
do 
  #Match selected bpmedialist tapes to vmquery tapes 
  awk '$2=="'$tape'" {print}' $TMPFILE2 
done | sort -n -k 1 

#Clean up 
[ -f $TMPFILE1 ] && rm $TMPFILE1 
[ -f $TMPFILE2 ] && rm $TMPFILE2 
exit 

HTH - Mark 

-----Original Message----- 
From: Steven L. Sesar [mailto:ssesar AT mitre DOT org] 
Sent: Wednesday, January 22, 2003 7:48 AM 
To: veritas-bu AT mailman.eng.auburn DOT edu 
Subject: [Veritas-bu] list media by last written date 


I need to write a tool, which will allow operations to determine tapes 
to pull out of a robot, in order to make room for scratch media. 
Is anyone actually doing this and would you be willing to share some 
suggestions with me? 
Thnkas, 
Steve 



_______________________________________________ 
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu 
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu 

------_=_NextPart_001_01C2C313.32B98850
Content-Type: text/html
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3DUS-ASCII">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
5.5.2653.12">
<TITLE>RE: [Veritas-bu] list media by last written date</TITLE>
</HEAD>
<BODY>

<P><FONT SIZE=3D2>the one line below should read:</FONT>
</P>

<P><FONT SIZE=3D2>&lt;12 hours until it expires:&nbsp; Why pull =
something that's about to become available for reuse? </FONT>
</P>

<P><FONT SIZE=3D2>(...*less* than 12 hours...)</FONT>
</P>

<P><FONT SIZE=3D2>-M</FONT>
</P>

<P><FONT SIZE=3D2>-----Original Message-----</FONT>
<BR><FONT SIZE=3D2>From: Donaldson, Mark [<A =
HREF=3D"mailto:Mark.Donaldson AT experianems DOT com">mailto:Mark.Donaldson@exp=
erianems.com</A>]</FONT>
</P>

<P><FONT SIZE=3D2>For daily tape rotations (pulling full tapes from the =
library) I've got a script with a ruleset like this: more than 1-week =
since last written AND more then 1 day since last read AND more than 12 =
hours until it expires AND in library AND (full OR frozen OR suspended) =
AND NOT Scratch. </FONT></P>

<P><FONT SIZE=3D2>It's written around vmquery &amp; bpmedialist output. =
</FONT>
</P>

<P><FONT SIZE=3D2>The rule choices are for these reasons: </FONT>
<BR><FONT SIZE=3D2>&gt;1 week since written: Since I run a weekly =
full/incr cycle, the image has probably been superceded by a more =
recent image - this keeps me from having to insert tapes back in for =
ad-hoc restores since they tend to want the latest image.</FONT></P>

<P><FONT SIZE=3D2>&gt;1 day since last read: If it's been read very =
recently, I consider this an &quot;active&quot; tape and assume it may =
be used again soon, therefore I don't pull it.</FONT></P>

<P><FONT SIZE=3D2>&gt;12 hours until it expires:&nbsp; Why pull =
something that's about to become available for reuse? </FONT>
<BR><FONT SIZE=3D2>in library: obvious - don't rotate out what's =
already out </FONT>
</P>

<P><FONT SIZE=3D2>NOT Scratch: also obvious - don't pull your scratch =
tapes </FONT>
</P>

<P><FONT SIZE=3D2>(full OR frozen OR suspended):&nbsp; These are tapes =
that will not be appended to.&nbsp; My rule is to not pull partially =
full tapes that may still yet have images added to them.</FONT></P>

<P><FONT SIZE=3D2>It takes two sets of queries to generate the list, =
one for the &quot;in-library&quot;, the other for all the expiration =
stuff.&nbsp; </FONT>
</P>

<P><FONT SIZE=3D2>This uses a little C program that I was given called =
&quot;seconds_since_epoch&quot; to generate the current unix time =
integer but I think Netbackup has a little sub function of a major =
command for this, too.&nbsp; I just don't remember what it =
is.</FONT></P>

<P><FONT SIZE=3D2>Here's the c code to compile if you wish: </FONT>
<BR><FONT SIZE=3D2>$ cat seconds_since_epoch.c </FONT>
<BR><FONT SIZE=3D2>/*&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; This is an annoying =
little program that does nothing </FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; more than =
print the current time since epoch as an </FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; unsigned =
integer.&nbsp; The value is used in comparisons </FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; between =
the current time and when the tapes will expire. </FONT>
<BR><FONT SIZE=3D2>*/ </FONT>
<BR><FONT SIZE=3D2>#include &lt;sys/types.h&gt; </FONT>
<BR><FONT SIZE=3D2>#include &lt;time.h&gt; </FONT>
<BR><FONT SIZE=3D2>#include &lt;stdio.h&gt; </FONT>
<BR><FONT SIZE=3D2>int main() </FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; { </FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; time_t =
value=3D0; </FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
printf(&quot;%u\n&quot;, time(&amp;value)); </FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
return(0); </FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; } </FONT>
<BR><FONT SIZE=3D2>/* END */ </FONT>
</P>
<BR>

<P><FONT SIZE=3D2>Here's the selection script.&nbsp; Change the top =
three variables to match whatever time selection settings you want. =
</FONT>
</P>

<P><FONT SIZE=3D2>$ cat ident_removable </FONT>
<BR><FONT SIZE=3D2>#!/bin/ksh </FONT>
<BR><FONT SIZE=3D2>#Default Hours since a backup was written to the =
tape </FONT>
<BR><FONT SIZE=3D2>HOURSWRITE=3D168 </FONT>
</P>

<P><FONT SIZE=3D2>#Hours within the tape will expire and be reusable =
</FONT>
<BR><FONT SIZE=3D2>HOURSEXP=3D12 </FONT>
</P>

<P><FONT SIZE=3D2>#Hours within the tape has not been read </FONT>
<BR><FONT SIZE=3D2>HOURSREAD=3D24 </FONT>
</P>

<P><FONT SIZE=3D2>PROGNAME=3D`basename $0` </FONT>
<BR><FONT SIZE=3D2>TMPFILE1=3D/tmp/$PROGNAME.1.$$ </FONT>
<BR><FONT SIZE=3D2>TMPFILE2=3D/tmp/$PROGNAME.2.$$ </FONT>
</P>

<P><FONT =
SIZE=3D2>PATH=3D$PATH:/usr/openv/netbackup/bin/admincmd:/usr/openv/volmg=
r/bin:/usr/openv/local </FONT>
</P>

<P><FONT SIZE=3D2>#call my little C program to get current time in =
epoch seconds </FONT>
<BR><FONT SIZE=3D2>now=3D`seconds_since_epoch` </FONT>
</P>

<P><FONT SIZE=3D2># Get scrach pool number </FONT>
<BR><FONT SIZE=3D2>spname=3D`vmpool -listscratch | tail -1` </FONT>
<BR><FONT SIZE=3D2>&nbsp;spnum=3D`vmpool -listall | \ </FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; awk =
'{if ($2=3D=3D&quot;number:&quot;) {pnum=3D$3} </FONT>
<BR><FONT =
SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&nbsp; if ($2=3D=3D&quot;name:&quot; &amp;&amp; =
$3=3D=3D&quot;'$spname'&quot;) {print pnum}}'` </FONT>
</P>

<P><FONT SIZE=3D2>#get list of media &amp; filter </FONT>
<BR><FONT SIZE=3D2>bpmedialist -l -mlist | \ </FONT>
<BR><FONT SIZE=3D2>&nbsp; awk 'BEGIN {&nbsp; now=3D'$now' </FONT>
<BR><FONT =
SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
scpool=3D'$spnum' </FONT>
<BR><FONT =
SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp; write=3D'$HOURSWRITE' * 3600 </FONT>
<BR><FONT =
SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
expire=3D'$HOURSEXP' * 3600 </FONT>
<BR><FONT =
SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp; read=3D'$HOURSREAD' * 3600 </FONT>
<BR><FONT SIZE=3D2>&nbsp;} </FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp; {if ( =
$13!=3D$scpool&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &amp;&amp; \ </FONT>
<BR><FONT =
SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
now&gt;=3D$6+write&nbsp;&nbsp;&nbsp;&nbsp; &amp;&amp; \ </FONT>
<BR><FONT =
SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
now&lt;=3D$7-expire&nbsp;&nbsp;&nbsp; &amp;&amp; \ </FONT>
<BR><FONT =
SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
now&gt;=3D$8+read&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &amp;&amp; \ </FONT>
<BR><FONT =
SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ( =
int($15/8)%2=3D=3D1&nbsp; || \ </FONT>
<BR><FONT =
SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&nbsp;&nbsp; $15%2=3D=3D1&nbsp;&nbsp;&nbsp;&nbsp; || \ =
</FONT>
<BR><FONT =
SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp; int($15/2)%2=3D=3D1 )) {print $1}}' &gt; $TMPFILE1 </FONT>
</P>

<P><FONT SIZE=3D2># TMPFILE1 contains a list of matching tapes =
regardless </FONT>
<BR><FONT SIZE=3D2># of location so now sort to only in-library stuff =
</FONT>
</P>

<P><FONT SIZE=3D2>#Use vmquery to find all tapes in library </FONT>
<BR><FONT SIZE=3D2>vmquery -a -w | awk '$8!=3D&quot;-&quot; {print =
$8&quot;\t&quot;$1}' &gt;$TMPFILE2 </FONT>
</P>

<P><FONT SIZE=3D2>#Print header </FONT>
<BR><FONT SIZE=3D2>echo &quot;Robot\tTape\n-----\t------&quot; </FONT>
<BR><FONT SIZE=3D2>for tape in `cat $TMPFILE1` </FONT>
<BR><FONT SIZE=3D2>do </FONT>
<BR><FONT SIZE=3D2>&nbsp; #Match selected bpmedialist tapes to vmquery =
tapes </FONT>
<BR><FONT SIZE=3D2>&nbsp; awk '$2=3D=3D&quot;'$tape'&quot; {print}' =
$TMPFILE2 </FONT>
<BR><FONT SIZE=3D2>done | sort -n -k 1 </FONT>
</P>

<P><FONT SIZE=3D2>#Clean up </FONT>
<BR><FONT SIZE=3D2>[ -f $TMPFILE1 ] &amp;&amp; rm $TMPFILE1 </FONT>
<BR><FONT SIZE=3D2>[ -f $TMPFILE2 ] &amp;&amp; rm $TMPFILE2 </FONT>
<BR><FONT SIZE=3D2>exit </FONT>
</P>

<P><FONT SIZE=3D2>HTH - Mark </FONT>
</P>

<P><FONT SIZE=3D2>-----Original Message----- </FONT>
<BR><FONT SIZE=3D2>From: Steven L. Sesar [<A =
HREF=3D"mailto:ssesar AT mitre DOT org">mailto:ssesar AT mitre DOT org</A>] 
</FONT>
<BR><FONT SIZE=3D2>Sent: Wednesday, January 22, 2003 7:48 AM </FONT>
<BR><FONT SIZE=3D2>To: veritas-bu AT mailman.eng.auburn DOT edu </FONT>
<BR><FONT SIZE=3D2>Subject: [Veritas-bu] list media by last written =
date </FONT>
</P>
<BR>

<P><FONT SIZE=3D2>I need to write a tool, which will allow operations =
to determine tapes </FONT>
<BR><FONT SIZE=3D2>to pull out of a robot, in order to make room for =
scratch media. </FONT>
<BR><FONT SIZE=3D2>Is anyone actually doing this and would you be =
willing to share some </FONT>
<BR><FONT SIZE=3D2>suggestions with me? </FONT>
<BR><FONT SIZE=3D2>Thnkas, </FONT>
<BR><FONT SIZE=3D2>Steve </FONT>
</P>
<BR>
<BR>

<P><FONT SIZE=3D2>_______________________________________________ =
</FONT>
<BR><FONT SIZE=3D2>Veritas-bu maillist&nbsp; -&nbsp; =
Veritas-bu AT mailman.eng.auburn DOT edu </FONT>
<BR><FONT SIZE=3D2><A =
HREF=3D"http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu"; =
TARGET=3D"_blank">http://mailman.eng.auburn.edu/mailman/listinfo/veritas=
-bu</A> </FONT>
</P>

</BODY>
</HTML>
------_=_NextPart_001_01C2C313.32B98850--