Veritas-bu

[Veritas-bu] Graphing NetBackup Statistics with RRD TOOL (wha t to graph?)

2002-02-06 09:13:18
Subject: [Veritas-bu] Graphing NetBackup Statistics with RRD TOOL (wha t to graph?)
From: msmiles AT agere DOT com (Mark Smiles)
Date: Wed, 06 Feb 2002 14:13:18 +0000
--------------060808040809050105050306
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

We are using Big Brother(bb) to monitor and graph services. I've only 
just started looking at monitoring netbackup in bb and confess i haven't 
done this yet. However, I did find some goodies on the deadcat url for 
some netbackup scripts that you may want to look at.
http://www.deadcat.net/

With regards to the graphing, we capture the logs and redirect this into 
the graph but output the log into /dev/null every 5 minutes.
BB can be found at  http://bb4.com/


Regards,
Mark

Timothy J. Arnold wrote:

>I am also looking at creating graphs for netbackup and other services.
>
>I am looking at using RRD Tool with Cacti as the front end. You said
>that there were other graphing tools which you could use, what are these
>tools?
>
>Can you explain how you generate the graphs, I presume you have a script
>which runs every 5 minutes and then invokes rrd tool to give it the new
>data?
>
>Timothy
>
>>-----Original Message-----
>>From: veritas-bu-admin AT mailman.eng.auburn DOT edu 
>>[mailto:veritas-bu-admin AT mailman.eng.auburn DOT edu] On Behalf Of 
>>Peter L. Buschman
>>Sent: 06 February 2002 11:31
>>To: SIBLEY, Ken R. - ACCOREL; 'NetBackup List'
>>Subject: RE: [Veritas-bu] Graphing NetBackup Statistics with 
>>RRD TOOL (wha t to graph?)
>>
>>
>>
>>Ken:
>>
>>Drive utilization is an easy one.   I might do that next.*
>>
>>Throughput statistics per drive are easy on Solaris (iostat 
>>-xnM) but much 
>>tougher on other systems since
>>NetBackup doesn't track this data.  Does anybody know if 
>>there are ways to 
>>get this data on Windows?
>>
>>Throughput per job is an easy calculation to make, but you 
>>end up with an 
>>average over the entire run of
>>the job, not the peaks and valleys over time.  RRD TOOL has 
>>ways of working 
>>around this, but the database
>>would have to be populated at every step interval to track 
>>the growth of 
>>the kbytes counter in the job record.
>>(This would probably necessitate running bpdbjobs too 
>>frequently.)  Right 
>>now, I can update on 5 minute intervals
>>going back 48 hours, but only having to run bpdbjobs once.  
>>My logging 
>>script is very efficient and not tied to
>>the time interval it graphs.
>>
>>Average throughput per job is a great one to add to a gannt 
>>chart that 
>>shows the time period for which the
>>job ran.  I like this thought a lot (a colored line for each 
>>jobid and the 
>>average throughput as a subscript.)
>>RRD TOOL isn't the right graphing tool for this one, but 
>>there are other 
>>graphing modules out there.*
>>
>>The wheels are really turning now..... there are some interesting 
>>directions we could go in terms of "backup visualization."
>>
>>--PLB
>>
>>
>>At 03:38 PM 02/05/2002 -0600, SIBLEY, Ken R. - ACCOREL wrote:
>>
>>>I would be very interested to see (in order of priority):
>>>Drive Utilization over time (very big)
>>>Throughput Statistics (per drive and per client/class)
>>>Job Statistics
>>>Number of available/up/down drives
>>>Client/Class active over time
>>>
>>>I think these would be great.
>>>
>>>Ken
>>>
>>>Ken Sibley
>>>Sr. Unix Administrator
>>>Accor Economy Lodging
>>>ksibley AT accorel DOT com
>>>469-737-3370
>>>
>>>>-----Original Message-----
>>>>From: Peter L. Buschman [mailto:plb AT iotk DOT com]
>>>>Sent: Tuesday, February 05, 2002 1:34 PM
>>>>To: Veritas-bu AT mailman.eng.auburn DOT edu
>>>>Subject: [Veritas-bu] Graphing NetBackup Statistics with RRD
>>>>TOOL (what
>>>>to graph?)
>>>>
>>>>
>>>>
>>>>All:
>>>>
>>>>I've recently been experimenting with using RRD TOOL for
>>>>storing NetBackup
>>>>statistics.
>>>>I finally got my first pair of jobs stats loggers and
>>>>graphers working
>>>>tonight.  You can see
>>>>the results here (although it isn't very impressive given my
>>>>small system
>>>>at home.)
>>>>
>>>>         http://www.iotk.com/nbux/rrd/jobstats.gif
>>>>
>>>>I think this type of time-series data tracking could be a valuable
>>>>operational resource for
>>>>backup administrators.  In that regard, I'd like to poll the
>>>>list and ask
>>>>what statistics would
>>>>you like to see tracked over time?
>>>>
>>>>The ones that come to my mind are:
>>>>
>>>>         + Job Statistics (extending my first example to
>>>>include per-error
>>>>tracking.)
>>>>         + Throughput Statistics (how much data am I backing
>>>>up over time?)
>>>>         + Media Statistics (how many full / expired / frozen
>>>>/ suspended,
>>>>etc, tapes in my media database.)
>>>>         + Size of catalogs.
>>>>         + Number of available / up / down drives.
>>>>         + Drive utilization over time.
>>>>
>>>>...and much, much more.  If you can quantify a variable at a
>>>>point in time,
>>>>it can be stored in an RRD and graphed.
>>>>
>>>>If you've ever seen NetBackup Advanced Reporter, you have an
>>>>idea of the
>>>>data that can be collected and
>>>>processed.  NBAR is an expensive option, however, and not all that
>>>>customizable.  I'd like to add to the tools
>>>>available at no cost to the user community.
>>>>
>>>>I'd be very interested to hear everyone's thoughts on this.
>>>>How useful
>>>>would you find this kind of extension
>>>>to the base NetBackup functionality?
>>>>
>>>>Best regards,
>>>>
>>>>--PLB
>>>>
>>>>
>>>>
>>>>_______________________________________________
>>>>Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
>>>>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>>>>
>>>_______________________________________________
>>>Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
>>>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>>>
>>_______________________________________________
>>Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
>>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>>
>>
>
>
>_______________________________________________
>Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>


--------------060808040809050105050306
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <title></title>
</head>
<body>
We are using Big Brother(bb) to monitor and graph services. I've only just
started looking at monitoring netbackup in bb and confess i haven't done
this yet. However, I did find some goodies on the deadcat url for some netbackup
scripts that you may want to look at.<br>
<a class="moz-txt-link-freetext" 
href="http://www.deadcat.net/";>http://www.deadcat.net/</a><br>
<br>
With regards to the graphing, we capture the logs and redirect this into
the graph but output the log into /dev/null every 5 minutes.<br>
BB can be found at &nbsp;<a class="moz-txt-link-freetext" 
href="http://bb4.com/";>http://bb4.com/</a><br>
<br>
<br>
Regards,<br>
Mark<br>
<br>
Timothy J. Arnold wrote:<br>
<blockquote type="cite" cite="mid:004401c1af04$75cb9210$77333fc1 AT ngfl.gov 
DOT uk">
  <pre wrap="">I am also looking at creating graphs for netbackup and other 
services.<br><br>I am looking at using RRD Tool with Cacti as the front end. 
You said<br>that there were other graphing tools which you could use, what are 
these<br>tools?<br><br>Can you explain how you generate the graphs, I presume 
you have a script<br>which runs every 5 minutes and then invokes rrd tool to 
give it the new<br>data?<br><br>Timothy<br><br></pre>
  <blockquote type="cite">
    <pre wrap="">-----Original Message-----<br>From: <a 
class="moz-txt-link-abbreviated" href="mailto:veritas-bu-admin AT 
mailman.eng.auburn DOT edu">veritas-bu-admin AT mailman.eng.auburn DOT edu</a> 
<br>[<a class="moz-txt-link-freetext" href="mailto:veritas-bu-admin AT 
mailman.eng.auburn DOT edu">mailto:veritas-bu-admin AT mailman.eng.auburn DOT 
edu</a>] On Behalf Of <br>Peter L. Buschman<br>Sent: 06 February 2002 
11:31<br>To: SIBLEY, Ken R. - ACCOREL; 'NetBackup List'<br>Subject: RE: 
[Veritas-bu] Graphing NetBackup Statistics with <br>RRD TOOL (wha t to 
graph?)<br><br><br><br>Ken:<br><br>Drive utilization is an easy one.   I might 
do that next.*<br><br>Throughput statistics per drive are easy on Solaris 
(iostat <br>-xnM) but much <br>tougher on other systems since<br>NetBackup 
doesn't track this data.  Does anybody know if <br>there are ways to <br>get 
this data on Windows?<br><br>Throughput per job is an easy calculation to make, 
but you <br>end up with an <br>average over the entire run of<br>the 
job, not the peaks and valleys over time.  RRD TOOL has <br>ways of working 
<br>around this, but the database<br>would have to be populated at every step 
interval to track <br>the growth of <br>the kbytes counter in the job 
record.<br>(This would probably necessitate running bpdbjobs too 
<br>frequently.)  Right <br>now, I can update on 5 minute intervals<br>going 
back 48 hours, but only having to run bpdbjobs once.  <br>My logging <br>script 
is very efficient and not tied to<br>the time interval it 
graphs.<br><br>Average throughput per job is a great one to add to a gannt 
<br>chart that <br>shows the time period for which the<br>job ran.  I like this 
thought a lot (a colored line for each <br>jobid and the <br>average throughput 
as a subscript.)<br>RRD TOOL isn't the right graphing tool for this one, but 
<br>there are other <br>graphing modules out there.*<br><br>The wheels are 
really turning now..... there are some interesting <br>directions we could go 
in terms of "backup v
isualization."<br><br>--PLB<br><br><br>At 03:38 PM 02/05/2002 -0600, SIBLEY, 
Ken R. - ACCOREL wrote:<br></pre>
    <blockquote type="cite">
      <pre wrap="">I would be very interested to see (in order of 
priority):<br>Drive Utilization over time (very big)<br>Throughput Statistics 
(per drive and per client/class)<br>Job Statistics<br>Number of 
available/up/down drives<br>Client/Class active over time<br><br>I think these 
would be great.<br><br>Ken<br><br>Ken Sibley<br>Sr. Unix Administrator<br>Accor 
Economy Lodging<br><a class="moz-txt-link-abbreviated" href="mailto:ksibley AT 
accorel DOT com">ksibley AT accorel DOT com</a><br>469-737-3370<br><br></pre>
      <blockquote type="cite">
        <pre wrap="">-----Original Message-----<br>From: Peter L. Buschman [<a 
class="moz-txt-link-freetext" href="mailto:plb AT iotk DOT com">mailto:plb AT 
iotk DOT com</a>]<br>Sent: Tuesday, February 05, 2002 1:34 PM<br>To: <a 
class="moz-txt-link-abbreviated" href="mailto:Veritas-bu AT mailman.eng.auburn 
DOT edu">Veritas-bu AT mailman.eng.auburn DOT edu</a><br>Subject: [Veritas-bu] 
Graphing NetBackup Statistics with RRD<br>TOOL (what<br>to 
graph?)<br><br><br><br>All:<br><br>I've recently been experimenting with using 
RRD TOOL for<br>storing NetBackup<br>statistics.<br>I finally got my first pair 
of jobs stats loggers and<br>graphers working<br>tonight.  You can see<br>the 
results here (although it isn't very impressive given my<br>small system<br>at 
home.)<br><br>         <a class="moz-txt-link-freetext" 
href="http://www.iotk.com/nbux/rrd/jobstats.gif";>http://www.iotk.com/nbux/rrd/jobstats.gif</a><br><br>I
 think this type of time-series data tracking could be a 
valuable<br>operational resource for<br
>backup administrators.  In that regard, I'd like to poll the<br>list and 
>ask<br>what statistics would<br>you like to see tracked over time?<br><br>The 
>ones that come to my mind are:<br><br>         + Job Statistics (extending my 
>first example to<br>include per-error<br>tracking.)<br>         + Throughput 
>Statistics (how much data am I backing<br>up over time?)<br>         + Media 
>Statistics (how many full / expired / frozen<br>/ suspended,<br>etc, tapes in 
>my media database.)<br>         + Size of catalogs.<br>         + Number of 
>available / up / down drives.<br>         + Drive utilization over 
>time.<br><br>...and much, much more.  If you can quantify a variable at 
>a<br>point in time,<br>it can be stored in an RRD and graphed.<br><br>If 
>you've ever seen NetBackup Advanced Reporter, you have an<br>idea of 
>the<br>data that can be collected and<br>processed.  NBAR is an expensive 
>option, however, and not all that<br>customizable.  I'd like to add to the 
>tools<br>available at 
no cost to the user community.<br><br>I'd be very interested to hear everyone's 
thoughts on this.<br>How useful<br>would you find this kind of extension<br>to 
the base NetBackup functionality?<br><br>Best 
regards,<br><br>--PLB<br><br><br><br>_______________________________________________<br>Veritas-bu
 maillist  -  <a class="moz-txt-link-abbreviated" href="mailto:Veritas-bu AT 
mailman.eng.auburn DOT edu">Veritas-bu AT mailman.eng.auburn DOT edu</a><br><a 
class="moz-txt-link-freetext" 
href="http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu";>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu</a><br><br></pre>
        </blockquote>
        <pre 
wrap="">_______________________________________________<br>Veritas-bu maillist  
-  <a class="moz-txt-link-abbreviated" href="mailto:Veritas-bu AT 
mailman.eng.auburn DOT edu">Veritas-bu AT mailman.eng.auburn DOT edu</a><br><a 
class="moz-txt-link-freetext" 
href="http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu";>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu</a><br></pre>
        </blockquote>
        <pre 
wrap="">_______________________________________________<br>Veritas-bu maillist  
-  <a class="moz-txt-link-abbreviated" href="mailto:Veritas-bu AT 
mailman.eng.auburn DOT edu">Veritas-bu AT mailman.eng.auburn DOT edu</a><br><a 
class="moz-txt-link-freetext" 
href="http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu";>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu</a><br><br><br></pre>
        </blockquote>
        <pre 
wrap=""><!----><br><br>_______________________________________________<br>Veritas-bu
 maillist  -  <a class="moz-txt-link-abbreviated" href="mailto:Veritas-bu AT 
mailman.eng.auburn DOT edu">Veritas-bu AT mailman.eng.auburn DOT edu</a><br><a 
class="moz-txt-link-freetext" 
href="http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu";>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu</a><br></pre>
        </blockquote>
        <br>
        </body>
        </html>

--------------060808040809050105050306--