Veritas-bu

[Veritas-bu] multi bpstart/bpend scripts test (NBU3.4) example

2003-09-18 16:23:42
Subject: [Veritas-bu] multi bpstart/bpend scripts test (NBU3.4) example
From: scott.kendall AT abbott DOT com (scott.kendall AT abbott DOT com)
Date: Thu, 18 Sep 2003 15:23:42 -0500
This is a multipart message in MIME format.
--=_alternative 0070115B86256DA5_=
Content-Type: text/plain; charset="us-ascii"

for those that take this and run with it, a few things to think about...

1.) you're usually running the scripts to do something, like take down a 
database.  you have to make sure the database is down before ANY of the 
streams start their backups to get a useful backup.  you have to have 
subsequent streams know that the first stream is still trying to take down 
the database and wait for it to finish before they start, otherwise, 
depending on where you put the logic to take down the database, you'll 
either try to stop the database multiple times or you'll start subsequent 
streams before the database is completely down and not get a good backup.

2.) what happens if a stream dies and becomes requeued? you might want to 
have the bpend script only rm the temp file if the backup completed ok. 
otherwise, bpend deletes the temp file and then when the last active job 
completes, it sees no more temp files and starts the database back up. 
then when the requeud job goes active it will shutdown the database, 
backup the data, and start the database... but it won't be consistent with 
the other streams.

3.) and for that matter, what happens if all the jobs are queued and they 
only go active one at a time, or some finish before others go active 
(because of drives not available, mpx settings, jobs per client setting)? 
the database could go up and down several times before all streams are 
finished.  the script will probably need to use the variable for total 
streams (STREAM_COUNT) somehow (maybe to write another temp file during 
the bpend and make sure you have as many of these as total streams before 
bringing up the database)... only problem is I think this variable was 
added in 4.5.

4.) what happens if multi-stream backups from different Policies run at 
the same time and they both need to call scripts with this type of logic? 
the temp files are going to cause issues between the two Policies backups. 
 you're either going to have to put the temp files in unique directories 
or name the files in a unique way (maybe including the Policy name or 
something).

5.) what happens if the system crashes or there are NetBackup problems 
during this backup.  if you rely on something like an rm to clean up 
things during the bpend script only, you could get in a situation where 
the next backup runs but because the temp files are still there from the 
previous backup's bpstarts (that were never cleaned up), you would not 
shut down the database and would get a backup thinking it was ok.


- Scott





ida3248b AT post.cybercity DOT dk
Sent by: veritas-bu-admin AT mailman.eng.auburn DOT edu
09/18/2003 01:04 PM

 
        To:     veritas-bu AT mailman.eng.auburn DOT edu
        cc: 
        Subject:        [Veritas-bu] multi bpstart/bpend scripts test (NBU3.4) 
example


Hello All

As there have been a lot of interest in these scripts, I decided to mail 
them to the list.

These scripts are NOT tested with any kind of application/database, our 
customer decided to pay for a Netbackup-Extension after all.

But I think they should could get people started.

Regards
Michael

bpstart_notify

#
# Script for of test multi stream bpnotify scripts
#
# Michael Graff Andersen 
#
LOCKYES=`ls /tmp/.lock.*`
if [ -z "$LOCKYES" ]; then
echo "BEGIN BACKUP" >> /tmp/bpstart.log
fi
touch /tmp/.lock.$STREAM_NUMBER
echo $STREAM_NUMBER >> /tmp/bpstart.log


bpend_notify
#!/bin/ksh
#
# Script for of test multi stream bpnotify scripts
#
# Michael Graff Andersen 
#
echo $STREAM_NUMBER >> /tmp/bpend.log
rm /tmp/.lock.$STREAM_NUMBER
LOCKYES=`ls /tmp/.lock.*`
if [ -z "$LOCKYES" ]; then
echo "END BACKUP" >> /tmp/bpend.log
fi
--
Cybercity Webhosting (http://www.cybercity.dk)

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu



--=_alternative 0070115B86256DA5_=
Content-Type: text/html; charset="us-ascii"


<br><font size=2 face="Arial">for those that take this and run with it, a few 
things to think about...</font>
<br>
<br><font size=2 face="Arial">1.) you're usually running the scripts to do 
something, like take down a database. &nbsp;you have to make sure the database 
is down before ANY of the streams start their backups to get a useful backup. 
&nbsp;you have to have subsequent streams know that the first stream is still 
trying to take down the database and wait for it to finish before they start, 
otherwise, depending on where you put the logic to take down the database, 
you'll either try to stop the database multiple times or you'll start 
subsequent streams before the database is completely down and not get a good 
backup.</font>
<br>
<br><font size=2 face="Arial">2.) what happens if a stream dies and becomes 
requeued? you might want to have the bpend script only rm the temp file if the 
backup completed ok. otherwise, bpend deletes the temp file and then when the 
last active job completes, it sees no more temp files and starts the database 
back up. &nbsp;then when the requeud job goes active it will shutdown the 
database, backup the data, and start the database... but it won't be consistent 
with the other streams.</font>
<br>
<br><font size=2 face="Arial">3.) and for that matter, what happens if all the 
jobs are queued and they only go active one at a time, or some finish before 
others go active (because of drives not available, mpx settings, jobs per 
client setting)? &nbsp;the database could go up and down several times before 
all streams are finished. &nbsp;the script will probably need to use the 
variable for total streams (STREAM_COUNT) somehow (maybe to write another temp 
file during the bpend and make sure you have as many of these as total streams 
before bringing up the database)... only problem is I think this variable was 
added in 4.5.</font>
<br>
<br><font size=2 face="Arial">4.) what happens if multi-stream backups from 
different Policies run at the same time and they both need to call scripts with 
this type of logic? &nbsp;the temp files are going to cause issues between the 
two Policies backups. &nbsp;you're either going to have to put the temp files 
in unique directories or name the files in a unique way (maybe including the 
Policy name or something).</font>
<br>
<br><font size=2 face="Arial">5.) what happens if the system crashes or there 
are NetBackup problems during this backup. &nbsp;if you rely on something like 
an rm to clean up things during the bpend script only, you could get in a 
situation where the next backup runs but because the temp files are still there 
from the previous backup's bpstarts (that were never cleaned up), you would not 
shut down the database and would get a backup thinking it was ok.</font>
<br>
<br>
<br><font size=2 face="Arial">- Scott</font>
<br>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td>
<td><font size=1 face="sans-serif"><b>ida3248b AT post.cybercity DOT 
dk</b></font>
<br><font size=1 face="sans-serif">Sent by: veritas-bu-admin AT 
mailman.eng.auburn DOT edu</font>
<p><font size=1 face="sans-serif">09/18/2003 01:04 PM</font>
<br>
<td><font size=1 face="Arial">&nbsp; &nbsp; &nbsp; &nbsp; </font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; To: &nbsp; 
&nbsp; &nbsp; &nbsp;veritas-bu AT mailman.eng.auburn DOT edu</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; cc: &nbsp; 
&nbsp; &nbsp; &nbsp;</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; Subject: &nbsp; 
&nbsp; &nbsp; &nbsp;[Veritas-bu] multi bpstart/bpend scripts test (NBU3.4) 
example</font></table>
<br>
<br>
<br><font size=2 face="Courier New">Hello All<br>
<br>
As there have been a lot of interest in these scripts, I decided to mail <br>
them to the list.<br>
<br>
These scripts are NOT tested with any kind of application/database, our <br>
customer decided to pay for a Netbackup-Extension after all.<br>
<br>
But I think they should could get people started.<br>
<br>
Regards<br>
Michael<br>
<br>
bpstart_notify<br>
<br>
#<br>
# Script for of test multi stream bpnotify scripts<br>
#<br>
# Michael Graff Andersen <br>
#<br>
LOCKYES=`ls /tmp/.lock.*`<br>
if [ -z &quot;$LOCKYES&quot; ]; then<br>
echo &quot;BEGIN BACKUP&quot; &gt;&gt; /tmp/bpstart.log<br>
fi<br>
touch /tmp/.lock.$STREAM_NUMBER<br>
echo $STREAM_NUMBER &gt;&gt; /tmp/bpstart.log<br>
<br>
<br>
bpend_notify<br>
#!/bin/ksh<br>
#<br>
# Script for of test multi stream bpnotify scripts<br>
#<br>
# Michael Graff Andersen <br>
#<br>
echo $STREAM_NUMBER &gt;&gt; /tmp/bpend.log<br>
rm /tmp/.lock.$STREAM_NUMBER<br>
LOCKYES=`ls /tmp/.lock.*`<br>
if [ -z &quot;$LOCKYES&quot; ]; then<br>
echo &quot;END BACKUP&quot; &gt;&gt; /tmp/bpend.log<br>
fi<br>
--<br>
Cybercity Webhosting (http://www.cybercity.dk)<br>
<br>
_______________________________________________<br>
Veritas-bu maillist &nbsp;- &nbsp;Veritas-bu AT mailman.eng.auburn DOT edu<br>
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu<br>
</font>
<br>
<br>
--=_alternative 0070115B86256DA5_=--

<Prev in Thread] Current Thread [Next in Thread>