Veritas-bu

[Veritas-bu] bpstart_notify & bpend_notify for Oracle

2002-09-09 12:39:19
Subject: [Veritas-bu] bpstart_notify & bpend_notify for Oracle
From: Mark.Donaldson AT experianems DOT com (Donaldson, Mark)
Date: Mon, 9 Sep 2002 10:39:19 -0600
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C2581F.7201DB70
Content-Type: text/plain;
        charset="iso-8859-1"

We do a similar thing - and yes the scripts involved were spawned in some
evil space in my core.

We have the advantage of having BCV's for some of our databases.  I place
the entire database in hot-backpup mode at once, split the BCV mirrors, then
restore the DB back to run-mode. I then mount and backup the BCV's on
another server.  The DB spends maybe 5 minutes in backup mode keeping us
from generating excess numbers of redo logs.  You could do the same thing
with filesystem snapshots or maybe split mirror plexes.

For hot-backup on servers that are not on BCV, I drive it from a cron on the
DB server.  We have a script that queries for all tablespaces in the DB.
The tablespaces are placed in hotbackup mode in sequence, and the datafiles
for that tablespace are sent.  I tie the many per-tablespace images together
with a common backup keyword.  the class is available to receive bacups 7x24
so there's no backup window to close.  If a image is bad from some reason,
the error code from bpbackup can be captured but still the script can put
the tablespace back in run mode.

A little P-Code looks like this:

  all_tablespaces="SQL Query tablespace names
  for tablespace in all_tablespaces
  do
    filelist="SQL query files in tablespace"
    bpbackup -class <blah> -schedule <sched> -f filelist
  done
  Switch log files
  Backup control file to trace
  send trace file with bpbackup

You can do this with a multistream behavior by wrapping a generic
multi-process wrapper around it.

  #!/bin/ksh
  work_proc () }{
     #function for whatever you want to run as multiples
  }

  # Count of simultaneous streams
  MULTICOUNT=3

  for param in <some list>
  do
    #call function in background
    work_proc param &

    while [ `jobs | wc -l` -ge $MULTICOUNT` ]
    do
      #throttle loop to keep from having to many jobs at once
      sleep 60
    done
  done
  while [ `jobs | wc -l` -gt 0 ]
  do
    #holding loop to keep script from ending before all jobs completed
    sleep 60
  done
  exit

Hope something here is useful - Mark


-----Original Message-----
From: Weber, Philip [mailto:Philip.Weber AT egg DOT com]
Sent: Monday, September 09, 2002 9:44 AM
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: [Veritas-bu] bpstart_notify & bpend_notify for Oracle


Hi all,

This topic has been discussed before, so I hope this is not a repeat of a
question that has already been resolved.

We use bpstart_notify and bpend_notify scripts to switch Oracle into hot
backup mode (or shut it down, depending on server/database) at the start of
the backup and switch it out of hot backup mode (or start it up) at the end
of the backup.  As the backups are multi-stream, the logic surrounding this
is a bit tricky.  We use a set of lock files created by the first stream and
removed by each stream as it completes, to ensure the processing occurs at
the correct time in the backup.

So far so good, but if for some reason (like some streams failing due to
lack of tapes or having passed their scheduled window) not all streams run,
the databases will be left in hot mode (messy) or down (embarrassing).

While I try to get my head around what to do next, has anyone out there any
helpful suggestions?  I think the only way to be sure of what is happening
is for the client scripts to remote call back to the master to query the
jobs database.  All of this is on Solaris with NBU 3.4.

thanks, Phil

Phil Weber
IT Infrastructure Unix Systems Engineer

Phone: 01384 26 4136
Mobile: n/a


This private and confidential e-mail has been sent to you by Egg.
The Egg group of companies includes Egg Banking plc
(registered no. 2999842), Egg Financial Products Ltd (registered
no. 3319027) and Egg Investments Ltd (registered no. 3403963) which
carries out investment business on behalf of Egg and is regulated
by the Financial Services Authority.  
Registered in England and Wales. Registered offices: 1 Waterhouse Square,
138-142 Holborn, London EC1N 2NA.
If you are not the intended recipient of this e-mail and have
received it in error, please notify the sender by replying with
'received in error' as the subject and then delete it from your
mailbox.

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

------_=_NextPart_001_01C2581F.7201DB70
Content-Type: text/html;
        charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Diso-8859-1">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
5.5.2653.12">
<TITLE>RE: [Veritas-bu] bpstart_notify &amp; bpend_notify for =
Oracle</TITLE>
</HEAD>
<BODY>

<P><FONT SIZE=3D2>We do a similar thing - and yes the scripts involved =
were spawned in some evil space in my core.</FONT>
</P>

<P><FONT SIZE=3D2>We have the advantage of having BCV's for some of our =
databases.&nbsp; I place the entire database in hot-backpup mode at =
once, split the BCV mirrors, then restore the DB back to run-mode. I =
then mount and backup the BCV's on another server.&nbsp; The DB spends =
maybe 5 minutes in backup mode keeping us from generating excess =
numbers of redo logs.&nbsp; You could do the same thing with filesystem =
snapshots or maybe split mirror plexes.</FONT></P>

<P><FONT SIZE=3D2>For hot-backup on servers that are not on BCV, I =
drive it from a cron on the DB server.&nbsp; We have a script that =
queries for all tablespaces in the DB.&nbsp; The tablespaces are placed =
in hotbackup mode in sequence, and the datafiles for that tablespace =
are sent.&nbsp; I tie the many per-tablespace images together with a =
common backup keyword.&nbsp; the class is available to receive bacups =
7x24 so there's no backup window to close.&nbsp; If a image is bad from =
some reason, the error code from bpbackup can be captured but still the =
script can put the tablespace back in run mode.</FONT></P>

<P><FONT SIZE=3D2>A little P-Code looks like this:</FONT>
</P>

<P><FONT SIZE=3D2>&nbsp; all_tablespaces=3D&quot;SQL Query tablespace =
names</FONT>
<BR><FONT SIZE=3D2>&nbsp; for tablespace in all_tablespaces</FONT>
<BR><FONT SIZE=3D2>&nbsp; do</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp; filelist=3D&quot;SQL query files =
in tablespace&quot;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp; bpbackup -class &lt;blah&gt; =
-schedule &lt;sched&gt; -f filelist</FONT>
<BR><FONT SIZE=3D2>&nbsp; done</FONT>
<BR><FONT SIZE=3D2>&nbsp; Switch log files</FONT>
<BR><FONT SIZE=3D2>&nbsp; Backup control file to trace</FONT>
<BR><FONT SIZE=3D2>&nbsp; send trace file with bpbackup</FONT>
</P>

<P><FONT SIZE=3D2>You can do this with a multistream behavior by =
wrapping a generic multi-process wrapper around it.</FONT>
</P>

<P><FONT SIZE=3D2>&nbsp; #!/bin/ksh</FONT>
<BR><FONT SIZE=3D2>&nbsp; work_proc () }{</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp; #function for whatever you =
want to run as multiples</FONT>
<BR><FONT SIZE=3D2>&nbsp; }</FONT>
</P>

<P><FONT SIZE=3D2>&nbsp; # Count of simultaneous streams</FONT>
<BR><FONT SIZE=3D2>&nbsp; MULTICOUNT=3D3</FONT>
</P>

<P><FONT SIZE=3D2>&nbsp; for param in &lt;some list&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp; do</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp; #call function in =
background</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp; work_proc param &amp;</FONT>
</P>

<P><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp; while [ `jobs | wc -l` -ge =
$MULTICOUNT` ]</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp; do</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; #throttle loop to =
keep from having to many jobs at once</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sleep 60</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp; done</FONT>
<BR><FONT SIZE=3D2>&nbsp; done</FONT>
<BR><FONT SIZE=3D2>&nbsp; while [ `jobs | wc -l` -gt 0 ]</FONT>
<BR><FONT SIZE=3D2>&nbsp; do</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp; #holding loop to keep script from =
ending before all jobs completed</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp;&nbsp; sleep 60</FONT>
<BR><FONT SIZE=3D2>&nbsp; done</FONT>
<BR><FONT SIZE=3D2>&nbsp; exit</FONT>
</P>

<P><FONT SIZE=3D2>Hope something here is useful - Mark</FONT>
</P>
<BR>

<P><FONT SIZE=3D2>-----Original Message-----</FONT>
<BR><FONT SIZE=3D2>From: Weber, Philip [<A =
HREF=3D"mailto:Philip.Weber AT egg DOT com">mailto:Philip.Weber AT egg DOT 
com</A>]</F=
ONT>
<BR><FONT SIZE=3D2>Sent: Monday, September 09, 2002 9:44 AM</FONT>
<BR><FONT SIZE=3D2>To: veritas-bu AT mailman.eng.auburn DOT edu</FONT>
<BR><FONT SIZE=3D2>Subject: [Veritas-bu] bpstart_notify &amp; =
bpend_notify for Oracle</FONT>
</P>
<BR>

<P><FONT SIZE=3D2>Hi all,</FONT>
</P>

<P><FONT SIZE=3D2>This topic has been discussed before, so I hope this =
is not a repeat of a</FONT>
<BR><FONT SIZE=3D2>question that has already been resolved.</FONT>
</P>

<P><FONT SIZE=3D2>We use bpstart_notify and bpend_notify scripts to =
switch Oracle into hot</FONT>
<BR><FONT SIZE=3D2>backup mode (or shut it down, depending on =
server/database) at the start of</FONT>
<BR><FONT SIZE=3D2>the backup and switch it out of hot backup mode (or =
start it up) at the end</FONT>
<BR><FONT SIZE=3D2>of the backup.&nbsp; As the backups are =
multi-stream, the logic surrounding this</FONT>
<BR><FONT SIZE=3D2>is a bit tricky.&nbsp; We use a set of lock files =
created by the first stream and</FONT>
<BR><FONT SIZE=3D2>removed by each stream as it completes, to ensure =
the processing occurs at</FONT>
<BR><FONT SIZE=3D2>the correct time in the backup.</FONT>
</P>

<P><FONT SIZE=3D2>So far so good, but if for some reason (like some =
streams failing due to</FONT>
<BR><FONT SIZE=3D2>lack of tapes or having passed their scheduled =
window) not all streams run,</FONT>
<BR><FONT SIZE=3D2>the databases will be left in hot mode (messy) or =
down (embarrassing).</FONT>
</P>

<P><FONT SIZE=3D2>While I try to get my head around what to do next, =
has anyone out there any</FONT>
<BR><FONT SIZE=3D2>helpful suggestions?&nbsp; I think the only way to =
be sure of what is happening</FONT>
<BR><FONT SIZE=3D2>is for the client scripts to remote call back to the =
master to query the</FONT>
<BR><FONT SIZE=3D2>jobs database.&nbsp; All of this is on Solaris with =
NBU 3.4.</FONT>
</P>

<P><FONT SIZE=3D2>thanks, Phil</FONT>
</P>

<P><FONT SIZE=3D2>Phil Weber</FONT>
<BR><FONT SIZE=3D2>IT Infrastructure Unix Systems Engineer</FONT>
</P>

<P><FONT SIZE=3D2>Phone: 01384 26 4136</FONT>
<BR><FONT SIZE=3D2>Mobile: n/a</FONT>
</P>
<BR>

<P><FONT SIZE=3D2>This private and confidential e-mail has been sent to =
you by Egg.</FONT>
<BR><FONT SIZE=3D2>The Egg group of companies includes Egg Banking =
plc</FONT>
<BR><FONT SIZE=3D2>(registered no. 2999842), Egg Financial Products Ltd =
(registered</FONT>
<BR><FONT SIZE=3D2>no. 3319027) and Egg Investments Ltd (registered no. =
3403963) which</FONT>
<BR><FONT SIZE=3D2>carries out investment business on behalf of Egg and =
is regulated</FONT>
<BR><FONT SIZE=3D2>by the Financial Services Authority.&nbsp; </FONT>
<BR><FONT SIZE=3D2>Registered in England and Wales. Registered offices: =
1 Waterhouse Square,</FONT>
<BR><FONT SIZE=3D2>138-142 Holborn, London EC1N 2NA.</FONT>
<BR><FONT SIZE=3D2>If you are not the intended recipient of this e-mail =
and have</FONT>
<BR><FONT SIZE=3D2>received it in error, please notify the sender by =
replying with</FONT>
<BR><FONT SIZE=3D2>'received in error' as the subject and then delete =
it from your</FONT>
<BR><FONT SIZE=3D2>mailbox.</FONT>
</P>

<P><FONT =
SIZE=3D2>_______________________________________________</FONT>
<BR><FONT SIZE=3D2>Veritas-bu maillist&nbsp; -&nbsp; =
Veritas-bu AT mailman.eng.auburn DOT edu</FONT>
<BR><FONT SIZE=3D2><A =
HREF=3D"http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu"; =
TARGET=3D"_blank">http://mailman.eng.auburn.edu/mailman/listinfo/veritas=
-bu</A></FONT>
</P>

</BODY>
</HTML>
------_=_NextPart_001_01C2581F.7201DB70--

<Prev in Thread] Current Thread [Next in Thread>