Veritas-bu

[Veritas-bu] Recovery of Production data onto Test hardware

2002-04-24 16:08:17
Subject: [Veritas-bu] Recovery of Production data onto Test hardware
From: Mark.Donaldson AT experianems DOT com (Donaldson, Mark)
Date: Wed, 24 Apr 2002 14:08:17 -0600
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C1EBCB.C629A130
Content-Type: text/plain;
        charset="iso-8859-1"

Yeah, the cron thing would be just to make the DBA's happy.  Surely they
have something there to manage log files or something.  What's there for
test instances might be sufficient.

As far as splitting into multiple classes for backup, you can do it but it's
not really what I intended.  The comment was meant mostly about restorals.
The concept is to restore only the instance-specific stuff - have the OS and
application be used from what's already on the test system.  If you backed
it up like this using multiple classes then it's easier, though, just
restore a class as a whole and forget having to come up with a filelist
during the panic of failing-over.

If you're doing cold backups, then you won't have to worry about the redo
logs to get the oracle database simply "up" and recovered.  You might still
want to restore the logs to get the DB to a "more recent" state, though.
Are you incrementally backing up the redo logs every several hours? If not,
you should think about it.

If you have a SAN, then you can't beat being able to move datafiles via
volume group deport/import.  I have five oracle instances I can assign to
any one of three servers just by moving the disks around.  There's some
caveats & "be carefuls" but it works pretty well.  If I lose a server, I
just pop the disks off and apply them to a working server.  Makes me look
like a hero - I can initiate a complete DB shutdown, move, and restart on
another server in about 5 min.  

Of course, you have to have highly-redundant disk arrays to make this
nice-and-safe and I still do nightly backups of the DB ("Trust in God but
tie up your camel at night").  This has the added benefit that when your
primary server is fixed, the changes applied to the data while on the
failover server come back to the primary server as part of the disk move.

You don't need a SAN, by the way.  I do this without any fiber.  My arrays
have multiple connectors and I do it all with SCSI.  SAN just makes it
easier and more flexible.

Veritas bought Bare Metal Restore from the Kernel group and is now marketing
it as their product.  They're doing a series of dog-and-pony shows around
the country.  You might contact your local rep and see if there's one in
your neighborhood.

-Mark


-----Original Message-----
From: William Enestvedt [mailto:Will.Enestvedt AT jwu DOT edu]
Sent: Wednesday, April 24, 2002 1:47 PM
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: RE: [Veritas-bu] Recovery of Production data onto Test hardware


Mark Donaldson wrote:
>
> Things to do ahead of time: 
> 1. Install all your oracle binaries on Test server...
> 2. Sync the OS versions on test & prod 
>
   Done.
>
> Things you're going to restore from prod to test: 
> 1. Data files & control files (OFA structure): 
> /uXX/oradata/<instance>/*.dbf, /u01/app/oracle/admin/<instance>
>
   Done.
>
> 2. Enough archived redo to roll the instance to current 
> time (or nearly so) 
>
   We do cold backups of the whole server, so I'm guessing (without asking
the dba, who's in a meeting) that these are on one of the RAID volumes.
>
> 3. /var/spool/cron/crontabs/oracle 
>
   What would I have here if I had something here? Or does this depend on
the dba's whims? :7)
>
> If your production system is on the same subnet as your 
> test system...
>
   And it is.
>
> ...you should be able to bring the production IP address up 
> on the same interface as your test address as an alias.
>
   That's a cool thing that I honestly hadn't thought of. Frankly, if we
lose the Production database, I really won't be in a mood to accommodate the
developers -- and they can use the Development servers (which, in our
set-up, are the systems they ought to be on anyway!).
>
> Now you don't have to make tnsnames changes to the clients.
>
   A bonus, as they'll have been disrupted enough. :7)
>
> Don't try to recover the OS from production - you'll 
> struggle getting it right.
>
   Agreed. (Which reminds me: anyone heard anything about availability of
The Kernel's Group's "Bare Metal" product and NBU?)
>
> Make a clean distinction between, OS, application, and data 
> and you can treat this like a clustered or HA system.
>
   What do you mean -- should I be making this distinction when I'm planning
my backups?
>
> If you can get the data on some sort of shared storage 
> system, then you can move the data using Veritas volume 
> groups...
>
   Hmmm...we were supposed to meet with some Compaq guys about getting the
Suns on our SAN but they had "car trouble" and we all left. I'll definitely
bring this up when we re-schedule!
>
> Drop me a line if you have questions.
>
   That's very generous -- thanks!
>
> This note is deliberately short...
>
   And yet very helpful. :7)
-wde
--
Will Enestvedt
UNIX System Administrator
Johnson & Wales University -- Providence, RI

------_=_NextPart_001_01C1EBCB.C629A130
Content-Type: text/html;
        charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Diso-8859-1">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
5.5.2653.12">
<TITLE>RE: [Veritas-bu] Recovery of Production data onto Test =
hardware</TITLE>
</HEAD>
<BODY>

<P><FONT SIZE=3D2>Yeah, the cron thing would be just to make the DBA's =
happy.&nbsp; Surely they have something there to manage log files or =
something.&nbsp; What's there for test instances might be =
sufficient.</FONT></P>

<P><FONT SIZE=3D2>As far as splitting into multiple classes for backup, =
you can do it but it's not really what I intended.&nbsp; The comment =
was meant mostly about restorals.&nbsp; The concept is to restore only =
the instance-specific stuff - have the OS and application be used from =
what's already on the test system.&nbsp; If you backed it up like this =
using multiple classes then it's easier, though, just restore a class =
as a whole and forget having to come up with a filelist during the =
panic of failing-over.</FONT></P>

<P><FONT SIZE=3D2>If you're doing cold backups, then you won't have to =
worry about the redo logs to get the oracle database simply =
&quot;up&quot; and recovered.&nbsp; You might still want to restore the =
logs to get the DB to a &quot;more recent&quot; state, though.&nbsp; =
Are you incrementally backing up the redo logs every several hours? If =
not, you should think about it.</FONT></P>

<P><FONT SIZE=3D2>If you have a SAN, then you can't beat being able to =
move datafiles via volume group deport/import.&nbsp; I have five oracle =
instances I can assign to any one of three servers just by moving the =
disks around.&nbsp; There's some caveats &amp; &quot;be carefuls&quot; =
but it works pretty well.&nbsp; If I lose a server, I just pop the =
disks off and apply them to a working server.&nbsp; Makes me look like =
a hero - I can initiate a complete DB shutdown, move, and restart on =
another server in about 5 min.&nbsp; </FONT></P>

<P><FONT SIZE=3D2>Of course, you have to have highly-redundant disk =
arrays to make this nice-and-safe and I still do nightly backups of the =
DB (&quot;Trust in God but tie up your camel at night&quot;).&nbsp; =
This has the added benefit that when your primary server is fixed, the =
changes applied to the data while on the failover server come back to =
the primary server as part of the disk move.</FONT></P>

<P><FONT SIZE=3D2>You don't need a SAN, by the way.&nbsp; I do this =
without any fiber.&nbsp; My arrays have multiple connectors and I do it =
all with SCSI.&nbsp; SAN just makes it easier and more =
flexible.</FONT></P>

<P><FONT SIZE=3D2>Veritas bought Bare Metal Restore from the Kernel =
group and is now marketing it as their product.&nbsp; They're doing a =
series of dog-and-pony shows around the country.&nbsp; You might =
contact your local rep and see if there's one in your =
neighborhood.</FONT></P>

<P><FONT SIZE=3D2>-Mark</FONT>
</P>
<BR>

<P><FONT SIZE=3D2>-----Original Message-----</FONT>
<BR><FONT SIZE=3D2>From: William Enestvedt [<A =
HREF=3D"mailto:Will.Enestvedt AT jwu DOT edu">mailto:Will.Enestvedt AT jwu DOT 
edu</A>=
]</FONT>
<BR><FONT SIZE=3D2>Sent: Wednesday, April 24, 2002 1:47 PM</FONT>
<BR><FONT SIZE=3D2>To: veritas-bu AT mailman.eng.auburn DOT edu</FONT>
<BR><FONT SIZE=3D2>Subject: RE: [Veritas-bu] Recovery of Production =
data onto Test hardware</FONT>
</P>
<BR>

<P><FONT SIZE=3D2>Mark Donaldson wrote:</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; Things to do ahead of time: </FONT>
<BR><FONT SIZE=3D2>&gt; 1. Install all your oracle binaries on Test =
server...</FONT>
<BR><FONT SIZE=3D2>&gt; 2. Sync the OS versions on test &amp; prod =
</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp; Done.</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; Things you're going to restore from prod to =
test: </FONT>
<BR><FONT SIZE=3D2>&gt; 1. Data files &amp; control files (OFA =
structure): </FONT>
<BR><FONT SIZE=3D2>&gt; /uXX/oradata/&lt;instance&gt;/*.dbf, =
/u01/app/oracle/admin/&lt;instance&gt;</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp; Done.</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; 2. Enough archived redo to roll the instance to =
current </FONT>
<BR><FONT SIZE=3D2>&gt; time (or nearly so) </FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp; We do cold backups of the whole server, =
so I'm guessing (without asking</FONT>
<BR><FONT SIZE=3D2>the dba, who's in a meeting) that these are on one =
of the RAID volumes.</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; 3. /var/spool/cron/crontabs/oracle </FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp; What would I have here if I had =
something here? Or does this depend on</FONT>
<BR><FONT SIZE=3D2>the dba's whims? :7)</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; If your production system is on the same subnet =
as your </FONT>
<BR><FONT SIZE=3D2>&gt; test system...</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp; And it is.</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; ...you should be able to bring the production =
IP address up </FONT>
<BR><FONT SIZE=3D2>&gt; on the same interface as your test address as =
an alias.</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp; That's a cool thing that I honestly =
hadn't thought of. Frankly, if we</FONT>
<BR><FONT SIZE=3D2>lose the Production database, I really won't be in a =
mood to accommodate the</FONT>
<BR><FONT SIZE=3D2>developers -- and they can use the Development =
servers (which, in our</FONT>
<BR><FONT SIZE=3D2>set-up, are the systems they ought to be on =
anyway!).</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; Now you don't have to make tnsnames changes to =
the clients.</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp; A bonus, as they'll have been disrupted =
enough. :7)</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; Don't try to recover the OS from production - =
you'll </FONT>
<BR><FONT SIZE=3D2>&gt; struggle getting it right.</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp; Agreed. (Which reminds me: anyone heard =
anything about availability of</FONT>
<BR><FONT SIZE=3D2>The Kernel's Group's &quot;Bare Metal&quot; product =
and NBU?)</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; Make a clean distinction between, OS, =
application, and data </FONT>
<BR><FONT SIZE=3D2>&gt; and you can treat this like a clustered or HA =
system.</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp; What do you mean -- should I be making =
this distinction when I'm planning</FONT>
<BR><FONT SIZE=3D2>my backups?</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; If you can get the data on some sort of shared =
storage </FONT>
<BR><FONT SIZE=3D2>&gt; system, then you can move the data using =
Veritas volume </FONT>
<BR><FONT SIZE=3D2>&gt; groups...</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp; Hmmm...we were supposed to meet with =
some Compaq guys about getting the</FONT>
<BR><FONT SIZE=3D2>Suns on our SAN but they had &quot;car trouble&quot; =
and we all left. I'll definitely</FONT>
<BR><FONT SIZE=3D2>bring this up when we re-schedule!</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; Drop me a line if you have questions.</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp; That's very generous -- thanks!</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; This note is deliberately short...</FONT>
<BR><FONT SIZE=3D2>&gt;</FONT>
<BR><FONT SIZE=3D2>&nbsp;&nbsp; And yet very helpful. :7)</FONT>
<BR><FONT SIZE=3D2>-wde</FONT>
<BR><FONT SIZE=3D2>--</FONT>
<BR><FONT SIZE=3D2>Will Enestvedt</FONT>
<BR><FONT SIZE=3D2>UNIX System Administrator</FONT>
<BR><FONT SIZE=3D2>Johnson &amp; Wales University -- Providence, =
RI</FONT>
</P>

</BODY>
</HTML>
------_=_NextPart_001_01C1EBCB.C629A130--

<Prev in Thread] Current Thread [Next in Thread>