Veritas-bu

Re: [Veritas-bu] [ORACLE] Policy Naming ... Best Practices ?

2007-06-14 16:23:37
Subject: Re: [Veritas-bu] [ORACLE] Policy Naming ... Best Practices ?
From: "Jeff Lightner" <jlightner AT water DOT com>
To: "Steven L. Sesar" <ssesar AT mitre DOT org>
Date: Thu, 14 Jun 2007 16:03:35 -0400
You're incorrect.  NBU only reports what it knows.  If you kick off
mssql backup it will tell you what it did when it got that far but not
what happened afterwards.   Similarly - NetBackup DOES show you what
happened to the tape jobs it was told to kick off by RMAN - it simply
can't tell you end to end.   The job started BEFORE it got to NBU and
ended AFTER it left NBU.  There has been more than one occasion where
I've seen RMAN fail with a status 6 even though the NBU portion of the
job ran fine.   It always had to do with configuration errors in RMAN.
My analogy was about this.  

Certainly if BillyBob in fact gives me the $10,000 NOT in a sealed
envelope I can report that as a "fact".  However, I can not prove a
negative (i.e. that he DIDN'T give me the money.  I wouldn't even know
to state that he didn't unless someone asked me (or looked at my
"activity monitor" to see there was no information regarding it at all).
Restated "information" can help "prove" a point but absence of
information can not "prove" the opposite point.  The best one can say is
"I have no information about this."

I just went this same scenario regarding an Oracle stored procedure and
a script that were supposed to be sending email via the system's
sendmail.  I'm responsible for the sendmail piece (and the next server
which is another sendmail relay) but not for the script or the stored
procedure or Oracle.  User's weren't getting emails.   The best I could
do is show that sendmail never got them which to me implied they were
never sent to sendmail.   However, it might have been that sendmail was
rejecting them without logging them for some reason.   I told them that
my strong suspicion owing to the lack of any mention in sendmail log
regarding this was that they were never being sent to sendmail in the
first place.  On pushing back and asking them to show me a log on their
end that indicated it was going to sendmail they couldn't do it.  What
they DID finally find was that there is a known issue with Oracle_SMTP
whereby it will only do 15 connections and then error out.  It had been
doing this but they hadn't noted it until I did the push back.  Once
they found the known issue and the recommended work around the emails
started showing up in sendmail logs because they finally got there.

All of this means that it is certain there IS positive information you
can gain from NetBackup when things work the way they should but it will
still only be NBU's portion of the job and not RMAN's.    

So to reiterate - your end to end solution for the Oracle backup is
actually RMAN and NOT NetBackup and that is why you have to rely on RMAN
(an Oracle Product) to tell you what IT did including what status it got
from NetBackup when NBU did its PORTION of the job.

You however, are entitled to your opinion no matter how wrong it is.  :p

-----Original Message-----
From: Steven L. Sesar [mailto:ssesar AT mitre DOT org] 
Sent: Thursday, June 14, 2007 3:12 PM
To: Jeff Lightner
Cc: Martin, Jonathan; Hudson, Steve; Wilkinson, Alex;
veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] [ORACLE] Policy Naming ... Best Practices ?

Our stakeholders don't care about which component of a backup failed - 
they (at least around here) care about failures and resolving any issues

around failures. It would be nice if libobk would have hooks into RMAN 
which would perform exception handling.

The problem that I have with your analogy, is that NBU accurately 
reports and records job status for everyone, but "BillyBob Oracle". 
Therefore, it would be reasonable to expect that job status would be 
returned correctly for "BillyBob".

The way that it is now, an Oracle backup kicked off via the NBU 
scheduler, can fail, while returning a status 0. It's bad enough that 
NBU doesn't have an alerting mechanism, but furthermore, NBU would never

report the job as having failed, nor could an operator watching the job 
summary window in the GUI.

As an end user, I expect a backup product to accurately return the exit 
status of a backup job. I don't think that's too much to ask for. So, it

requires a bit more coding to interface with RMAN. Easily accomplished -

we wrote one.

Just my $.02



Jeff Lightner wrote:
> NBU isn't terrible at determining RMAN success/failure - it simply
> doesn't do it.  RMAN by its nature is client initiated.  NBU cares
about
> the actual transfer to storage unit.  RMAN cares about the end to end
> process so of course should be the tool that determines the
> success/failure.   More than once I've seen RMAN fail before it ever
got
> to NetBackup.
>
> It would be much like telling me I have to deposit all the money I
> receive form cashiers then asking me where the $10,000 that BillyBob
> absconded with instead of turning it into me went.  I'd have no clue
> because BillyBob didn't do his part.  The best I could do is say "I
> never got anything from BillyBob."  At that point you would want to
see
> me arrested rather than tracking down BillyBob simply because I'm in
> easy reach.  You'd still be out the money but would feel happy because
> you'd found a scapegoat  :p 
>
>
>
>
> -----Original Message-----
> From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
> [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of
Martin,
> Jonathan
> Sent: Thursday, June 14, 2007 1:34 PM
> To: Hudson, Steve; Wilkinson, Alex; veritas-bu AT mailman.eng.auburn DOT edu
> Subject: Re: [Veritas-bu] [ORACLE] Policy Naming ... Best Practices ?
>
>
> I guess you can do it anyway that floats your boat.  I personally try
to
> limit the number of policies.
>
> We have policies grouping application and server.
>
> We use the following parameters.
>
> 1 - script passes the schedule name  (Weekly-Full) to .cmd or .sh
> 2 - RMAN script calls the $schedule variable with an underscore
> (Weekly_Full_) which is the application backup schedule.
> 3 - We have one script per database in the policy under the selection
> list.  If a database doesn't exist on a server that server still has
the
> script, its just empty (Echo 123 etc..)
>
> With regards to monitoring for success, Netbackup is TERRIBLE at
> handling RMAN successes and failures.  We have scripts monitor the
RMAN
> output logs.
>
> -Jonathan
>
> -----Original Message-----
> From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
> [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of
Hudson,
> Steve
> Sent: Thursday, June 14, 2007 12:54 PM
> To: Wilkinson, Alex; veritas-bu AT mailman.eng.auburn DOT edu
> Subject: Re: [Veritas-bu] [ORACLE] Policy Naming ... Best Practices ?
>
> We create a separate policy for each Rman Database.....
>
> -----Original Message-----
> From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
> [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of
> Wilkinson, Alex
> Sent: Thursday, June 14, 2007 2:56 AM
> To: veritas-bu AT mailman.eng.auburn DOT edu
> Subject: [Veritas-bu] [ORACLE] Policy Naming ... Best Practices ?
>
> Hi all,
>
> Is it best to create a separate policy for each oracle instance ?
>
> Currently I have 4 instances being backed up in a single policy and it
> is impossible to distinguish which instance succeeded and which
failed.
>
> What do other people do ?
>
>  -aW
>
> IMPORTANT: This email remains the property of the Australian Defence
> Organisation and is subject to the jurisdiction of section 70 of the
> CRIMES ACT 1914.  If you have received this email in error, you are
> requested to contact the sender and delete the email.
>
>
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>
> -----------------------------------------
> The information contained in this email message and its attachments is
> intended only for the private and confidential use of the
> recipient(s) named above, unless the sender expressly agrees
otherwise.
> Transmission of email over the Internet is not a secure communications
> medium. If you are requesting or have requested the transmittal of
> personal data, as defined in applicable privacy laws by means of email
> or in an attachment to email, you must select a more secure alternate
> means of transmittal that supports your obligations to protect such
> personal data.
> If the reader of this message is not the intended recipient and/or you
> have received this email in error, you must take no action based on
the
> information in this email and you are hereby notified that any
> dissemination, misuse or copying or disclosure of this communication
is
> strictly prohibited. If you have received this communication in error,
> please notify us immediately by email and delete the original message.
>
>
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>   


-- 
===================================

   Steven L. Sesar
   Lead Operating Systems Programmer/Analyst
   UNIX Application Services R101
   The MITRE Corporation
   202 Burlington Road - MS K101
   Bedford, MA 01730
   tel: (781) 271-7702
   fax: (781) 271-2600
   mobile: (617) 519-8933
   email: ssesar AT mitre DOT org

=================================== 


_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu