Veritas-bu

Re: [Veritas-bu] "Successful" NDMP Backups Missing Data

2008-03-14 14:45:55
Subject: Re: [Veritas-bu] "Successful" NDMP Backups Missing Data
From: "Koping Wang" <kwang AT esri DOT com>
To: <veritas-bu AT mailman.eng.auburn DOT edu>, <JMARTI05 AT intersil DOT com>
Date: Fri, 14 Mar 2008 11:25:15 -0700
Martin,
1. I don't have experience with NDMP backup, but I know NBU6.0 MP5 is
not a good service pack. Last time NBU could not open compressed catalog
after I patched to MP5, I end up unpatched and move back to MP4.

2. If you want backup through NFS, make sure the NFS connection is on a
separate NIC from your backup NIC, otherwise your backup will be very
very slow.

Thanks
Koping  
 


------------------------------

Message: 4
Date: Fri, 14 Mar 2008 10:11:33 -0400
From: "Martin, Jonathan" <JMARTI05 AT intersil DOT com>
Subject: Re: [Veritas-bu] "Successful" NDMP Backups Missing Data
To: <veritas-bu AT mailman.eng.auburn DOT edu>
Message-ID:
        
<13E204E614D8E04FAF594C9AA9ED0BB709CC1537 AT PBCOMX02.intersil DOT corp>
Content-Type: text/plain; charset="us-ascii"

Thanks everyone for the comments.  Here's some more information:
 
Client: Sun StorageTek 5370 (nbnfs)
Master/Media Server: Windows 2003 STD SP2 running Netbackup Enterprise
6.0 MP5 (nbbk01)
 
nbnfs 12/14/2007 8:00:00 PM PR_NDMP 46865896 772513720 Full-Weekly Full
21:49:02 No
nbnfs 12/21/2007 8:00:00 PM PR_NDMP 47584824 796582733 Full-Weekly Full
21:55:57 No
nbnfs 12/28/2007 8:00:00 PM PR_NDMP 47683917 805472963 Full-Monthly Full
22:59:43 No
nbnfs 1/8/2008 9:06:28 AM PR_NDMP 33581624 536189959 Full-Weekly Full
13:56:53 No
nbnfs 1/11/2008 8:00:00 PM PR_NDMP 48033174 810362561 Full-Weekly Full
22:28:07 No
nbnfs 1/18/2008 8:00:00 PM PR_NDMP 37250565 611787136 Full-Weekly Full
17:26:27 No
nbnfs 1/25/2008 8:00:00 PM PR_NDMP 48885260 833748549 Full-Monthly Full
23:03:54 No
nbnfs 2/1/2008 8:00:00 PM PR_NDMP 49524885 838532189 Full-Weekly Full
23:47:23 No
nbnfs 2/8/2008 8:00:00 PM PR_NDMP 49908417 848834139 Full-Weekly Full
23:07:55 No
nbnfs 2/15/2008 8:00:00 PM PR_NDMP 50243186 855850214 Full-Weekly Full 1
00:01:05 No
nbnfs 2/22/2008 8:00:00 PM PR_NDMP 50511980 863140542 Full-Weekly Full 1
00:46:31 No
nbnfs 3/3/2008 12:10:12 PM PR_NDMP 19450344 316300098 Full-Monthly Full
07:01:54 No
nbnfs 3/7/2008 8:00:00 PM PR_NDMP 47423669 775305192 Full-Weekly Full
20:16:58 No

 
If you notice, all of the red entries are less than they should be.  The
full backup qty gets incrementally bigger each week with the exception
of the "red" failures.  The 316GB backup was a failed robotic controller
so I expected that but the other three backups were status 0 in
Netbackup.  I've got three sites all simmilarly configured and two sites
(this one and another) are showing this same issue.  One intresting
point about this particular host is that the same directories are
missing from all three "successful" failed backups.  We've got a ticket
open with Sun, as Netbackup seems to think its a filer issue.  (We
agree.)  We've had issues in the past with super long directory paths on
this NAS device, maybe this is simmilar.
 
Personally, I'm appauled that there seems to be no better communication
between the filer and Netbackup about the backup status and that NBU
just assumes everything's 100% hunky dory.  We use the activty monitor
and BPDBJOBS in a NUMBER of scripts to monitor completion.  I think its
unreasonable to assume the storage guys need to check their filer logs
every week to make sure their backups are complete.  This may be an
excuse for me to write the script I've been planning that measures job
success on how much data was backed up (compared to the trend.)  For
this weekend we're going to backup the data via NFS mount on another
server.  Stoopid NAS.
 
I love this job... /sigh
 
-Jonathan
 
PS: Should have bought a NetApp =P


________________________________

From: Jeff Lightner [mailto:jlightner AT water DOT com]
Sent: Thursday, March 13, 2008 5:10 PM
To: Martin, Jonathan; veritas-bu AT mailman.eng.auburn DOT edu
Subject: RE: [Veritas-bu] "Successful" NDMP Backups Missing Data



You might want to verify your policy "selections" included everything
and also that you don't have an exclude list specification on the client
that would have made it skip certain things. 

 

It would be very bad indeed if NBU gave a status 0 for backups that
didn't backup exactly what they were told to backup.   My first
suspicion would be that this didn't occur - that either its not in the
selections for the policy or it was excluded due an exclude list.

 

Of course there are other things that could cause this - e.g. do not
cross mount points, don't backup NFS etc... that might have been set on
the policy by default.

 

________________________________

From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Martin,
Jonathan
Sent: Thursday, March 13, 2008 4:11 PM
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: [Veritas-bu] "Successful" NDMP Backups Missing Data

 

I'm looking at an Full NDMP backup from this past weekend.  According to
the activity monitor it was Status - 0 Successful.  However, when
drilling down into the data via Backup and Recover I notice that certain
key folders are missing.  Upon further inspection, I notice the past 4
weeks backups look something like this - 833GB, 833.2GB, 833.7GB, 780GB.
Although Netbackup gives me the "0" I'm obviously missing data.  I just
opened a case with Symantec (they will call me tomorrow), and I'm pretty
sure the data is gone but can anyone give me any hints on where to start
troubleshooting this?  The good news is that it looks like MOST of the
data made it to tape.  The VERY BAD news is that 100% of the data was
not written to tape and that Netbackup did not give me an error.

 

-Jonathan

----------------------------------
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or
confidential information and is for the sole use of the intended
recipient(s). If you are not the intended recipient, any disclosure,
copying, distribution, or use of the contents of this information is
prohibited and may be unlawful. If you have received this electronic
transmission in error, please reply immediately to the sender that you
have received the message in error, and delete it. Thank you.
----------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mailman.eng.auburn.edu/pipermail/veritas-bu/attachments/20080314/
f54396b4/attachment.html

------------------------------

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


End of Veritas-bu Digest, Vol 23, Issue 30
******************************************


_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

<Prev in Thread] Current Thread [Next in Thread>