Yeah, that's what I've gathered from our systems and those of several friends
elsewhere.
The restarting repeatedly smells like an nbpem bug. Quite plausibly a bug in
the code added along these lines:
http://seer.entsupport.symantec.com/docs/301752.htm , especially given the
proximity to a DST change (although this occured for us well prior to the US
Eastern changover).
A symptom I maybe hadn't mentioned is that one of our environments requeued
jobs even after the policy had been marked inactive and NetBackup restarted, to
which a former co-worker suggested that I inspect the output of nbpemreq
-tables screen (on the premise that the requeueing was stuck in that DB), which
would be a great suggestion if that flag hadn't been removed between 6.5 GA and
6.5.2. I want to think that nbpemreq -persisted is somewhat related, but
significantly less verbose and clear.
I've updated our environments to 6.5.2A and left a non-prod one running for a
while now, and things appear to be okay for now. But, well, "more later". (No,
I haven't opened a case yet, because I don't have enough information yet to be
assertive about the issue. If anyone who's experienced this has, please drop me
a line privately so that we can help Symantas correlate information.)
--
gabriel rosenkoetter
Radian Group Inc, Unix/Linux/VMware Sysadmin / Backup & Recovery
gabriel.rosenkoetter AT radian DOT biz, 215 231 1556
-----Original Message-----
From: Bryan S. Leaman [mailto:leaman AT bitbytes DOT com]
Sent: Sunday, November 02, 2008 6:59 PM
To: Rosenkoetter, Gabriel
Cc: veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Serious bug w/ 6.5.2 (and beyond?): perpetually
requeueing jobs, starting this weekend.
I saw the same behavior with 6.5.2 on Solaris (with the rev4 nbpem EEB
installed). After the DST changeover, several of our schedules also
requeued immediately after completing and ran several times. Luckily they
stopped requeueing later in the morning, possibly because the schedules do
not have a 24hr window. I didn't have to restart NBU, and the future job
forecast looks normal.
I'm not sure about the "fulls instead of incrementals" issue being
related. As I understand that one, it only affects a policy the first
time it runs after 6.5.2 is applied.
Bryan
> We encountered a relatively serious problem with our (HP-UX, NBU 6.5.2)
> environments this weekend.
>
> Scheduled backups ran, as expected, Friday and Saturday evenings. Then,
> after finishing successfully, new, identical jobs requeued and ran again.
> And again. So forth until we shut NBU down completely.
>
> I'm now in the process of installing 6.5.2A (we hadn't bothered because we
> hadn't encountered any known 6.5.2 bugs, except for the initial "full
> instead of incremental" one, and we were past that), starting things back
> up, and hoping that the fix for the "fulls instead of incrementals" bug
> will fix this as well. I'll report that when I know.
>
> We have one site that's still 6.5 GA and it was NOT affected. If you're
> running anything more recent, you SHOULD check your systems right now.
> They're probably still running backups over and over. I say this because I
> just checked with my friends at a former employer, a major financial
> institution whose name I won't mention because I know their policies, and
> they're also affected in environments with Solaris, AIX (maybe?), and
> Windows masters. You are probably also affected. Stop reading this and go
> check. Now.
>
> _______________________________________________
> Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>
_______________________________________________
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
|
