Veritas-bu

[Veritas-bu] Backup jobs not kicking off any more....

2007-03-13 15:55:55
Subject: [Veritas-bu] Backup jobs not kicking off any more....
From: Len.Boyle at sas.com (Len Boyle)
Date: Tue, 13 Mar 2007 15:55:55 -0400
Chris, 
 
Did the support folks look at the media end of things? the resource broker 
could be the problem. That is the piece that sits between the scheduler and the 
media you are going to write on. If that get's messed up netbackup may believe 
that it never has anything to write to. 
 
I do not remember the command but there is an EMM command to suspend backups 
(like the old bpconfig -tries 0). Is it possible that this command was issued?
 
len
 
 

________________________________

From: Christopher Jay Manders [mailto:cjmanders at lbl.gov] 
Sent: Tuesday, March 13, 2007 3:32 PM
To: veritas-bu at mailman.eng.auburn.edu
Cc: Preston, Douglas L; Len Boyle
Subject: Re: [Veritas-bu] Backup jobs not kicking off any more....


Hi,

Thanks for the responses so far. Doug, yours comes close to what support has 
suggested, and we have tried it. More on that below... Still no go... :(

A couple of new facts:
1 - We are on Solaris, but it is similar enuf for the below.
2 - We are at 6.0 MP4
3 - We did the following steps, which add to the step you mention below:

 Stop NetBackup by exiting any open NetBackup consoles and issuing the 
following command:
# /usr/openv/netbackup/bin/goodies/netbackup stop
- After NetBackup has stopped, run the following command to check for 
hung NetBackup processes:
# usr/openv/netbackup/bin/bpps -a
- Using kill -9 command, stop any remaining hung processes that are 
listed by the bpps command.
- Verify that all Netbackup processes have been properly terminated.
- Remove the pempersist file: (Note: I would recommend just renaming the 
original file, or make a copy of this file before proceeding)
The pempersist file location is as follows:
/usr/openv/netbackup/bin/bpsched.d/
- Remove the retirepersist file: (Note: I would recommend just renaming 
the original file, or make a copy of this file before proceeding)
The retirepersist file location is as follows:
/usr/openv/netbackup/bin/bpsched.d/
- Remove the bpjobd.act.db file: (Note: I would recommend just renaming 
the original file, or make a copy of this file before proceeding)
The bpjobd.act.db file location is as follows:
/usr/openv/netbackup/db/jobs/bpjobd.act.db
- Start NetBackup again:
# /usr/openv/netbackup/bin/goodies/netbackup start

So far nothing seems to be working at all. Still no jobs get Queued or Active. 
Nothing at all appears in the Activity Monitor.

Any other ideas?

I am almost ready to start pulling my hair out.

Tx!!!

--Chris



        Under your C:\Program Files\VERITAS\NetBackup\db\jobs folder there is a
        file called permpersist stop all your services and delete this file and
        restart your services.  This should fix the problem
        
        
        
        Doug Preston
        Systems Engineer
        Land America Tax and Flood Services
        Phone 626-339-5221 Ext 104
        Email  dlpreston at landam.com
        
        
        ------------------------------------------------------------------------
        ------------
        NOTICE: This electronic mail transmission may constitute a communication
        that is legally privileged. It is not intended for transmission to, or
        receipt by, any unauthorized persons. If you have received this
        electronic mail transmission in error, please delete it from your system
        without copying it, and notify the sender by reply e-mail, so that our
        address record can be corrected.
        ------------------------------------------------------------------------
        ------------
        
        
        -----Original Message-----
        From: veritas-bu-bounces at mailman.eng.auburn.edu
        [mailto:veritas-bu-bounces at mailman.eng.auburn.edu] On Behalf Of
        Christopher Jay Manders
        Sent: Tuesday, March 13, 2007 10:05 AM
        To: veritas-bu at mailman.eng.auburn.edu
        Subject: [Veritas-bu] Backup jobs not kicking off any more....
        
        Hi,
        
        So, we have an 6.0 environment that has three times now over the course
        of the last 2 months stopped accepting or kicking jobs off. The solution
        we found previously, before this last Sunday the 11th, was to reboot the
        master server.
        
        Unfortunately, now rebooting does not seem to help.
        bpbackup -i -p <policy> does nothing.
        No jobid is assigned and no jobs are showing up in the Activity Monitor.
        In fact, bperror -U -backstat -by_statcode -hoursago 48 shows that no
        jobs have gone off for the last 2 days.
        
        We are at 6.0 MP4 on all of our systems, and the DST patches were
        applied awhile back. All system time is correct.
        
        Since this issue has happened before the DST thing, I am thinking this
        is something else altogether, but no logs show anything of any use that
        I can see, and the front-line support folks have not had any clues at
        all.
        
        I see from the Reports->Problems that the consistent error appears to be
        connected to the issue:
        
        get_string() failed - premature end of file encountered (5) could not
        process request
        
        The error is repeated for each job that would normally fire off. So, the
        logs are filled with thousands of the repeated error message.
        
        Anyone else seen anything like this or have any ideas?
        
        TIA!
        
        --Chris
        _______________________________________________
        Veritas-bu maillist  -  Veritas-bu at mailman.eng.auburn.edu
        http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
        
        _______________________________________________
        Veritas-bu maillist  -  Veritas-bu at mailman.eng.auburn.edu
        http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
          


-------------- next part --------------
An HTML attachment was scrubbed...
URL: 
http://mailman.eng.auburn.edu/pipermail/veritas-bu/attachments/20070313/4bda3a1a/attachment.html