Re: [Networker] nsrjobd fubar in 7.3.4 and 7.4 SP2?

I just started to see hanged sessions in the group's 'show details'window after the upgrade to 7.3.4. It seems that I was getting thesemessages in daemon.log:


nsrjobd: jobsdb size at 22938674 exceeded high size watermark.

  And then, many of these:

nsrjobd: Jobs error: Unable to find record for job 236744 during anattempt to send message to it

After discussing this with my support engineer, he found esg93651 (towhich I have no access via powerlink) which suggests raising the jobsdbsize till there are no watermark messages (the default is 20Mb). Isuspect (it is hinted by the esg) that the process that trims the jobsdbdoes a poor job and therefore it is better to let records expire bytime. Come to think of it, it seems like a poor design decision to havethis limit to begin with, as most people can allocate even a 1Gb for thejobsdb and avoid the hanged save problem.


Oscar Olsson wrote:

On 2008-05-27 12:21, Peter Viertel revealed:
PV> I messed it up back when I had 733 by fiddling with the setting forPV> maximum jobsdb size. I'd added a zero to the end thinking thatPV> allowing it to be bigger would mean less issues with its GC routinesPV> but Emc told us to put it back to the default and it seemed to workPV> since then.
The settings we have changed, per EMC support recommendation is to lowerthe data retention in the jobsdb to three days, and increase the size to100MB. This has had no effect, at least not a positive one. :)
Another thing we have changed, also per their recommendation is toincrease the number of TCP connections that can be opened or be half-openper second. I also belive that has no effect, especially considering thatwe still see the same problems. :P
PV> Have you tried moving the whole jobsdb directory aside and restarting 
networker?
Several times, it works OK for a day or two, but then the messages startappearing in the logs indicating that stuff can't talk to it, somesavesets get aborted due to inactivity, nsrjobd takes lots of CPU andmemory etc etc, until nothing works. That process takes about a week tops.
PV> I share your pain with emc support.
Yes. Nothing has really changed during the last years when it comes totheir ability to identify and solve software bugs. Although, I am gettingthe feeling that the industry as a whole is closing in to the EMCnetworker level of support (sadly enough).
//Oscar

To sign off this list, send email to listserv AT listserv.temple DOT edu and type 
"signoff networker" in the body of the email. Please write to networker-request 
AT listserv.temple DOT edu if you have any problems with this list. You can access the 
archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER



--

-- Yaron.

To sign off this list, send email to listserv AT listserv.temple DOT edu and type 
"signoff networker" in the body of the email. Please write to networker-request 
AT listserv.temple DOT edu if you have any problems with this list. You can access the 
archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER