>
> Thank's
>
>
>
> 2010/1/9 Renaud Marquet <
rmarquet AT gmail DOT com>
>
> > Le samedi 09 janvier 2010 à 21:25 +0100, Kern Sibbald a écrit :
> > > Hello,
> > >
> > > On Saturday 09 January 2010 20:20:01 Renaud Marquet wrote:
> > > > Kern,
> > > >
> > > > altough I searched for a possible workaround, I didn't found the ones
> > > > you talk about. But your statement is not correct as pointing to a
> >
> > valid
> >
> > > > smtp server is not a proper workaround. Actually, if for some reason,
> > > > the *valid* smtp server is down, the problem will occur and I bet
> > > > users will not figure out the reason.
> > >
> > > I never claimed that my suggestion was a "proper" workaround nor that
> > > it
> >
> > was a
> >
> > > fix. It is a workaround.
> >
> > Nevermind then ;)
> >
> > > If you want, you can backport the fixes (applied 23 October 2009), but
> >
> > since
> >
> > > we are close to release, and we have a workaround, we are not planning
> > > to backport them.
> >
> > No need to backport. This is not a 'blocker' problem, I just mailed here
> > in case someone else run into the same problem because there wasn't any
> > answer when googling. Bacula now runs perfectly fine on my system, so I
> > can wait for the upcoming release without any trouble.
> >
> > > > That's why I came up with this patch. It correctly fixes the problem
> >
> > but
> >
> > > > I recognize this could affect performances so it should certainly not
> >
> > be
> >
> > > > put in the trunk. It will even probably be useless as you pointed out
> > > > it's already fixed in developpement version.
> > >
> > > Unfortunately your patch does not fix the problem -- it masks the
> >
> > problem. I
> >
> > > didn't look at your patch in detail, but I believe that it will make
> > > all locks recursive, which is not really what we want and may lead to
> > > some surprises.
> > >
> > > Bacula does have recursive locks, but we use them only in situations
> >
> > where
> >
> > > they need to be used and they are portable. I am not so much worried
> >
> > about
> >
> > > the performance consequences of your patch, but your code is Linux only
> >
> > if I
> >
> > > am not mistaken (i.e. not portable), and as I said, the lock manager is
> >
> > not
> >
> > > production code. It is development should only be turned on for
> >
> > developer's
> >
> > > for debugging.
> >
> > As I said in another mail, I didn't do anything to activate this lock
> > manager, so I guess it's not. I think the confusion come from the fact
> > mutexes are handled through some functions in lockmgr.c (through a
> > macro), I think even with lock manager deactivated.
> >
> > > > That said, I didn't know lock manager should be turned off in
> >
> > production
> >
> > > > environment. Moreover, I'm not sure I understand your point because,
> > > > although I didn't read all the code, it seems pretty strange to me
> > > > that a multithreaded application should not use any mutexes in a
> > > > production environment.
> > >
> > > We use mutexes in production as in development. The lock manager
> >
> > "watches"
> >
> > > our lock usage and blows up Bacula if it detects a problem (deadlock,
> > > out
> >
> > of
> >
> > > order locks, ...). It is a debug tool and not meant or sufficently
> >
> > tested
> >
> > > for production use. Use it at your own risk.
> > >
> > > That said, you were very clever to figure out the problem. Not many
> > > users could do so.
> >
> > Thank you,
> > Regards.
> >
> > > Regards,
> > >
> > > Kern
> > >
> > > > Regards,
> > > > Renaud
> > > >
> > > > Le samedi 09 janvier 2010 à 00:03 +0100, Kern Sibbald a écrit :
> > > > > Hello Arno and Renaud,
> > > > >
> > > > > I can believe that there might be a bug in the lock manager
> > > > > software,
> >
> > but
> >
> > > > > I am very surprised that it is turned on. It should only be turned
> > > > > on
> >
> > for
> >
> > > > > developers, and thus though this patch may be correct (I don't
> > > > > think
> >
> > so,
> >
> > > > > but Eric can answer more definitively), it should never be needed
> > > > > in
> >
> > a
> >
> > > > > production system, and won't work in a production system because of
> >
> > the
> >
> > > > > lock manager being turned off.
> > > > >
> > > > > Can you explain why the lock manager code is turned on?
> > > > >
> > > > > If this is a problem with a misconfigured mail daemon, then it is
> >
> > very
> >
> > > > > likely that this problem has already shown up and has a very
> >
> > different
> >
> > > > > solution. The problem I just mentioned is fixed in the current
> > > > > development version, and the workaround for version 3.0.x is to
> >
> > ensure
> >
> > > > > that either email is turned off or you point to a valid smtp
> > > > > server.
> > > > >
> > > > > Regards,
> > > > >
> > > > > Kern
> > > > >
> > > > > On Friday 08 January 2010 21:32:18 Arno Lehmann wrote:
> > > > > > Hello,
> > > > > >
> > > > > > this is just forwarding your mail to bacula-devel, where it's
> > > > > > more likely to be picked up, looked at, and perhaps integrated
> > > > > > into the code base :-)
> > > > > >
> > > > > > Cheers, and thanks for not only analyzing the problem, but also
> > > > > > providing a possible fix!
> > > > > >
> > > > > > Arno
> > > > > >
> > > > > > 07.01.2010 16:34, Renaud Marquet wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > I'm using bacula 3.0.3 and the director's job queue was stuck
> >
> > after
> >
> > > > > > > running the first job. The others were waiting indefinitely for
> > > > > > > execution. If the director was restarted, I could run only one
> >
> > job,
> >
> > > > > > > and so on.
> > > > > > >
> > > > > > > Googling around I found these 2 posts without satisfying
> > > > > > > anwsers
> >
> >
http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-maili
> >
> > > > > > >ng-l
> >
> > ists-3/bacula-25/upgrade-to-3-0-3-job-is-waiting-for-execution-102156
> >
> > > > > > >/
> >
> >
http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-maili
> >
> > > > > > >ng-l ists-3/bacula-25/job-is-waiting-for-execuition-101508/
> > > > > > >
> > > > > > > I then looked at the code and found there is a deadlock
> > > > > > > happening
> >
> > in
> >
> > > > > > > message handling.
> > > > > > >
> > > > > > > The problem is located in close_msg(JCR *) function in
> > > > > > > message.c. When it encounters an error while sending an e-mail,
> > > > > > > it calls the macro Jmsg1 (line 485) to report it. This macro
> > > > > > > calls
> > > > > > > dispatch_message, which tries to acquire fides_mutex (line
> > > > > > > 738). Unfortunatly, this mutex was already acquired in
> > > > > > > close_msg (line 431), thus resulting in a deadlock (as stated
> > > > > > > in mutex
> >
> > documentation
> >
> > > > > > > for PTHREAD_MUTEX_INITIALIZER kind).
> > > > > > >
> > > > > > > This problem was affecting me because mail daemon was not
> >
> > properly
> >
> > > > > > > configured on my server.
> > > > > > >
> > > > > > > It could be interesting to review these parts of the code to
> >
> > avoid
> >
> > > > > > > such situation.
> > > > > > >
> > > > > > > However I wrote a quick patch for lockmgr.c which simply
> > > > > > > upgrades mutexes to PTHREAD_MUTEX_ERRORCHECK_NP kind and
> > > > > > > resolves this
> >
> > error.
> >
> > > > > > > Hope this would help someone,
> > > > > > > Renaud
> > > > > > >
> > > > > > > patch :
> > > > > > >
> > > > > > > diff -rupN bacula-3.0.3.vanilla/src/lib/lockmgr.c
> > > > > > > bacula-3.0.3.patched/src/lib/lockmgr.c
> > > > > > > --- bacula-3.0.3.vanilla/src/lib/lockmgr.c 2009-10-18
> > > > > > > 11:10:16.000000000 +0200
> > > > > > > +++ bacula-3.0.3.patched/src/lib/lockmgr.c 2009-12-31
> > > > > > > 18:05:59.000000000 +0100
> > > > > > > @@ -616,6 +616,15 @@ void lmgr_cleanup_main()
> > > > > > > */
> > > > > > > int lmgr_mutex_lock(pthread_mutex_t *m, const char *file, int
> >
> > line)
> >
> > > > > > > {
> > > > > > > + /* Patch to avoid deadlock if mutex is locked more than
> > > > > > > once
> >
> > */
> >
> > > > > > > + /* There's some performance hit which makes it probably not
> > > > > > > acceptable */
> > > > > > > + /* for large system usage. */
> > > > > > > + if(*m == PTHREAD_MUTEX_INITIALIZER) {
> > > > > > > + pthread_mutexattr_t attr;
> > > > > > > + pthread_mutexattr_settype( &attr,
> >
> > PTHREAD_MUTEX_ERRORCHECK_NP
> >
> > > > > > > ); + pthread_mutex_init( m, &attr );
> > > > > > > + }
> > > > > > > +
> > > > > > > int ret;
> > > > > > > lmgr_thread_t *self = lmgr_get_thread_info();
> > > > > > > self->pre_P(m, file, line);
> >
> > ---------------------------------------------------------------------
> >
> > > > > > >---- ----- This SF.Net email is sponsored by the Verizon
> > > > > > > Developer Community Take advantage of Verizon's best-in-class
> > > > > > > app
> >
> > development
> >
> > > > > > > support A streamlined, 14 day to market process makes app
> > > > > > > distribution fast and easy Join now and get one step closer to
> > > > > > > millions of Verizon customers
> >
> >
http://p.sf.net/sfu/verizon-dev2dev
> >
> > > > > > > _______________________________________________
> > > > > > > Bacula-users mailing list
> > > > > > >
Bacula-users AT lists.sourceforge DOT net
> > > > > > >
https://lists.sourceforge.net/lists/listinfo/bacula-users
> >
> > -------------------------------------------------------------------------
> >
> > > > >----- This SF.Net email is sponsored by the Verizon Developer
> >
> > Community
> >
> > > > > Take advantage of Verizon's best-in-class app development support A
> > > > > streamlined, 14 day to market process makes app distribution fast
> > > > > and easy Join now and get one step closer to millions of Verizon
> >
> > customers
> >
> > > > >
http://p.sf.net/sfu/verizon-dev2dev
> > > > > _______________________________________________
> > > > > Bacula-users mailing list
> > > > >
Bacula-users AT lists.sourceforge DOT net
> > > > >
https://lists.sourceforge.net/lists/listinfo/bacula-users
> >
> > -------------------------------------------------------------------------
> >----- This SF.Net email is sponsored by the Verizon Developer Community
> > Take advantage of Verizon's best-in-class app development support A
> > streamlined, 14 day to market process makes app distribution fast and
> > easy
> > Join now and get one step closer to millions of Verizon customers
> >
http://p.sf.net/sfu/verizon-dev2dev
> > _______________________________________________
> > Bacula-users mailing list
> >
Bacula-users AT lists.sourceforge DOT net
> >
https://lists.sourceforge.net/lists/listinfo/bacula-users
