Bacula-users

Re: [Bacula-users] bacula-fd crashes on FreeBSD 9.2

2013-10-16 15:43:30
Subject: Re: [Bacula-users] bacula-fd crashes on FreeBSD 9.2
From: dweimer <dweimer AT dweimer DOT net>
To: bacula-users AT lists.sourceforge DOT net
Date: Wed, 16 Oct 2013 14:40:37 -0500
On 10/16/2013 1:21 pm, David Newman wrote:
> On 10/16/13 8:35 AM, dweimer wrote:
>> On 10/09/2013 6:41 pm, David Newman wrote:
>>> FreeBSD 9.2-RELEASE, bacula-client-5.2.12_3 installed from ports
>>> 
>>> Ever since upgrading this host to FreeBSD 9.2, bacula-fd crashes as 
>>> soon
>>> as bacula-dir starts a backup job. The entry in /var/log/messages is:
>>> 
>>> Oct  9 16:25:50 o bacula-fd: Bacula interrupted by signal 0: UNKNOWN
>>> SIGNAL
>>> 
>>> Backups worked fine on this host running FreeBSD 9.1 and other hosts
>>> upgraded to FreeBSD 9.2 run backups OK.
>>> 
>>> I've done the uninstall/reinstall thing with the bacula-client port, 
>>> but
>>> that made no difference.
>>> 
>>> Thanks in advance for troubleshooting clues.
>>> 
>>> dn
>>> 
>>> 
>> 
>> David,
>> 
>>    I have hit this on one of my servers as well, I have two upgraded 
>> to
>> 9.2, both built from source upgraded from 9.1p7 to 9.2.  There are 
>> some
>> differences, one is the Bacula server, and exists entirely to manage
>> backups.  The other is a web / email server, the web / email server is
>> working fine, the Bacula server was working fine, and has now hit the
>> problem yours has, oddly enough, I can run the database backup just
>> fine, its just the normal backup that fails.
>> 
>>    I have a backup running now though, keeping my fingers crossed, as
>> its actually made it past the point it normally fails, and I am
>> wondering, is it that I selected a full backup instead of incremental?
>> Since the catalog backup that's full is working, this makes me 
>> curious,
>> have you ran into this problem on full backups or just incremental 
>> backups?
>> 
>>    I haven't been able to find any difference's in the Bacula
>> dependencies on either of the systems, they are all running at the 
>> same
>> revision levels, I am using lzo compression on both (tried disabling,
>> didn't stop crash), both systems are running on ZFS file system, both
>> zpools have been upgraded after the update, both run the same script 
>> to
>> generate database dumps, and take ZFS snapshots before backup, there 
>> is
>> a different list of file sets to snapshot, and backup, but that's it.
>> The web / email server does have a lot of additional ports installed
>> that the Bacula server doesn't.  And of course the Bacula server is
>> running both the bacula-sd and bacula-dir in addition to the bacula-fd
>> service.
> 
> I've isolated my issue to a database backup job that runs before the
> backup, via a ClientRunBeforeJob call on the bacula server. Without the
> database backup, backups complete OK; with the backup, they don't.
> 
> This worked OK before the upgrade. The script works OK when executed
> manually. Another 9.2 host running an identical script backs up OK, 
> also
> using ClientRunBeforeJob.
> 
> Next step is to try Martin Simmons' suggestion about using a debugger 
> to
> look deeper. Will let you know what I find.
> 
> dn
> 
Interesting, subsequent incremental backups ran fine after the full 
worked, guess if it occurs again, I will play around with removing my 
database and snapshot jobs and see if I can narrow it down more.

I have a client run before job on each of the servers as well, both run 
a single before.sh script (using /bin/sh as interpreter) which calls two 
other scripts, the first is a database backup, also just a shell script 
and the second is a python script to create the ZFS snapshots.  
According to the console logs they completed successfully, prior to the 
crash.

The scripts were copied from one server to the other, only change is the 
value of the list variable I am using to specify which ZFS file sets to 
snapshot in the python script, odd that I have only hit the problem on 
the one server.

-- 
Thanks,
    Dean E. Weimer
    http://www.dweimer.net/

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users