Bacula-users

Re: [Bacula-users] backup job failing on Debian client

2015-07-30 13:02:50
Subject: Re: [Bacula-users] backup job failing on Debian client
From: Kern Sibbald <kern AT sibbald DOT com>
To: "Rao, Uthra R. (GSFC-672.0)[ADNET SYSTEMS INC]" <uthra.r.rao AT nasa DOT gov>, Wanderlei Huttel <wanderleihuttel AT gmail DOT com>
Date: Thu, 30 Jul 2015 18:56:33 +0200
Some times debug output creates bus errors on architectures such as Sparc for two reasons.
1. They are debug statements so they are not always carefully written nor checked.

2. We do not usually run on Sparc machines. 

If you are getting a bus error, we will need a traceback then we can fix the problem.

Best regards,
Kern

On 30.07.2015 15:37, Rao, Uthra R. (GSFC-672.0)[ADNET SYSTEMS INC] wrote:

Wanderlei,

 

I enabled debug on the Debian (sparc) client and manually tried to run a full backup of this client. When it is just about to start spooling the bacula-fd crashes on the client. Here is the debug output:

 

*******-fd: job.c:288-0 Executing JobId= command.

*******-fd: job.c:1733-0 set sd auth key

*******-fd: job.c:546-0 JobId=0 Auth=dummy

*******-fd: fd_plugins.c:1033-0 plugin list is NULL

*******-fd: job.c:272-0 <dird: status*******-fd: job.c:288-0 Executing status command.

*******-fd: runscript.c:108-0 runscript: running all RUNSCRIPT object (ClientAfterJob) JobStatus=C

*******-fd: pythonlib.c:225-0 No startup module.

*******-fd: job.c:401-0 Calling term_find_files

*******-fd: job.c:406-0 Done with term_find_files

*******-fd: runscript.c:286-0 runscript: freeing all RUNSCRIPTS object

*******-fd: job.c:408-0 Done with free_jcr

*******-fd: mem_pool.c:375-0 garbage collect memory pool

*******-fd: job.c:2377-0 3000 OK data

*******-fd: pythonlib.c:225-0 No startup module.

*******-fd: job.c:1936-0 begin blast ff=5ac60

*******-fd: backup.c:89-0 bfiled: opened data connection 6 to stored

*******-fd: find.c:94-0 Enter set_find_options()

*******-fd: find.c:97-0 Leave set_find_options()

*******-fd: find.c:211-0 Verify=<VVpingus5> Accurate=<Cmcs> BaseJob=<Jspug5>

*******-fd: find.c:215-0 F /

*******-fd: find_one.c:374-0 File ----: /

*******-fd: htable.c:79-0 malloc buf=f5956020 size=262144 rem=262132

*******-fd: htable.c:198-0 Allocated big buffer of 262144 bytes

*******-fd: htable.c:140-0 Leave hash_index hash=0xd index=46

*******-fd: htable.c:349-0 Insert: hash=0 index=13

*******-fd: htable.c:352-0 Insert hp=f595602c index=46 item=f595602c offset=0

Bacula interrupted by signal 10: BUS error

Kaboom! bacula-fd, *******-fd got signal 10 - BUS error. Attempting traceback.

Kaboom! exepath=/usr/sbin/

*******-fd: signal.c:197-0 Working=/opt/bacula/var

*******-fd: signal.c:198-0 btpath=/usr/sbin/btraceback

*******-fd: signal.c:199-0 exepath=/usr/sbin/bacula-fd

*******-fd: signal.c:228-0 Doing waitpid

Calling: /usr/sbin/btraceback /usr/sbin/bacula-fd 16733 /opt/bacula/var

 

I see the “BUS error” in the above output. I am not sure if this is the problem?  Thank you.

 

Uthra

 

 

From: Wanderlei Huttel [mailto:wanderleihuttel AT gmail DOT com]
Sent: Tuesday, July 28, 2015 4:09 PM
To: Rao, Uthra R. (GSFC-672.0)[ADNET SYSTEMS INC]
Cc: bacula-users AT lists.sourceforge DOT net
Subject: Re: [Bacula-users] backup job failing on Debian client

 

Hi Uthra

 

Another thing that you would try is enable debug mode and do a backup:

 

Getting Debug Output from Bacula

 

Each of the daemons normally has debug compiled into the program, but disabled. There are two ways to enable the debug output. One is to add the -d nnn option on the command line when starting the debugger. The nnn is the debug level, and generally anything between 50 and 200 is reasonable. The higher the number, the more output is produced. The output is written to standard output.

The second way of getting debug output is to dynamically turn it on using the Console using the setdebug command. The full syntax of the command is:

 

 setdebug level=nnn client=client-name storage=storage-name dir

 

If none of the options are given, the command will prompt you. You can selectively turn on/off debugging in any or all the daemons (i.e. it is not necessary to specify all the components of the above command).

 

 

Best Regards

Wanderlei

 

2015-07-28 16:07 GMT-03:00 Rao, Uthra R. (GSFC-672.0)[ADNET SYSTEMS INC] <uthra.r.rao AT nasa DOT gov>:

Iptables? Is there any way you are using a 32bits client on a 64 environment?


- My bacula server is RHEL 6 (64-bit) and the client is Debian O.S. (64-bit sparc)

 

I opened port 9102 on the client.

 

Thank you.

Uthra

 

 

From: Heitor Faria [mailto:heitor AT bacula.com DOT br]
Sent: Tuesday, July 28, 2015 11:50 AM
To: Rao, Uthra R. (GSFC-672.0)[ADNET SYSTEMS INC]
Cc: Ana Emília M. Arruda; bacula-users AT lists.sourceforge DOT net
Subject: Re: [Bacula-users] backup job failing on Debian client

 

 

Hello Uthra: do you have selinux enabled on the storage daemon side (RHEL)? Does your connection go through a firewall or maybe a nasty router?

Depending on your answers you may try to use the "Heartbeat Interval" option in the affected daemons.

 

Hello Heitor,

 

SELINUX is not enabled on the bacula server. All the other client backups configured on this bacula server is working fine. I tried adding the “Heartbeat Interval 60” in the client–dir and client–fd files. The backup of the Debian client is still failing:

 

***Fatal error: Network error with FD during Backup: ERR=No data available 28-Jul 11:28 lindy-sd JobId 18819: Fatal error: append.c:160 Error reading data header from FD. ERR=No data available****

Iptables? Is there any way you are using a 32bits client on a 64 environment?

 

Regards,

===========================================================================
Heitor Medrado de Faria - LPIC-III | ITIL-F |  Bacula Systems Certified Administrator II

===========================================================================

Thank you.

Uthra

 

From: Heitor Faria [mailto:heitor AT bacula.com DOT br]
Sent: Tuesday, July 28, 2015 10:03 AM
To: Rao, Uthra R. (GSFC-672.0)[ADNET SYSTEMS INC]
Cc: Ana Emília M. Arruda; bacula-users AT lists.sourceforge DOT net


Subject: Re: [Bacula-users] backup job failing on Debian client

Hi Ana,

 

I had just cut and pasted the relevant part of the log. The fileset is -> “File = /” (it has to back-up everything from root). 

 

Thank you.

 

From: Ana Emília M. Arruda [mailto:emiliaarruda AT gmail DOT com]
Sent: Tuesday, July 28, 2015 1:05 AM
To: Rao, Uthra R. (GSFC-672.0)[ADNET SYSTEMS INC]
Cc: bacula-users AT lists.sourceforge DOT net
Subject: Re: [Bacula-users] backup job failing on Debian client

Hello Uthra,

 

Do you have more log lines for this job? Have you checked if you have the asked filset in this client? It seems that there is no data in this Debian client for the fileset configured.

 

Best regards,

Ana

On Mon, Jul 27, 2015 at 12:12 PM, Rao, Uthra R. (GSFC-672.0)[ADNET SYSTEMS INC] <uthra.r.rao AT nasa DOT gov> wrote:

I am having a problem with a Debian client.  Here is the information:

-          Bacula Server is a  RHEL 6 : Bacula version 5.2.12 with a Tape Library connected to it (LTO5)

-          Bacula client is Debian O.S. (bacula version 5.2.6)

I am able to connect to the client from “bconsole”. I am also able to start a backup job but it fails before it starts spooling.  The bacula report shows:

Fatal error: Network error with FD during Backup: ERR=No data available 27-Jul 10:53 lindy-sd JobId 18794: Fatal error: append.c:160 Error reading data header from FD. ERR=No data available

Error: Director's comm line to SD dropped

Hello Uthra: do you have selinux enabled on the storage daemon side (RHEL)? Does your connection go through a firewall or maybe a nasty router?

Depending on your answers you may try to use the "Heartbeat Interval" option in the affected daemons.

 

Regards,

=======================================================================
Heitor Medrado de Faria  - LPIC-III | ITIL-F |  Bacula Systems Certified Administrator II

29 de junho a 13 de julho: Treinamento Telepresencial Bacula: http://www.bacula.com.br/?p=2174

Site: www.bacula.com.br | Facebook: heitor.faria

========================================================================

 

I ran the bacua-sd in a debug mode and got the output:

 

*****-sd: dircmd.c:175-0 Conn: Hello Director *****-dir calling

*****-sd: dircmd.c:191-0 Got a DIR connection at 27-Jul-2015 10:53:38

*****-sd: cram-md5.c:72-0 send: auth cram-md5 <1298524369.1438008818@*****-sd> ssl=0

*****-sd: cram-md5.c:131-0 cram-get received: auth cram-md5 <1052271812.1438008818@*****-dir> ssl=0

*****-sd: cram-md5.c:150-0 sending resp to challenge: y//5v5k543Fl2i/YWyF5uB

*****-sd: dircmd.c:213-0 Message channel init completed.

*****-sd: dircmd.c:220-0 <dird: cancel Job=explore.2015-07-27_10.52.05_03

 

*****-sd: dircmd.c:234-0 Do command: cancel

*****-sd: askdir.c:345-18794 Update cat VolBytes=1024

*****-sd: askdir.c:368-18794 >dird CatReq Job=explore.2015-07-27_10.52.05_03 UpdateMedia VolName=B01160L5 VolJobs=1 VolFiles=1 VolBlocks=0 VolBytes=1024 VolMounts=3 VolErrors=0 VolWrites=15122819 MaxVolBytes=0 

EndTime=1438008823 VolStatus=Append Slot=32 relabel=0 InChanger=1 VolReadTime=0 VolWriteTime=11237175650 VolFirstWritten=0 VolParts=0

*****-sd: acquire.c:528-18794 dir_update_vol_info. Release vol=B01160L5 dev="Drive-1" (/dev/nst0)

*****-sd: vol_mgr.c:549-18794 === clear in_use vol=B01160L5

*****-sd: vol_mgr.c:565-18794 === set not reserved vol=B01160L5 num_writers=0 dev_reserved=0 dev="Drive-1" (/dev/nst0)

*****-sd: acquire.c:545-18794 0 writers, 0 reserve, dev="Drive-1" (/dev/nst0)

*****-sd: acquire.c:584-18794 JobId=18794 broadcast wait_device_release at 27-Jul-2015 10:53:43

*****-sd: acquire.c:605-18794 ===== Device "Drive-1" (/dev/nst0) released by JobId=18794

*****-sd: spool.c:785-18794 Close attr spool file at 27-Jul-2015 10:53:43

*****-sd: append.c:326-18794 return from do_append_data() ok=0

*****-sd: dircmd.c:237-18794 Command run requests quit

*****-sd: jcr.c:183-18794 write_last_jobs seek to 192

 

Any help is appreciated.

 

Thank you.

Uthra

 

 


------------------------------------------------------------------------------

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

 


------------------------------------------------------------------------------

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

 



------------------------------------------------------------------------------


_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

------------------------------------------------------------------------------
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
<Prev in Thread] Current Thread [Next in Thread>