Hi Josh,
Can you tell me how you come to the conclusion that it's timing out after 2
minutes?
>> 20-Oct 03:04 backup1-dir JobId 18582: Volume used once. Marking Volume
>> "Vol2709" as Used.
>> 20-Oct 04:05 host01 JobId 18582: Fatal error: backup.c:892 Network send
>> error to SD. ERR=Broken pipe
There are 59 minutes between above 2 lines.
Or did I mis something?
Best,
Nextpertise
On Oct 22, 2013, at 4:22 PM, Josh Fisher <jfisher AT pvct DOT com> wrote:
>
> On 10/21/2013 7:57 AM, Teun Ouwehand [Nextpertise] wrote:
>> Hi Bacula enthusiasts,
>>
>> Since one year ago we implemented for our hosted servers a bacula backup
>> domain. Since the beginning we have sometimes (regularly on sundays with
>> full backups) the problem: "Fatal error: backup.c:892 Network send error to
>> SD. ERR=Broken pipe"
>>
>> This is an hot issue on this mailing list and commonly the firewall is the
>> problem. We have done several things as changing TTL's on the firewall side
>> and implemented the heartbeat line on both of FD and SD.
>>
>> Heartbeat setting: Heartbeat Interval = 300
>
> From the log below it looks like something is timing out at 2 minutes
> (120 seconds), but you have heartbeat interval at 5 minutes (300
> seconds). I would try heartbeat interval = 90 and see if that helps.
>
>>
>> It looks like the problem always have to do with a pre-backup-script in this
>> case "automysqlbackup" and when it run's for at least 15 minutes. Why is the
>> FD starting a connection to the SD before the pre-backup-script?
>>
>> Below you find the backup log of one of the failing hosts. We made host01 of
>> the hostname to make it anonymous.
>>
>> Hope someone can put us in the right direction!
>>
>> Best,
>>
>> Nextpertise
>>
>> Log:
>> 20-Oct 03:03 backup1-dir JobId 18582: Start Backup JobId 18582,
>> Job=Backup-host01.2013-10-20_01.55.00_57
>> 20-Oct 03:03 backup1-dir JobId 18582: Using Device "FileStorage"
>> 20-Oct 03:03 host01 JobId 18582: ClientRunBeforeJob: run command
>> "/usr/local/bin/automysqlbackup"
>> 20-Oct 03:03 host01 JobId 18582: ClientRunBeforeJob: Invoking backup method.
>> 20-Oct 03:03 host01 JobId 18582: ClientRunBeforeJob:
>> 20-Oct 03:03 host01 JobId 18582: ClientRunBeforeJob: Parsed config file
>> "/etc/automysqlbackup/automysqlbackup.conf"
>> 20-Oct 03:03 host01 JobId 18582: ClientRunBeforeJob:
>> 20-Oct 03:03 host01 JobId 18582: ClientRunBeforeJob: # Checking for
>> permissions to write to folders:
>> 20-Oct 03:03 host01 JobId 18582: ClientRunBeforeJob: base folder
>> /var/data/backups ... exists ... ok.
>> 20-Oct 03:03 host01 JobId 18582: ClientRunBeforeJob: backup folder
>> /var/data/backups/mysql ... exists ... writable? yes. Proceeding.
>> 20-Oct 03:04 backup1-sd JobId 18582: Recycled volume "Vol2709" on device
>> "FileStorage" (/var/data/backups/bacula/), all previous data lost.
>> 20-Oct 03:04 backup1-dir JobId 18582: Volume used once. Marking Volume
>> "Vol2709" as Used.
>> 20-Oct 04:05 host01 JobId 18582: Fatal error: backup.c:892 Network send
>> error to SD. ERR=Broken pipe
>> 20-Oct 04:05 backup1-sd JobId 18582: JobId=18582
>> Job="Backup-host01.2013-10-20_01.55.00_57" marked to be canceled.
>> 20-Oct 04:05 backup1-dir JobId 18582: Error: Bacula backup1-dir 5.0.2
>> (28Apr10): 20-Oct-2013 04:05:17
>> Build OS: x86_64-pc-linux-gnu debian squeeze/sid
>> JobId: 18582
>> Job: Backup-host01.2013-10-20_01.55.00_57
>> Backup Level: Full
>> Client: "host01" 2.4.4 (28Dec08)
>> x86_64-redhat-linux-gnu,redhat,Enterprise release
>> FileSet: "SetHostFileset" 2013-07-10 01:55:05
>> Pool: "File" (From Job resource)
>> Catalog: "MyCatalog" (From Client resource)
>> Storage: "File" (From Job resource)
>> Scheduled time: 20-Oct-2013 01:55:00
>> Start time: 20-Oct-2013 03:04:09
>> End time: 20-Oct-2013 04:05:17
>> Elapsed time: 1 hour 1 min 8 secs
>> Priority: 10
>> FD Files Written: 105,990
>> SD Files Written: 0
>> FD Bytes Written: 14,873,606,478 (14.87 GB)
>> SD Bytes Written: 0 (0 B)
>> Rate: 4055.0 KB/s
>> Software Compression: None
>> VSS: no
>> Encryption: no
>> Accurate: no
>> Volume name(s): Vol2709|Vol2710
>> Volume Session Id: 6040
>> Volume Session Time: 1373120953
>> Last Volume Bytes: 50,996,788,622 (50.99 GB)
>> Non-fatal FD errors: 0
>> SD Errors: 0
>> FD termination status: Error
>> SD termination status: Error
>> Termination: *** Backup Error ***
>> ------------------------------------------------------------------------------
>> October Webinars: Code for Performance
>> Free Intel webinars can help you accelerate application performance.
>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
>> the latest Intel processors and coprocessors. See abstracts and register >
>> http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Bacula-users mailing list
>> Bacula-users AT lists.sourceforge DOT net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
