Bacula-users

Re: [Bacula-users] Improve bacula performance with software compression

2013-10-22 10:41:28
Subject: Re: [Bacula-users] Improve bacula performance with software compression
From: Josh Fisher <jfisher AT pvct DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Tue, 22 Oct 2013 10:39:16 -0400
On 10/22/2013 8:16 AM, Федорищев Игорь Николаевич wrote:
> Hello.
> Bacula 5.0.0, CentOS 6.4 x86_64, MySQL 5.5.34. Bacula was installed from 
> CentOS Remi repository.
> Bacula server: bacula-dir, bacula-sd. Bacula client: bacula-fd. Both machines 
> are in local network and have 1Gbit network cards. Maximum io
> capacity on bacula server is about 180 Mbytes/per sec (tested). Backups are 
> stored on disk, no tape drive used.

It doesn't matter how fast the server machine is so long as it is faster 
than the client. The backup speed will still be determined by how fast 
the client can stream data to the server. So the client is simply unable 
to perform gzip compression fast enough. Either use no compression, use 
LZO or some faster compression method on the client, or increase the 
hardware performance of the client machine.

Also, make sure the bacula database, the backup storage area, and the 
spool directory are not on the same physical drives. That will cause 
disk thrashing and likely degrade disk write performance on the server.


>
> Bacula client is a machine with 1.2Tb data (over 1M files). When I enable 
> compression on the client (compression=GZIP),
> backup performance is extremely degraded, network interface load is down to 
> 100 Mbit/s. Then I disable compression, restarts backup and achive up to 500 
> Mbit
> interface load. Looks like good: 500Mbit/s - 1.2Tb data backup time is about 
> 6 hours. But I want to store files compressed on the backup server to reduce 
> disk
> space usage. I googled up it and found that no way to enable server-side 
> compression in Bacula. Also I found that people are disables compression
> in Bacula and store volumes on filesystem with compression support, like ZFS. 
> I think that no good result in this way because Bacula had it's own archive
> binary format and volume (like binary file) compression by any external 
> algorithm must be ineffective (~ 1.1:1 ratio).
>
> Ways to resolution problem?
> 1. Backups uncompressed, then unpack volume with external tools, compress and 
> pack again.
> (..How Bacula does operate with that again? No way..)
>
> What else?
>
> I want to store files in volumes compressed without losts network copy 
> performance.
>
> Thanks, sorry for my English.
>
> --
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users