Re: [Bacula-users] Fwd: Re: Bandwidth in Bacula (slow rate)
2017-03-23 05:10:57
Hello Alan,
I think you are speaking about what we call Comm Line compression
in Bacula. Note, currently (until community version 9.0.x) Comm
line compression exists only in the Bacula Enterprise Edition, so
it is not generally an issue on this list. There is a data
compression feature in Bacula, but the primary purpose of that is
for data storage compression providing you are not using
deduplication. Obviously compressed data will speed up comm line
transfers, but whether or not there is any total throughput speed
up is a bit hard to estimate because it depends on the compression
algorithm, the FD CPU speed, and the comm line speed.
I forget the exact numbers but if I remember right for lz4, which
is what Bacula Enterprise uses for the comm line compression, the
CPU will max out at about 400-500 MB/s. The comm line compression
is by default on if both components (mostly FD<->SD) support
it, but can be turned off with a directive depending on your
situation.
For the moment, Bacula is not using multithread compression, but
this is planned in the future.
I don't think you are wrong in anything you wrote.
Best regards,
Kern
On 03/22/2017 06:31 PM, Alan Brown
wrote:
FWIW: There is virtually no benefit in network compression for
link speeds of 1Gb/s or faster.
It's a net benefit on WAN links or on 100Mb/s networks, but I
found it had a tendency to slow things down (and use a lot of
CPU!) on 1Gb/s networks vs letting the networking traffic run
uncompressed. On 10Gb/s interconnects it is definitely slower
than letting things run uncompressed.
Experimentation has shown that most modern CPUs seem to max out
about 120-150MB/s throughput on singlethreaded simple
compression, or about 35-40MB/s if gzip is used.
As in many things there are tradeoffs of CPU vs speed and if
your network is "fast enough" then all compressing data in
transit is really achieving is making your system run hotter
with no real benefit.
As far as I'm aware bacula is not using using multithreaded
compression libraries.
Kern, can you correct me if I'm wrong?
Alan
On 22/03/17 15:15, Norbert Gomes wrote:
-------- Message transféré --------
Hi
I've been confronted at the same situation, and that was
due to the compression that was enabled on the FileSet. You
can try to disable it and see the network transfer
performance.
The workaround was to disable compression for some file
extensions
Regards
Norbert
Le 21/03/2017 à 15:35, Josh Fisher a écrit :
On 3/21/2017 9:44 AM, Petar
Kozić wrote:
Hi,
I
will try to describe my problem with slow transfer rate.
I
was setup Bacula server and now every day I will put one
by one instances in backup.
Yesterday
I was put one instances and backup was done fine. But
for about 47,000 files, sum size 930 MB, bacula is
transfer for 11 minutes. Transfer rate is about 1,230
KB/s.
That's
very slow. I try to measure bandwidth with iperf. Both
instances are on linux. That was Gigabite network
between two nodes.
Iperf
says:
Interval
Transfer Bandwidth
0.0-10.0
sec 1.08 GBytes 928 Mbits/sec
I don’t have any Bandwidth statement in configuration
file.
How can I run backup faster ?
What are the SpoolData and SpoolAttributes settings for the
job? This is likely due to database issues. Make sure that
SpoolAttributes=yes so that database updates happen in a
batch at the end of the job, rather than during the job. If
writing to tape, then make sure SpoolData=yes. If writing to
disk, then SpoolData=no.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
|
|