Bacula-users

Re: [Bacula-users] Slow backup rate

2009-12-09 10:57:44
Subject: Re: [Bacula-users] Slow backup rate
From: Sean M Clark <smclark AT tamu DOT edu>
To: bacula-users <bacula-users AT lists.sourceforge DOT net>
Date: Wed, 09 Dec 2009 09:54:41 -0600
Carlo Filippetto wrote:
> Hi,
> I would like to know if is true that I have so slow troughput as this:
[...]
> FULL
> -----
>   Elapsed time:           1 day 22 hours 13 mins 37 secs
[...]
>   Rate:                   371.7 KB/s
>   Software Compression:   15.5 %
> 
[...]
> All my jobs have the maximum compression
> 
> /    Options {
>         compression = GZIP9 #aggiungo compressione massima
> 
> 
> /P.S. is this the maximum compression??

You didn't mention what speed your network connection is, but that's
slow even for 10BaseT.  I would expect to see at least about 800 KB/s on
10Mbit, or at least 6-8MB/s on 100MB, or at least 25-35MB/s on Gigabit
(Yeah, I know Gigabit ought to be up around 80MB/s at least, but in
practice I've yet to see anything manage it personally.  I know it's
possible, though.)

I saw someone else mentioned this already, but yes, GZIP9 is the
maximum, and that might actually be WHY the rate is slow.
The higher you set the compression rate, the more time the program
spends trying to cram more data into each packet before sending it.
If the compression rate is high enough, it may actually take much
more time to do the compression than is saved by sending less data.

A stupid analogy: if bacula-fd is the shipping department of a company,
"no compression" means the stuff (file data) being shipped is dumped
into a box until it reaches the top, then the box is closed and sent on
its way.  Compression level 1 would be like pausing to press down on the
stuff in the box once and then top off the extra space with a little bit
more file data before sending it.  Compression level 9 is like dumping
the stuff in, smashing it down, dumping more in, smashing it down,
dumping more in, jumping up and down on top of it, then recruiting some
guys from the next department over to stand on top while you seal the
box.  The box ends up holding a lot more, but it takes so much longer to
get the box ready to go that you end up not getting as much shipped out
in the same amount of time.

It can be even worse if the client machine is comparatively low in CPU
power or is heavily loaded (e.g. an old Windows box running Symantec
antivirus doing "active protection" and scanning every file that bacula
tries to examine or send...).

Unless space on the backup media or bandwidth usage are the biggest
concerns, I tend to drop the compression all the way down to
GZIP1-GZIP4, or turn it off altogether.

On the other hand (or "other thread", as the case may be, looking at the
discussion of bandwidth throttling), setting an unnecessarily high
compression level might also be used as a crude way of limiting
bandwidth usage if you don't care so much how long the backup actually
takes.

( I'm hoping to someday see "LZMA1-LZMA9" or "XZ1-XZ9" compression
options, too... )

------------------------------------------------------------------------------
Return on Information:
Google Enterprise Search pays you back
Get the facts.
http://p.sf.net/sfu/google-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

<Prev in Thread] Current Thread [Next in Thread>