On 05/30/2011 02:11 PM, reiserfs wrote:
> Hello, im new with Bacula scene, i have used the HP Dataprotector with a HP 
> Library fiber channel backup.
>
> With the Dataprotector i got 1gbps interfaces and switch and all jobs get 
> done very fast, with transfer like 50-80MB/s.
>
> Now im using Bacula with a DELL TL2000 iSCSI, and with my first experience i 
> got only 6MB/s transfer with 1gbps interfaces and switch.
>
> So what im missing
>
> Used to tes:
> Bacula Director runing on Slackware64 13.1
> Bacula Cliente Windows 2003 Server
Turning on software gzip compression on the client is definitely a major
performance killer, unfortunately, so that would be my first guess as
well.  This looks like a good place to mention some testing I've done.

I've been doing some testing lately due to also being somewhat
aggravated as the apparently slow transfer rates I get during Bacula
backups, but it's starting to look like it's not really Bacula's fault
most of the time.  Most of the time, it looks like the problem is just
how fast the client can read files off of the disk and send them.  The
network (at least on Gb) is not usually the problem, nor even database
activity on the director (attribute spooling will help if you DO have
any problems with that).

Encryption and gzip compression by the client introduce major latency
that unavoidably slows down the transfer, and this isn't specifically a
bacula client issue.  Other things I have seen that cause major
slowdowns are antivirus software on Windows (particularly "on-access
scanning") and active use of the computer while the backup is running.

Regarding compression, specifically, though - testing on my laptop here,
I tested just reading files from /usr and /home with "tar", piping them
through "pv" to get the transfer rate (and then dumping them directly to
/dev/null).  I repeated the tests then with some different compression
schemes inserted.
for example:

tar -cf - /usr | pv -b -r -a > /dev/null ("No Compression")
tar -cf - /usr | gzip -c | pv -b -r -a > /dev/null ("GZIP")
tar -cf - /usr | gzip -1 -c | pv -b -r -a > /dev/null ("GZIP1")
tar -cf - /usr/ | lzop -c | pv -b -r -a > /dev/null ("LZO")

(and repeated for /home)

Here are my results:

/usr
No Compression: 5.58GB total data, Avg 13.1MB/s (436s to finish)
GZIP: 2.11GB Total data, Avg 2.97MB/s (727s to finish)
GZIP1: 2.36GB Total data, Avg 4.13MB/s (585s to finish)
LZO: 2.82GB Total Data, Avg 6.48MB/s (445s to finish)

/home (includes a lot of e.g. media files that are not very compressible)
No Compression: 91.56GB Total Data, 34.5MB/s Avg,, (~2700s to finish)
GZIP: 77.1GB Total Data, 9.78MB/s Avg, (8072s to finish)
GZIP1: 77.6GB Total Data, 11.7MB/s Avg, (~6790s to finish)
LZO: 80.6GB Total Data, 28.3MB/s Avg, (~2900s to finish)

So, yes, if you have gzip compression turned on, you'll almost certainly
see a huge increase in speed if you turn it off (I believe most tape
drives can or will do compression in hardware, so you don't need to
pre-compress at the client). 

If you are backing up to disk as I am (or for some reason aren't doing
hardware compression on the tape drive), you can also get a small speed
increase by dropping the gzip compression down to the minimum
("Compress=GZIP1" in the FileSet), which seem to compress almost as well
overall but induces less latency.

FEATURE REQUEST:
However, assuming my tests so far are representative, it looks like LZO
compression can get backup jobs transferred in almost the same amount of
time as no compression at all, while still substantially reducing the
amount of data transferred and stored (not as much as GZIP does, but
still a noteworthy amount).  Is it possible we could get a
"Compress=LZOP" capability added to bacula-fd?

tl;dr: Turn off compression until and unless an LZO compression option
is implemented, unless you are desperate for space on your backup media,
in which case you'll just have to cope with the slow backups.

------------------------------------------------------------------------------
Simplify data backup and recovery for your virtual environment with vRanger. 
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today. 
http://p.sf.net/sfu/quest-sfdev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users