Bacula-users

[Bacula-users] Very low performance with compression and encryption !

2011-01-20 09:40:06
Subject: [Bacula-users] Very low performance with compression and encryption !
From: Hugo Letemplier <hugo.let.35 AT gmail DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Thu, 20 Jan 2011 15:36:37 +0100
Hi,

I am running bacula 5.0.3 on CentOS 5.6.

When I run a simple Job I can have rates between 20 or 40 MB/s over a
Gigabyte network but when I am running this job with client encryption
and compression everything become slow below 5 MB/s and sometimes
under 500 KB/s.
Generally I test with a backup job on a full system and then do Inc.
Both are very slow.
What should I check ?
Maybe it's my file set ?
I used both MacOSX client and CentOS clients with equivalent slowness.
Does anybody got this problem ?
Maybe it can come from zlib or openssl that I use ?
I am practically sure it's coming from the File Daemon.
I use compression at 3/10 level

Thank you very much

Hugo

I joined the config I used to generate the RPM :
#!/bin/sh
cat << __EOC__
$ ./configure  '--host=i686-redhat-linux-gnu'
'--build=i686-redhat-linux-gnu' '--target=i386-redhat-linux'
'--program-prefix=' '--exec-prefix=/usr' '--bindir=/usr/bin'
'--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include'
'--libdir=/usr/lib' '--libexecdir=/usr/libexec' '--localstatedir=/var'
'--sharedstatedir=/usr/com' '--infodir=/usr/share/info'
'--prefix=/usr' '--sbindir=/usr/sbin' '--sysconfdir=/etc/bacula'
'--mandir=/usr/share/man' '--with-scriptdir=/usr/lib/bacula'
'--with-working-dir=/var/lib/bacula'
'--with-plugindir=/usr/lib/bacula' '--with-pid-dir=/var/run'
'--with-subsys-dir=/var/lock/subsys' '--enable-smartalloc'
'--disable-gnome' '--disable-bwx-console' '--disable-tray-monitor'
'--disable-conio' '--enable-readline' '--with-postgresql'
'--disable-bat' '--with-dir-user=bacula' '--with-dir-group=bacula'
'--with-sd-user=bacula' '--with-sd-group=disk' '--with-fd-user=root'
'--with-fd-group=bacula'
'--with-dir-password=XXX_REPLACE_WITH_DIRECTOR_PASSWORD_XXX'
'--with-fd-password=XXX_REPLACE_WITH_CLIENT_PASSWORD_XXX'
'--with-sd-password=XXX_REPLACE_WITH_STORAGE_PASSWORD_XXX'
'--with-mon-dir-password=XXX_REPLACE_WITH_DIRECTOR_MONITOR_PASSWORD_XXX'
'--with-mon-fd-password=XXX_REPLACE_WITH_CLIENT_MONITOR_PASSWORD_XXX'
'--with-mon-sd-password=XXX_REPLACE_WITH_STORAGE_MONITOR_PASSWORD_XXX'
'--with-openssl' 'build_alias=i686-redhat-linux-gnu'
'host_alias=i686-redhat-linux-gnu' 'target_alias=i386-redhat-linux'
'CFLAGS=-O2 -g -m32 -march=i386 -mtune=generic
-fasynchronous-unwind-tables' 'CXXFLAGS=-O2 -g -m32 -march=i386
-mtune=generic -fasynchronous-unwind-tables'

Configuration on Thu Oct 14 13:33:48 CEST 2010:

   Host:                    i686-redhat-linux-gnu -- redhat
   Bacula version:          Bacula 5.0.3 (30 August 2010)
   Source code location:    .
   Install binaries:        /usr/sbin
   Install libraries:       /usr/lib
   Install config files:    /etc/bacula
   Scripts directory:       /usr/lib/bacula
   Archive directory:       /tmp
   Working directory:       /var/lib/bacula
   PID directory:           /var/run
   Subsys directory:        /var/lock/subsys
   Man directory:           /usr/share/man
   Data directory:          /usr/share
   Plugin directory:        /usr/lib/bacula
   C Compiler:              gcc 4.1.2
   C++ Compiler:            /usr/bin/g++ 4.1.2
   Compiler flags:           -O2 -g -m32 -march=i386 -mtune=generic
-fasynchronous-unwind-tables -fno-strict-aliasing -fno-exceptions
-fno-rtti
   Linker flags:        
   Libraries:               -lpthread -ldl
   Statically Linked Tools: no
   Statically Linked FD:    no
   Statically Linked SD:    no
   Statically Linked DIR:   no
   Statically Linked CONS:  no
   Database type:           PostgreSQL
   Database port:       
   Database lib:            -L/usr/lib -lpq -lcrypt
   Database name:           bacula
   Database user:           bacula

   Job Output Email:        root@localhost
   Traceback Email:         root@localhost
   SMTP Host Address:       localhost

   Director Port:           9101
   File daemon Port:        9102
   Storage daemon Port:     9103

   Director User:           bacula
   Director Group:          bacula
   Storage Daemon User:     bacula
   Storage DaemonGroup:     disk
   File Daemon User:        root
   File Daemon Group:       bacula

   SQL binaries Directory   /usr/bin

   Large file support:      yes
   Bacula conio support:    no -lreadline -lncurses
   readline support:        yes
   TCP Wrappers support:    no
   TLS support:             yes
   Encryption support:      yes
   ZLIB support:            yes
   enable-smartalloc:       yes
   enable-lockmgr:          no
   bat support:             no
   enable-gnome:            no
   enable-bwx-console:      no
   enable-tray-monitor:     no
   client-only:             no
   build-dird:              yes
   build-stored:            yes
   Plugin support:          yes
   AFS support:             no
   ACL support:             yes
   XATTR support:           yes
   Python support:          no
   Batch insert enabled:    yes


__EOC__


Here is my file set :
FileSet {
        Name = "MacFull"
        Include {
            Options {
                    HFSPlus Support = yes
                    Signature = MD5
                    ACL Support = yes
            }
            Options {
            @/etc/bacula/DO_NOT_INCLUDE_MAC
            exclude = yes
        }
        Options {
            @/etc/bacula/DO_NOT_COMPRESS
            exclude = no
        }
        Options {
            compression = GZIP3
        }
            File = /
    }
}


##########
# DO_NOT_COMPRESS contains lots of file ext that I listed like that :
##########
wildfile = ".mp3"
wildfile = ".mp2"
wildfile = ".m4a"
wildfile = ".wma"
wildfile = ".flv"
wildfile = ".mpg"
wildfile = ".mpeg"
wildfile = ".divx"
wildfile = ".wmv"
wildfile = ".7z"
wildfile = ".rar"
wildfile = ".zip"
wildfile = ".gz"
wildfile = ".jpg"
wildfile = ".jpe"
wildfile = ".tif"

It's only a part of my list, there are more than 60 wildfile lines

#############
# Here is the job.
#############
Client {
 Name = Toto
 Address = toto.toto.com
 FDPort = 9102
 Catalog = MyCatalog
 Password = "XXXXXX"
 AutoPrune = yes                     # Prune expired Jobs/Files
}

Job {
  Name = Sauvegarde:Toto
  Type = Backup
  JobDefs = WorkstationJob
  Client = Toto
  FileSet = MacFull
  }
###########
# Client Config
###########
FileDaemon {                          # this is me
  Name = Toto
  FDport = 9102                  # where we listen for the director
  WorkingDirectory = /private/var/bacula/working
  Pid Directory = /var/run
  Maximum Concurrent Jobs = 20
  PKI Signatures = Yes# Enable Data Signing
  PKI Encryption = Yes# Enable Data Encryption
  PKI Keypair = "/Library/Preferences/bacula/Toto.pem"    # Public and
Private Keys for this client
  PKI Master Key = "/Library/Preferences/bacula/master.cert"  # ONLY
the Public Key of the master Keypair
}

------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand 
malware threats, the impact they can have on your business, and how you 
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users