Hi
I am using Bacula inside an Cent OS openVZ container on a
virtualization cluster. I tried with and without compression and/or
encryption and the rates are very various.
I am backing up a KVM client on the same nodes of the cluster and an
external mac client.
I can have rates of more than 30 or 40 MBytes/sec down to some hundred
of KBytes/sec
Here are some stats :
Backing up the mac client :
09-déc 10:48 bacula-dir JobId 42: Bacula bacula-dir 5.0.3 (30Aug10):
09-déc-2010 10:48:42
Build OS: i686-redhat-linux-gnu redhat
JobId: 42
Job: BackupStationMac.2010-12-08_18.04.32_07
Backup Level: Full
Client: "StationMac" 5.0.3 (04Aug10)
i386-apple-darwin10.4.0,osx,10.4.0
FileSet: "Test" 2010-12-08 17:53:46
Pool: "Test" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "DataTest" (From Pool resource)
Scheduled time: 08-déc-2010 18:04:28
Start time: 08-déc-2010 18:04:34
End time: 09-déc-2010 10:48:42
Elapsed time: 16 hours 44 mins 8 secs
Priority: 10
FD Files Written: 636,312
SD Files Written: 636,312
FD Bytes Written: 170,577,925,504 (170.5 GB)
SD Bytes Written: 170,856,886,548 (170.8 GB)
Rate: 2831.3 KB/s
Software Compression: 25.7 %
VSS: no
Encryption: yes
Accurate: yes
Volume name(s): => About 40 4GB Volumes
Volume Session Id: 11
Volume Session Time: 1291809192
Last Volume Bytes: 4,007,665,932 (4.007 GB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
Backing up the KVM :
I ran many test on the KVM client with various parameter and kind of files:
1) GZIP3 - Encryption - 1Go file from /dev/urandom
=> Rate about 7828.8 KB/s
2) GZIP3 - Encryption - 1Go file from /dev/zero
=> Rate: 113.1 KB/s
=> Software Compression: 99.5 %
=> Most of the time was to compress the file
3) NO GZIP3 - Encryption - 1Go file from /dev/urandom
=> Rate about 19785.7 KB/s
4) Lots of file of various size --> Typical base install of Cent
OS and some apps about 54 000 files
GZIP3 & Encryption => 901.2 KB/s
No GZIP3 & No Encryption => 31517.3 KB/s
I also tried some copy job from hard drive to LTO3 device :
=> I am about 7 MBytes/sec I think it's correct
On which aspects can I improve theses perfs ?
I already a list of files that are not to compress :
wildfile = ".rar"
wildfile = ".zip"
wildfile = ".gz"
.......
Or to exclude
wilddir = ".TemporaryItems"
wilddir = ".Spotlight-V100"
.......
In addition, I am looking for something to don't compress tiny files.
Is this possible ?
Also which compilation options can improve my rates that are horrible
on small files. I used the default config of my RPMs and maybe I
omitted something at this moment.
In order to continue bacula implementation into a testing - pre-prod
phase I must reach at least 2 or 3 MB/sec rates for every Jobs. I want
to validate that kind of stats
Can you compare with your stats ?
The Mac machine is a Mac Mini from the last Power PC Generation
The cluster node is a 8 x Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
Virtual machines ( OpenVZ : CentOS Bacula server fd+sd+dir+DB and a
KVM : CentOS Bacula fd ) have a 50/50 repartition of ressources and it
never been fully loaded.
What should I do, in order to have a better compromise between
Compression and the Transfert Rate.
Thanks
Hugo
------------------------------------------------------------------------------
Lotusphere 2011
Register now for Lotusphere 2011 and learn how
to connect the dots, take your collaborative environment
to the next level, and enter the era of Social Business.
http://p.sf.net/sfu/lotusphere-d2d
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
