Bacula-users

Re: [Bacula-users] Copy Job performance issues

2010-12-07 04:54:48
Subject: Re: [Bacula-users] Copy Job performance issues
From: Christoph Litauer <litauer AT uni-koblenz DOT de>
To: bacula-users <bacula-users AT lists.sourceforge DOT net>
Date: Tue, 7 Dec 2010 10:51:10 +0100
Am 07.12.2010 um 02:53 schrieb Dan Langille:

> On 12/6/2010 11:13 AM, Christoph Litauer wrote:
>> Dear bacula users,
>> 
>> I have a problem concerning the speed of copy jobs. My setup is:
>> - bacula server is OpenSuSE 11.3, version 5.0.3 using postgres, 2 Xeons (4 
>> cores) and 8 GB memory.
>> - Attached is an iSCSI-RAID containing File devices
>> - Copy Jobs run to a Quantum Scalar 50 Tapelibrary with 2 LTO4 tape drives. 
>> The library and the tapes are connected via eSATA. According to wikipedia, 
>> LTO4 should transfer 120MB/s without compression. Compression is enabled.
>> 
>> My backups go to the iSCSI-RAID. I run CopyJObs every day to copy all 
>> uncopied jobs to tape.
>> My problem is, that my copy jobs run with max. 45 Mbyte/s
>> 
>> I checked the following:
>> - btape shows a maximum speed of 80MByte/sec.
>> - I can read about 150 Mbyte/s from the RAID device
>> - Copying data from RAID to tape using dd I get rates about 80 MByte/s
>> - While copying the cpu load is about 20-50% (bacula-sd)
>> 
>> So it seems as if bacula-sd itself slows the copy jobs down, but I cannot 
>> imagine, why. Maybe it is a configuration issue, but reading the manuals 
>> didn't help.
>> 
>> Configuration of the tape device in bacula-sd.conf:
>> 
>> Autochanger {
>>   Name = "Scalar50-2"
>>   Device = "LTO4-0"
>>   Device = "LTO4-1"
>>   Changer Device = /dev/tape/by-id/scsi-3500e09e0bb562001
>>   Changer Command = "/usr/lib64/bacula/mtx-changer %c %o %S %a %d"
>> }
>> 
>> Device {
>>   Name = "LTO4-0"
>>   Device Type = Tape;
>>   Media Type = LTO4
>>   Archive Device = /dev/tape/by-id/scsi-3500110a00104374e-nst
>>   AutomaticMount = yes;
>>   AlwaysOpen = yes;
>>   RemovableMedia = yes;
>>   RandomAccess = no;
>>   Autochanger = yes
>>   Drive Index = 0
>>   Spool Directory = /var/spool/bacula
>> }
>> 
>> Device {
>>   Name = "LTO4-1"
>>   Device Type = Tape;
>>   Media Type = LTO4
>>   Archive Device = /dev/tape/by-id/scsi-3500110a0012be1ee-nst
>>   AutomaticMount = yes;
>>   AlwaysOpen = yes;
>>   RemovableMedia = yes;
>>   RandomAccess = no;
>>   Autochanger = yes
>>   Drive Index = 1
>>   Spool Directory = /var/spool/bacula
>> }
>> 
>> Any ideas?
> 
> I'm guessing.
> 
> A copy job does a great deal of disk access.  It had to read all the 
> data from the Volume.  It has to read the original Catalog entries.  It 
> has to write the new Catalog entries.
> 
> 

Hi Dan,

thanks for your suggestions. I am just doing a very large copy job (about 600 
GB):
- CPU load is 35-50% used by bacula-sd
- iostat gives me about 40-50 MB/s reading performance on my iSCSI-Device 
(which is able to read about 150MB/s).
- postmaster takes about 2% CPU load
- iowait is 0 to 0.2%

I can see no bottleneck at all ... but "status storage" still reports 40MB/s 
write performance.

--
Kind regards
Christoph
_________________________________________
Christoph Litauer
Uni Koblenz, Computing Centre, Office A 022    
Postfach 201602, 56016 Koblenz     
Fon: +49 261 287-1311, Fax: -100 1311





------------------------------------------------------------------------------
What happens now with your Lotus Notes apps - do you make another costly 
upgrade, or settle for being marooned without product support? Time to move
off Lotus Notes and onto the cloud with Force.com, apps are easier to build,
use, and manage than apps on traditional platforms. Sign up for the Lotus 
Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users