Bacula-users

Re: [Bacula-users] Need help with blocking factor and speeds

2008-08-12 18:47:47
Subject: Re: [Bacula-users] Need help with blocking factor and speeds
From: "T. Horsnell" <tsh AT mrc-lmb.cam.ac DOT uk>
To: bacula-users AT lists.sourceforge DOT net
Date: Wed, 13 Aug 2008 00:00:36 +0100
Mingus Dew wrote:
> All,
>      I apologize if this is a rehash of previous threads. I've tried 
> searching the archives for an answer, but am not fully understanding 
> what this means for me.
> 
> I am running Bacula 2.4.2 on Solaris 10_x86. I am using an Exabyte 
> Magnum 224 tape library with IBM Ultrium LTO-3 tape drive. I currently 
> have Bacula configured for variable block sizes.
> I have been unable to determine or set a default blocking factor or 
> compression on my tape unit. I do however believe that this unit accepts 
> the configuration of the application. I don't know if Bacula, other than 
> by using the /dev/rmt/0cbn device purposefully sets compression. I am 
> under the impression however that Bacula is using its default block size 
> of 65k.
> 
> I have been having problems with tape backups taking much longer than I 
> believe they should. I have not yet had to restore from tape and have 
> only done so in testing. In production I have a couple of jobs with 
> these properties
> 
> Job1 - 334G to tape - Avg 26.20 Mbytes/sec
> Job2 - 1.6T to tape - Avg 13.74 Mbytes/sec
> 
> Since I've not figured out if Bacula will tell me the write speed to 
> tape for the job, I've just used the start and end times along with the 
> amount of data to calculate the speed. This does not take into account 
> the time required to spool data. I've turned off data spooling for Job2 
> in order to compare after this week's run and see what the speed might be.
> 
> Testing with tar has shown that using the 65k block size is definitely a 
> performance bottleneck. I used real data (not empty or mkfile generated) 
> for my testing to accurately represent what I usually put to tape 
> (Oracle RMAN backup files).
> 
> Here are the numbers:
> 
> block size 65 Kbytes
> Test #1 (backup) tar > rmt/0cbn - 85 Mb/s
> Test #2 (restore) tar > rmt/0cbn - 42 Mb/s
> 
> This is similar, though still faster to what I see for Bacula jobs. I 
> think the difference is in the data spooling.
> 
> Block size 1024 Kbytes
> Test #1 tar > rmt/0cbn - 410 Mb/s
> Test #2 tar > rmt/0cbn - 409 Mb/s
> 
> Block size 2048 Kbytes
> Test #1 tar > rmt/0cbn - 410 Mb/s
> Test #2 tar > rmt/0cbn - 408 Mb/s
> 
> With these other tests and a much larger blocking factor I see greatly 
> improved performance.
> 
> I can tell that I need to change the blocking factor for bacula. 
> However, I'm very confused by the documentation's description of doing 
> so and the consequences. I'd like to know how I should setup my blocking 
> factor so that Bacula uses the tape drive most efficiently and also if I 
> change the blocking factor, how will it affect jobs I currently have on 
> tape or labeled volumes?
> 

Aside from the Bacula stuff, I'm interested in those 410Mb/sec data 
rates you have listed above. According to IBM's website, the max data 
rate for their LTO-3 drive is 120MBytes/sec. What exactly are you 
tarring to achieve 410Mbtyes/sec? (assuming your 410Mb/sec means 
Mbytes/sec). If you are tarring a file full of zeros, this will be 
compressing extremely well and will be giving you misleading transfer 
rates. I get no better than 105Mbytes/sec on an LTO-4 with tar, however 
big I make the blocksize.

Bacula has given me no better than 80MBytes/sec on to this LTO-4 drive 
with a blocksize of 256K, and doesnt improve on this with bigger block 
sizes.

Cheers,
Terry


> Thank you,
> Shon
> 
> 
> ------------------------------------------------------------------------
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users