Bacula-users

Re: [Bacula-users] Need help with blocking factor and speeds

2008-08-13 10:29:46
Subject: Re: [Bacula-users] Need help with blocking factor and speeds
From: "Mingus Dew" <shon.stephens AT gmail DOT com>
To: "T. Horsnell" <tsh AT mrc-lmb.cam.ac DOT uk>
Date: Wed, 13 Aug 2008 10:29:05 -0400
Its completely possible that my math is wrong. I am terrible at math. I also was a little surprised by the numbers.

On Tue, Aug 12, 2008 at 7:00 PM, T. Horsnell <tsh AT mrc-lmb.cam.ac DOT uk> wrote:
Mingus Dew wrote:
> All,
>      I apologize if this is a rehash of previous threads. I've tried
> searching the archives for an answer, but am not fully understanding
> what this means for me.
>
> I am running Bacula 2.4.2 on Solaris 10_x86. I am using an Exabyte
> Magnum 224 tape library with IBM Ultrium LTO-3 tape drive. I currently
> have Bacula configured for variable block sizes.
> I have been unable to determine or set a default blocking factor or
> compression on my tape unit. I do however believe that this unit accepts
> the configuration of the application. I don't know if Bacula, other than
> by using the /dev/rmt/0cbn device purposefully sets compression. I am
> under the impression however that Bacula is using its default block size
> of 65k.
>
> I have been having problems with tape backups taking much longer than I
> believe they should. I have not yet had to restore from tape and have
> only done so in testing. In production I have a couple of jobs with
> these properties
>
> Job1 - 334G to tape - Avg 26.20 Mbytes/sec
> Job2 - 1.6T to tape - Avg 13.74 Mbytes/sec
>
> Since I've not figured out if Bacula will tell me the write speed to
> tape for the job, I've just used the start and end times along with the
> amount of data to calculate the speed. This does not take into account
> the time required to spool data. I've turned off data spooling for Job2
> in order to compare after this week's run and see what the speed might be.
>
> Testing with tar has shown that using the 65k block size is definitely a
> performance bottleneck. I used real data (not empty or mkfile generated)
> for my testing to accurately represent what I usually put to tape
> (Oracle RMAN backup files).
>
> Here are the numbers:
>
> block size 65 Kbytes
> Test #1 (backup) tar > rmt/0cbn - 85 Mb/s
> Test #2 (restore) tar > rmt/0cbn - 42 Mb/s
>
> This is similar, though still faster to what I see for Bacula jobs. I
> think the difference is in the data spooling.
>
> Block size 1024 Kbytes
> Test #1 tar > rmt/0cbn - 410 Mb/s
> Test #2 tar > rmt/0cbn - 409 Mb/s
>
> Block size 2048 Kbytes
> Test #1 tar > rmt/0cbn - 410 Mb/s
> Test #2 tar > rmt/0cbn - 408 Mb/s
>
> With these other tests and a much larger blocking factor I see greatly
> improved performance.
>
> I can tell that I need to change the blocking factor for bacula.
> However, I'm very confused by the documentation's description of doing
> so and the consequences. I'd like to know how I should setup my blocking
> factor so that Bacula uses the tape drive most efficiently and also if I
> change the blocking factor, how will it affect jobs I currently have on
> tape or labeled volumes?
>

Aside from the Bacula stuff, I'm interested in those 410Mb/sec data
rates you have listed above. According to IBM's website, the max data
rate for their LTO-3 drive is 120MBytes/sec. What exactly are you
tarring to achieve 410Mbtyes/sec? (assuming your 410Mb/sec means
Mbytes/sec). If you are tarring a file full of zeros, this will be
compressing extremely well and will be giving you misleading transfer
rates. I get no better than 105Mbytes/sec on an LTO-4 with tar, however
big I make the blocksize.

Bacula has given me no better than 80MBytes/sec on to this LTO-4 drive
with a blocksize of 256K, and doesnt improve on this with bigger block
sizes.

Cheers,
Terry


> Thank you,
> Shon
>
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url="">
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Bacula-users mailing list
>
Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url="">
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users