On 21/07/10, Craig Miskell (craig.miskell AT opus.co DOT nz) wrote:
> Rory Campbell-Lange wrote:
> >On 19/07/10, Rory Campbell-Lange (rory AT campbell-lange DOT net) wrote:
> > > I ran a full backup of a volume last week on Bacula v2.4.4 and I
> > > have now upgraded to 5.0.2. I'm getting half the throughput last
> > > week as I am now.
> > > 
> > > Last week's test volume was a 2.1TB xfs volume and it read/wrote
> > > on average at 70MB/s. I am now backing up a 4.4TB ext3 volume and
> > > bconsole's status client shows backup running at less than 30MB/s.
> > > Hardware compression has been enabled in both cases.
> > > 
> > > Both volumes are RAID5 Coraid (ATA over Ethernet) devices. I'd be
> > > grateful for any pointers for getting to the bottom of this.
> > 
> > With reference to http://www.mail-archive.com/bacula-users AT 
> > lists.sourceforge DOT net/msg36340.html
> > I wonder if I should configure the following settings:
> > 
> >   Maximum File Size = 5G
> >   Maximum Spool Size = 370g
> >   Maximum Job Spool Size = 20g
> >   Maximum Block Size = 262144
> >   Maximum Network Buffer Size = 262144
> > 
> > for faster throughput.  I haven't set any of these parameters at the moment.
> 
> I can vouch for Maximum File Size = 5G at least; it's a totally
> safe/backwards compatible change to make, and doing that took my
> backups from 38MB/s to 46Mb/s (~20% improvement).  And I can give
> some tentative support to Maximum Block Size = 262144; in some small
> scale tests yesterday (on a small number of large files only), I
> went from 47MB/s to 64Mb/s (~25%).  I haven't yet done full-scale
> tests though.
> 
> Be aware that it seems probable that if you change Maximum Block
> Size on a device, you may (probably?) need to change that back to
> the default on the device if you wish to restore from backups done
> with the default block size.
> 
> The other settings (Spool size and Network Buffer Size) will help in
> getting your data onto the backup server and ensuring there's enough
> data for your tape to chew on, and are good.  Actual sizing for the
> Spool settings will depend on how big your backups are; I don't have
> enough experience (or data) to comment much.

Thanks very much for the comments, Craig. I'm leaving the majority of
the previous emails above because others may find it germane to anyone
else who may wish to comment on this debate. I'll definitely set the
Maximum File Size as you suggest.

Following your comments, Craig. From other comments on the list about
Maximum Block Size it appears that this could be a dangerous change to
make. In our case we may not be the party retrieving the data and we
have to make that process as easy as possible. It would be great to have
a comment on the safety of using Maximum Block Size from others on this
list.

Regarding spooling, I re-read the entry here:
http://www.bacula.org/manuals/en/concepts/concepts/Data_Spooling.html
Spooling specifically is used to avoid shoe-shining, so in my case it
looks like a good idea because the tape seems to be waiting for data.
The iotop tool shows quite big differences in read IO from the target
volume (i.e 100MB/s one moment, 18MB/s the next.

If I've got a 2.2TB backup writing to LTO4 tapes at just over 1TB/tape
using hardware compression I wonder what the approprate spool sizes
would be? I've only got a 300G internal volume available for spooling.

Spool Size = 250GB                 # maximum spool size for this job
Maximum Spool Size = 250GB         # only one job at a time
Maximum Job Spool Size = 250000000 # 250GB in bytes
Spool Directory = /tapespool       # XFS filesystem preferred?

I'd be grateful on any comments about spool sizes and whether this is
appropriate for my situtation with locally attached storage and and LTO4
tape library. The central issue is that the locally attached storage can
be read at well over 150MB/s and I'm only getting to tape at under
40MB/s.

Regards
Rory

-- 
Rory Campbell-Lange
rory AT campbell-lange DOT net

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users