Alex Chekholko wrote:
> On Thu, 14 Aug 2008 09:57:07 +0100
> James Cort <james.cort AT u4eatech DOT com> wrote:
>
>   
>> <snip lots of stuff about block sizes>
>>
>> If performance is an issue, I can strongly recommend ensuring that 
>> neither database nor disk seek time forms a bottleneck.
>>
>> I am also using an LTO drive and I find that backups of lots of small 
>> files are routinely 3-5 times slower than backups of a few very large 
>> files.  I've got a maximum blocksize set of 16MB which does help.
>>
>>     
> Hi James,
>
> Can you please elaborate on this?  What setting do you have set where?  Also, 
> have you been able to successfully run the btape tests with that block size?  
> Which version of bacula?
>
> Regards,
>   

This is a live system which is used to backup around 800GB of data daily 
with regular test restores. I've set the blocksize to be the largest 
supported by the tape drive (at least according to what the st driver 
reports in /var/log/messages):

Aug 13 06:34:02 gemini kernel: st0: Block limits 1 - 16777215 bytes.

I've been using a virtually identical configuration for about 18 months 
- I'm now using 2.2.8 but I'm pretty certain it'll work with earlier 
versions - I would have run btest when I first started but I haven't 
done so lately because I prefer real world tests which prove I can 
usefully restore data ;)

It helps performance, sure, but I find that the absolute killer is seek 
time - when you're backing up thousands of small files, this will make a 
hell of a dent in the performance you can achieve, both because you've 
got to write lots more records to the database and because the disk is 
having to seek all over the place to read the files.   I find that when 
thousands of small files are involved, a 6-disk RAID 1+0 array using the 
latest 15kRPM SAS drives cannot give the kind of performance you need 
from the catalogue or the source media (even when source media and 
database are on separate servers) to max out an LTO-3 drive's 
performance.  This is about at the limit of what I can afford, so I 
haven't been able to try anything bigger.  If anyone can max out an LTO3 
drive's performance while backing up thousands of small files from such 
a RAID, I'd love to know how.

I have in the past set it up to spool data direct to tape and leave the 
catalogue writing until after the data is laid to tape.  I found that 
this didn't help the overall backup time much and did tend to peg the 
server so hard it wasn't very responsive for much else while it was 
updating the catalogue - bit of a problem as I haven't got sufficient 
hardware to dedicate a server purely to backups.


In bacula-sd.conf

Device {
  # Name and media type can be anything you like, but bacula expects two
  # devices using the same media type to be able to use each others' tapes.
  Name = LTO3
  Media Type = LTO-Tape
  Drive Index = 0
  Archive Device = /dev/nst0
  AutomaticMount = yes              # when device opened, read it
  AlwaysOpen = yes
  RemovableMedia = yes
  RandomAccess = no
  SpoolDirectory = /spool
  Maximum Spool Size = 4G
  Maximum Block Size = 16777214
  Maximum Network Buffer Size = 65536
  AutoChanger = yes
}


-- 
James Cort

IT Manager
U4EA Technologies Ltd.


-- 
U4EA Technologies
http://www.u4eatech.com


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users