John Drescher wrote:

> I made a try with this results
>>
>> JobId 7: Spooling data ...
>> JobId 7: Job write elapsed time = 00:13:07, Transfer rate = 1.295 M 
>> Bytes/second
>> JobId 7: Committing spooled data to Volume "FullVolume-0004". Despooling 
>> 1,021,072,888 bytes ...
>> JobId 7: Despooling elapsed time = 00:02:19, Transfer rate = 7.345 M 
>> Bytes/second
>> JobId 7: Sending spooled attrs to the Director. Despooling 6,816,214 bytes 
>> ...
>>
>> There is only a little improvement of performances.

Spooling or not spooling is very much a case of "it depends".

There is little point in using spooling if the destination device is a 
disk drive and you are not using concurrent jobs.(*)

For a NFS mounted destination network delays are your biggest problem.

Make sure rsize and wsize are the maximum possible (32768 bytes on 
NFSv3, several Mb on NFSv4)

If you have 1Gb/s ethernet, use jumbo frames.

7-8Mb/s is about right for untuned NFS on a 100Mb/s network environment. 
Because of NFS overheads you won't go much faster even though the 
network theoretical maximum is 12.5Mb/s

(*) If you are using concurrent jobs and writing output to disk, 
spooling will usually help maintain speeds, because writing multiple 
files to the target disk will result in slowdowns due to head seeking. A 
single job despooling is more akin to streaming throughput.

>> So I'm asking you what is the transfer rate in your jobs? These rates are 
>> normal or are they slow compared to yours?
>>
> 
> I use gigabit and LTO2 so I am seeing 20 to 50 MB/s per tape drive in
> my 2 drive LTO2 autochanger.

Spooling is almost always worthwhile if the target is tape.

Spooling for differential/incremental jobs won't make much difference to 
the overall job rate but it WILL stop the tape drive "shoeshining" as it 
tries to write small amounts of data. That saves a lot of wear-and-tear.

Spooling for full jobs is a win as long as your spool disks are fast 
enough to keep up with the tape drive (or at least, faster than what's 
coming off the backup filesystem).

Spooling is _essential_ if you're running concurrent jobs, otherwise the 
jobs get interleaved on tape and restores will run incredibly slowly 
because the drive has to seek back and forth to get data in 64kB chunks.

Concurrent jobs slow down the effective read rate when despooling to 
tape due to head seeking penalties.

A single 7200rpm 320Gb HDD can _just_ despool fast enough to LTO2 to 
keep its buffers full whilst having another job simultaneously spooling 
data.

A single 7200rpm 320Gb drive _cannot_ despool fast enough to LTO3 (or 
higher levels), even when despooling is the only activity on the disk.

A 4 disk 7200rpm RAID0 stripe can just keep up with LTO4 whilst spooling 
another job.

I use a set of Intel X25E SSDs in RAID-0 layout for the spool area. They 
are slightly faster in streaming output than 7200rpm drives, but the 
head seek penalty reduces from avg ~8ms down to under 0.1ms. I can 
concurrently despool to 3 LTO5 drives whilst spooling 3 more jobs.





------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT data 
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users