Bacula-users

Re: [Bacula-users] low despooling speeds when parallelism is increased

2010-06-02 06:15:16
Subject: Re: [Bacula-users] low despooling speeds when parallelism is increased
From: Ralf Gross <Ralf-Lists AT ralfgross DOT de>
To: bacula-users AT lists.sourceforge DOT net
Date: Wed, 2 Jun 2010 12:11:57 +0200
Athanasios Douitsis schrieb:
> 
> Hi everyone,
> 
> Our setup consists of a Dell 2950 server (PERC6i) w/ FreeBSD7 and two HP
> Ultrium LTO4 drives installed in a twin Quantum autochanger enclosure. 
> Our bacula version is 5.0.0 (which is the current FreeBSD port version). 
> 
> Here is our problem:
> 
> When running a single job, our setup is able to consistently surpass
> 70Mbytes/sec (or even 80) on despooling, which should be reasonably
> enough.  Unfortunately, when running several jobs on both drives (for
> example 6+6 parallel jobs) our despooling speeds drop to about
> 20Mbytes/sec or even less. The speed of the jobs to finish last
> naturally ramp up, especially for the very last. Our hypothesis is that
> the spooling area cannot handle simultaneous reading (from jobs that are
> still tranfering) and writing (from the currently despooling job) too
> well, hence the performance loss. 
> 
> So far we were using a common spool area for both drives on these two
> test setups:
> 
> 1)A spool area on a Clarion CX4 Fibre Channel array (4Gbps) w/ 2x10Krpm
> disks on a raid0 configuration. 
> 2)A 2xSCSI320 raid0 striped configuration in the server itself (via the
> PERC6i controller).
> 
> Both setups yielded similarly poor results.
> 
> Our thoughts/questions:
> 
> -Should we use a separate spool area for each drive? 
> -Anyone else that has had problems with the despooling speed being too
> low? What are your proposed solutions, if any?
> 
> I realize this is not strictly a bacula question, however the matter
> should be of interest for any bacula admin out there. I understand that
> under like 40Mbytes/sec the drive constantly starts and stops, a
> process which  is detrimental to its expected lifetime (and the tape's
> as well). 

I've had the same problem. First I started with a simple 2 Disk RAID1
as spool area for our 3 LTO-4 drives. With more jobs running in
parallel this was simply not enought. In the end I put 6 WD Raptor
SATA drives in the server, configured a large RAID10 and limited the
number of concurrent jobs to 3. Now I get ~100 MB/s for each of the 3
jobs.

If you really need to run so many jobs concurrently put as many fast
disks in the server as possible and configure them as RAID10. Maybe
use SSD's instead of SATA/SAS Disk?

Ralf

------------------------------------------------------------------------------

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users