Bacula-users

Re: [Bacula-users] Backup of many many files

2008-09-04 16:13:17
Subject: Re: [Bacula-users] Backup of many many files
From: Arno Lehmann <al AT its-lehmann DOT de>
To: bacula-users AT lists.sourceforge DOT net
Date: Thu, 04 Sep 2008 22:12:53 +0200
Hi,

04.09.2008 18:33, Hemant Shah wrote:
> 
> --- On Wed, 9/3/08, Michael Galloway <mgx AT ornl DOT gov> wrote:
> 
>> From: Michael Galloway <mgx AT ornl DOT gov>
>> Subject: Re: [Bacula-users] Backup of many many files
>> To: "Ryan Novosielski" <novosirj AT umdnj DOT edu>
>> Cc: bacula-users AT lists.sourceforge DOT net
>> Date: Wednesday, September 3, 2008, 11:03 AM
>> On Wed, Sep 03, 2008 at 11:59:49AM -0400, Ryan Novosielski
>> wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> Chris Howells wrote:
>>>> Michael Galloway wrote:
>>>>
>>>>> how does concurency work for tape libraries?
>> can i use spooling and concurrency?
>>>>> would that help? i've already got
>> spooling job (2GB segments) and attributes.
>>>> You can use concurrency with tape, yes. I write
>> five concurrent jobs to 
>>>> LTO-4. (I have an autochanger, but it's only
>> got a single LTO4 tape 
>>>> drive in it).
>>>>
>>>> The bacula manual warns somewhat against using
>> concurrent jobs with 
>>>> tape, but I've been using it for a year,
>> every restore has been 100% 
>>>> perfect, and the restore speed is absolutely fine
>> for my needs, it's 
>>>> pretty quick anyway.
>>>>
>>>> I'm not sure about spooling as I don't
>> use it but I understand it can 
>>>> help to increase restore speed, but as I said, I
>> don't have a problem 
>>>> with restore speed anyway.
>>> You should use spooling. I'm not 100% sure that
>> interleaving should even
>>> be allowed, as I've heard of some people having
>> strange things happening
>>> with it enabled. Also takes life off your tape head,
>> if I'm not mistaken.
>> are there any recommendations on the size of the spool
>> segment?
> 
>   It depends on how much disk space you have, they are cheap nowadays.
> 
>   I run five concurrent jobs and spool complete job before writing it to the 
> tape. My largest backup client backs up 127 GB. I have setup 500GB filesystem 
> for spool so bacula is able to spool five job simultaneously before writing 
> them to tape.
> 
>   Bacula will spool five jobs at same time but write one job at a time to the 
> tape.

I usually recommend the following:
- If possible, allocate spool space so that all jobs that run 
concurrently fit into it.
- If that's not possible, you can tune spool space use per job; try to 
find settings that allow the smaller jobs to spool completely, and 
leave more than a few GB for the remaining jobs.
- Otherwise, allocate as much spool space as possible, but limit the 
size each job can use so that later-starting jobs don't run with very 
limited spool space.

If you get things tweaked, the overall backup time should be 
considerably less than the sum of the time for all the jobs. If things 
  go wrong you end up with a situation where a small number of larger 
and slower jobs can block most of the spool space and others jobs, 
which could run much quicker compete for a few remaining MB.

Remember that spooling will make each single job run longer, but they 
can overlap so the overall time for all jobs can be shorter.

Arno

>> -- michael
>>  
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move
>> Developer's challenge
>> Build the coolest Linux based applications with Moblin SDK
>> & win great prizes
>> Grand prize is a trip for two to an Open Source event
>> anywhere in the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>> Bacula-users mailing list
>> Bacula-users AT lists.sourceforge DOT net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 
> 
> 
> Hemant Shah
> E-mail: hjrrs AT yahoo DOT com
> 
> 
> 
>       
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

-- 
Arno Lehmann
IT-Service Lehmann
Sandstr. 6, 49080 Osnabrück
www.its-lehmann.de

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users