Bacula-users

Re: [Bacula-users] Running Multiple Jobs at once

2012-08-02 14:21:12
Subject: Re: [Bacula-users] Running Multiple Jobs at once
From: dweimer <dweimer AT dweimer DOT net>
To: <bacula-users AT lists.sourceforge DOT net>
Date: Thu, 02 Aug 2012 13:18:24 -0500
On 2012-08-02 09:32, John Drescher wrote:
> On Thu, Aug 2, 2012 at 9:39 AM, dweimer <dweimer AT dweimer DOT net> wrote:
>> I have been doing some searching, but I am apparently not searching 
>> on
>> the right key words, and can't seem to find out how to enable 
>> multiple
>> jobs to run simultaneously.  I am running my backups to Disk only 
>> and
>> not tape, so I do have random access media, and also knowing that
>> running multiple jobs to the volume may slow down restores.  The 
>> problem
>> that I am trying to solve is a slow link to one of the machines, it
>> takes a several hours to backup, so I need to start it early, but 
>> don't
>> want to make the remaining server have to wait for it to complete 
>> before
>> executing their backups.  Some of the faster machines are fairly 
>> large
>> volume and may take up to an hour so I would rather not just run 
>> them
>> first delaying the slow machines start for a couple of hours.
>>
>> I know there are some Max Concurrent Jobs settings, not sure if I am
>> just missing one of those or if I need to add more storage 
>> devices/pools
>> possibly a spool area, etc.  I have made sure to configure the
>> priorities correctly (since in my searching I have found that it 
>> will
>> only run same priority jobs at same time, which makes sense) so the 
>> main
>> backup jobs are one priority, a few less critical servers another, 
>> and
>> then the catalogue backup set to run last.
>>
>> If anyone knows of some documentation that describes all the steps 
>> is
>> necessary to allow multiple jobs at once, and can direct me to it, 
>> that
>> would be great.
>
> The manual does describe all of the steps necessary although maybe 
> not
> in one single place.
>
> At minimum you need to set "Max Concurrent Jobs" for the director,
> storage and possibly client resource of bacula-dir.conf. Also 
> remember
> that a single storage device (even a disk) can only load 1 volume at 
> a
> time. So that if you have multiple jobs going to different volumes 
> you
> can not use a single storage device. Also you need to have all jobs 
> at
> the same priority if you want to run them concurrently. If jobs are 
> at
> different priorities they will preempt jobs at lower priorities.
>
> John

 From the Tips and tricks section of the problem resolution guide, I 
have set the director, storage, client, and job resources all to use Max 
concurrent Jobs = 4, I have also set the device under the bacula-sd.conf 
to have a Max Concurrent Jobs = 4, as I found that was a new feature in 
the 5.0 branch from the documentation.  The file daemons and storage 
daemon, were already set to 20 max concurrent jobs. I don't believe that 
I need the entries under Job and Client, as I only intend to run one job 
per client and job, but I figured for testing I would start with them in 
there.  The jobs I am testing with are both set to the same priority.  I 
have reloaded the director, and restarted the storage daemon to get the 
new configurations read.
I then fired off 2 jobs, and no luck the second job reports waiting 
execution, and executes as soon as the first job finishes.  Is there any 
way to get a more detailed report of why the job is waiting execution, 
to see if I can find out what I am missing from that.
I know that in the long run it would be better to have separate devices 
and pool definitions, but as it appears to be technically possible 
without doing this, I would at least like to prove it will work before I 
add even more complexity to the configuration, of course if its not 
possible I don't have a choice, but as the volumes are file based on 
disk I know I will take a hit in restore performance by writing 2 jobs 
at a time to the volume, but at this point that’s an acceptable down 
side.  If I was writing to tape, I would think differently about that 
performance hit, as it would be much greater if that were the case.

-- 
Thanks,
    Dean E. Weimer
    http://www.dweimer.net/

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users