Bacula-users

Re: [Bacula-users] Concurrent Backups with a Virtual Autochanger

2014-11-18 14:56:36
Subject: Re: [Bacula-users] Concurrent Backups with a Virtual Autochanger
From: Ana Emília M. Arruda <emiliaarruda AT gmail DOT com>
To: "Brady, Mike" <mike.brady AT devnull.net DOT nz>
Date: Tue, 18 Nov 2014 16:52:59 -0300

On Tue, Nov 18, 2014 at 4:47 PM, Brady, Mike <mike.brady AT devnull.net DOT nz> wrote:
On 2014-11-16 05:36, Ana Emília M. Arruda wrote:
Hi Mike,

Despite that the white paper tell us about having "Maximum Concurrent
Jobs = 1" in device configuration. I think this could make sense when
using stand alone devices in a group as you have in the white papers.
When using autochangers, it is much better you use the "Maximum
Concurrent Jobs" in storage definitions, so Bacula will use the second
drive when the first one is in use. Maybe this way you will not have
the problem of the second job having to wait for a drive.

I do not understand what you mean.

What do the white papers say about having "Maximum Concurrent Jobs = 1"?  I can not find anything.

I do have Maximum Concurrent Jobs set in the Storage definition.

​I saw it too in your device configuration:

Device {
   Name = FileChgr1-Dev1
   Drive Index = 0
   Media Type = File01
   Archive Device = /bacula_storage/FileDevice
   LabelMedia = yes;
   Random Access = Yes;
   AutomaticMount = yes;
   RemovableMedia = no;
   AlwaysOpen = no;
   Maximum Concurrent Jobs = 1
   VolumePollInterval = 5s
   Autochanger = yes
}​



And the job is not waiting for a device.  The second device can not find an available volume.  But this only happens when a schedule first starts. Once the second device tries to get a volume again after a 5 minute timeout everything works as expected.  That is, in this case two devices each with a different volume mounted from the same pool with one job writing to each device at a time.

As I replied to Josh a short while ago, I do not believe that the volume selection and auto creation/labeling operations are thread safe. There may well be configuration options that reduce/work around the issues, but I do not currently have the time to do the necessary testing, so I am no longer pursuing this.


Now I was wondering if you have just the IncPool pool? For all your
jobs and for full, diff and inc backups? I would suggest you having
them separately for admin purposes. As your number of clients and
backups increase, maybe it could become a little difficult to manage a
so large and diversified pool.

I have Inc, diff and full pools.  The diff and full pools just aren't relevant to this issue.

Bacula is a very very flexible backup tool and will let you do
configurations that will not make sense at the end, but it will allow
you do it.
So I have found many times.


Another point here is about "RemovableMedia = no;". The white papers
set it to "yes". I'm not so sure if this directive is considered in a
virtual autochanger configuration. I had seen it set to "no" or "yes"
interchangeably. Maybe this could not tamper with your problem, but
for me it makes more sense to use it set to "yes" like it is in the
white papers.


For Autochanger devices the "Best Practices for Disk Based Backup" and "Disk Backup Design" white papers have RemovableMedia set to yes in one of them and no in the other (can't remember which way around it is).  This may well work around the race condition for volume selection by having the device search for a volume when the "Can not find an available volume" condition occurs.

I do not have a lot of volumes so this may well work for me, but if there are thousands of volumes in a device directory then this may impact performance significantly.

Do you have "prefer mounted volumes" set to no in your jobs
definition? It is recommended if you are using multiple devices and
one pool.

I do not have this set because of the warnings against it in both the manual and one of the white papers.

​But this is recommended when using multiple devices (autochanger or stand alone) and one pool.​




Thanks for your suggestions.


​You´re welcome.
Best regards,
Ana​

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users