Re: [Bacula-users] Concurrent Backups with a Virtual Autochanger

Hi Mike,

Despite that the white paper tell us about having "Maximum Concurrent Jobs = 1" in device configuration. I think this could make sense when using stand alone devices in a group as you have in the white papers. When using autochangers, it is much better you use the "Maximum Concurrent Jobs" in storage definitions, so Bacula will use the second drive when the first one is in use. Maybe this way you will not have the problem of the second job having to wait for a drive.

Now I was wondering if you have just the IncPool pool? For all your jobs and for full, diff and inc backups? I would suggest you having them separately for admin purposes. As your number of clients and backups increase, maybe it could become a little difficult to manage a so large and diversified pool.

Bacula is a very very flexible backup tool and will let you do configurations that will not make sense at the end, but it will allow you do it.

Another point here is about "RemovableMedia = no;". The white papers set it to "yes". I'm not so sure if this directive is considered in a virtual autochanger configuration. I had seen it set to "no" or "yes" interchangeably. Maybe this could not tamper with your problem, but for me it makes more sense to use it set to "yes" like it is in the white papers.

Do you have "prefer mounted volumes" set to no in your jobs definition? It is recommended if you are using multiple devices and one pool.

Best regards,

Ana

On Fri, Nov 14, 2014 at 8:17 PM, Brady, Mike <mike.brady AT devnull.net DOT nz> wrote:

First of all thanks to Kern and Bacula Systems for making the "Best
Practices for Disk Based Backup" and "Disk Back Design" documents
available.

I have been playing around with the best way for doing concurrent
backups for a while and these documents have helped my understanding
considerably. Using a Virtual Autochanger in particular seems an
elegant way of doing what I would like to do.

However, I am seeing some behaviour in my testing that I did not expect
and I need some input.

At a high level what I am trying to do is use a Virtual Autochanger to
write to multiple volumes in the same pool concurrently.

At the moment I have two devices limited to one concurrent job each.
Which, if I have understood things correctly, means that I should have
two jobs running concurrently writing to separate volumes. The schedule
below kicks off eight jobs simultaneously with the number of devices
limiting concurrency.

This issue that I am having is that the first job gets FileChgr1-Dev1
and a volume as expected.

The second job gets device FileChgr1-Dev2 as expected, but always says
"Cannot find any appendable volumes." and issues a mount request. There
are multiple purged volumes with the recycle flag set available in the
IncPoool pool. Even if there weren't, the pool has Auto Labelling
configured and has not reached the MaximumVolumes limit, so there should
"always" be a volume available.

Other jobs continue to use the FileChgr1-Dev1 as it becomes available
while FileChgr1-Dev2 is waiting for a volume.

The second job eventually retries on FileChgr1-Dev2, gets an available
volume and successfully completes without any operator intervention.

After this the remaining jobs utilise both FileChgr1-Dev1 and
FileChgr1-Dev2 as they become available as I expected.

Is this behaviour expected (I am assuming some sort of race condition at
the start of the schedule with multiple jobs trying to get a volume at
the same time) or am I trying to do something fundamentally wrong here?

My configuration is:

Pool {
Name = IncPool
Pool Type = Backup
Volume Use Duration = 23 hours
Recycle = yes
Action On Purge = Truncate
Auto Prune = yes
Maximum Volumes = 50
Volume Retention = 2 weeks
Storage = FileStorage01
Next Pool = "IncPoolCopy"
Label Format = "IncPool-"
}

Storage {
Name = FileStorage01
Address = 192.168.42.45
SDPort = 9103
Password = ***************************
Device = FileChgr1
Media Type = File01
Maximum Concurrent Jobs = 10
Autochanger = yes
}

Autochanger {
Name = FileChgr1
Device = FileChgr1-Dev1, FileChgr1-Dev2
Changer Command = /dev/null # For 7.0.0 and newer releases.
# Changer Command = "" # For 5.2 and older releases.
Changer Device = /dev/null
}

Device {
Name = FileChgr1-Dev1
Drive Index = 0
Media Type = File01
Archive Device = /bacula_storage/FileDevice
LabelMedia = yes;
Random Access = Yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = no;
Maximum Concurrent Jobs = 1
VolumePollInterval = 5s
Autochanger = yes
}

Device {
Name = FileChgr1-Dev2
Drive Index = 1
MediaType = File01
ArchiveDevice = /bacula_storage/FileDevice
LabelMedia = yes;
RandomAccess = Yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = no;
MaximumConcurrent Jobs = 1
VolumePollInterval = 5s
Autochanger = yes
}

Schedule {
Name = "DefaultBackupCycle"
Run = Level=Full 1st sun at 00:10
Run = Level=Differential 2nd-5th sun at 00:10
Run = Level=Incremental mon-sat at 00:10
}

Thanks

Mike

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users