Bacula-users

[Bacula-users] Cannot get concurrent jobs to run, all other jobs are waiting on max Storage jobs

2014-05-14 10:41:29
Subject: [Bacula-users] Cannot get concurrent jobs to run, all other jobs are waiting on max Storage jobs
From: Robert-Jean Denault <Robert-jean.Denault AT telus DOT com>
To: "bacula-users AT lists.sourceforge DOT net" <bacula-users AT lists.sourceforge DOT net>
Date: Wed, 14 May 2014 10:36:58 -0400

I am running bacula Version: 5.0.0 (26 January 2010) x86_64-redhat-linux-gnu redhat on CentOS with a single LTO3 Dell PowerVault 124T tape unit.

 

I have configured environment to perform spooling / despooling of backup jobs. I wanted to have the bacula server so that file daemons would run concurrently to spool the files to disk, and to despoil the backup to tape once they are complete.  When the backup starts multiple jobs are in running state, but only one is running all the other jobs are waiting on max Storage jobs.

 

================

Result of a status dir

================

 

Running Jobs:

Console connected at 14-May-14 08:57

JobId Level   Name                       Status

======================================================================

  3578 Full    Default-RHEL-test.2014-05-14_08.57.29_28 is running

  3579 Full    Default-RHEL-test.2014-05-14_08.57.38_29 is waiting on max Storage jobs

====

 

 

 

Here is my configuration; Ihave added a shortened list of clients and jobs:

 

==================

Bacula-dir.conf:

==================

 

Director {                            # define myself

  Name = dapbauu3-dir

  DIRport = 9101                # where we listen for UA connections

  QueryFile = "/opt/bacula/etc/query.sql"

  WorkingDirectory = "/export/BACKUP/bacula/work"

  PidDirectory = "/var/run"

  Maximum Concurrent Jobs = 4

  Password = "SOME_PASSWORD"         # Console password

  Messages = Daemon

}

#

#########File that contains all the jobdefs

@/opt/bacula/etc/bacula-JobDefs.conf

 

#

# file that contains the jobs

@/opt/bacula/etc/bacula-Jobs.conf

 

 

#

##########File that contains all the Schedules

@/opt/bacula/etc/bacula-Schedule.conf

 

##############File that contains all the filesets

@/opt/bacula/etc/bacula-Fileset.conf

 

##############files that contains all clients

@/opt/bacula/etc/bacula-Clients.SunDev.conf

@/opt/bacula/etc/bacula-Clients.SunQA.conf  

@/opt/bacula/etc/bacula-Clients.WinDev.conf 

@/opt/bacula/etc/bacula-Clients.WinQA.conf

@/opt/bacula/etc/bacula-Clients.RHELDev.conf

@/opt/bacula/etc/bacula-Clients.RHELQA.conf

 

# Generic catalog service

Catalog {

  Name = MyCatalog

  dbname = bacula

  user = bacula

  password = SOME_PASSWORD

  DB Address = localhost

}

 

 

 

# Definition of Storage

@/opt/bacula/etc/bacula-Storage.conf

 

# Definition of Messages

@/opt/bacula/etc/bacula-Messages.conf

 

# Definition of all pools by a file

@/opt/bacula/etc/bacula-Pools.conf

 

==================

Bacula-fd.conf

==================

 

#

# List Directors who are permitted to contact this File daemon

#

Director {

  Name = dapbauu3-dir

  Password = "SOMEPASSWORD"

}

 

Director {

  Name = dbacduu4-dir

  Password = "SOMEPASSWORD"

}

 

#

# "Global" File daemon configuration specifications

#

FileDaemon {                          # this is me

  Name = dapbauu3-fd

  FDport = 9102                  # where we listen for the director

  WorkingDirectory = /var/spool/bacula

  Pid Directory = /var/run

  Maximum Concurrent Jobs = 20

 

  PKI Signatures = Yes            # Enable Data Signing

  PKI Encryption = Yes            # Enable Data Encryption

  PKI Keypair = "/opt/bacula/certs/dapbauu3-fd.pem"    # Public and Private Keys

  PKI Master Key = "/opt/bacula/certs/dbacduu4-fd.cert"    # ONLY the Public Key

 

}

 

# Send all messages except skipped files back to Director

Messages {

  Name = Standard

  director = dbacduu4-dir = all, !skipped, !restored

}

 

==================

Bacula-sd.conf

==================

 

 

Storage {                             # definition of myself

  Name = dapbauu3-sd

  SDPort = 9103                  # Director's port     

  WorkingDirectory = "/export/BACKUP/bacula/work"

  Pid Directory = "/var/run"

  Maximum Concurrent Jobs = 20

}

 

#

# List Directors who are permitted to contact Storage daemon

#

Director {

  Name = dapbauu3-dir

  Password = "SOME_PASSWORD"

}

 

#

# Note, for a list of additional Device templates please

#  see the directory <bacula-source>/examples/devices

# Or follow the following link:

#  http://bacula.svn.sourceforge.net/viewvc/bacula/trunk/bacula/examples/devices/

#

 

#

# Devices supported by this Storage daemon

# To connect, the Director's bacula-dir.conf must have the

#  same Name and MediaType.

#

 

Device {

  Name = FileStorage

  Media Type = File

  Archive Device = /export/BACKUP/bacula/fileStorage

  LabelMedia = yes;                   # lets Bacula label unlabeled media

  Random Access = Yes;

  AutomaticMount = yes;               # when device opened, read it

  RemovableMedia = no;

  AlwaysOpen = no;

}

 

 

#

# A Linux or Solaris LTO-4 tape drive

#

Device {

  Name = LTO

  Device Type = Tape

  Media Type = lto

  Archive Device = /dev/nst0

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  AutoChanger = yes

# Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c ' /usr/sbin/tapeinfo -f %c |grep TapeAlert|cat'"

# If you have smartctl, enable this, it has more info than tapeinfo

# Alert Command = "sh -c '/usr/sbin/smartctl --health --log=error --device=scsi %c'" 

 #

# Max Spool Size = 1TB

#

Maximum Spool Size = 1099511627776

Spool Directory = /export/BACKUP/bacula/spool

Maximum Concurrent Jobs = 4

}

 

#

# An autochanger device with two drives

#

Autochanger {

  Name = PV-124T

  Device = LTO

  Changer Command = "/opt/bacula/etc/mtx-changer %c %o %S %a %d"

  Changer Device = /dev/changer

}

 

#

# Send all messages to the Director,

# mount messages also are sent to the email address

#

Messages {

  Name = Standard

  director = dapbauu3-dir = all

}

 

 

==================

Bacula-Pools.conf

==================

 

Pool {

  Name = Default

  Pool Type = Backup

  Recycle = yes                       # Bacula can automatically recycle Volumes

  AutoPrune = yes                     # Prune expired volumes

  Volume Retention = 365 days         # one year

  Volume Use Duration = 325 hours

  Cleaning Prefix = "CLN"

  Maximum Concurrent Jobs = 4

}

 

Pool {

  Name = TEMPO

  Pool Type = Backup

  Recycle = yes                       # Bacula can automatically recycle Volumes

  AutoPrune = yes                     # Prune expired volumes

  Volume Retention = 365 days         # one year

  Volume Use Duration = 325 hours

  Cleaning Prefix = "CLN"

  Maximum Concurrent Jobs = 4

}

 

==================

Bacula-Schedule.conf

==================

 

Schedule {

  Name = "Coredev-Full-Inc"

  Run = Level=Full fri at 17:30

  Run = Level=Incremental mon-thu at 19:00

  SpoolData = yes

}

 

 

==================

Bacula-Storage.conf

==================

 

Storage {

  Name = File

# Do not use "localhost" here   

  Address = dapbauu3                # N.B. Use a fully qualified name here

  SDPort = 9103

  Password = "SOME_PASSWORD"

  Device = FileStorage

  Media Type = File

}

 

 

# Definition of LTO tape storage device

Storage {

  Name = LTO-STORAGE

  #Do not use "localhost" here

  Address = dapbauu3                # N.B. Use a fully qualified name here

  SDPort = 9103

  Password = "SOME_PASSWORD"          # password for Storage daemon

  Device =  PV-124T                    # must be same as Device in Storage daemon

  Media Type = lto                # must be same as MediaType in Storage daemon

  Autochanger = yes                   # enable for autochanger device

}

 

=========================== 

Bacula-Clients.RHELDev.conf

===========================

 

Client {

  Name = DAPGWUU1-fd

  Address = 10.10.10.219

  FDPort = 9102

  Catalog = MyCatalog

  Password = "SOME_PASSWORD"          # password for FileDaemon

  File Retention = 62 days   # 62 days

  Job Retention = 365 days            # 365 days

  AutoPrune = yes                     # Prune expired Jobs/Files

}

 

#############

# add client DAPGWUU2

#############

 

Client {

  Name = DAPGWUU2-fd

  Address = 10.10.10.222

  FDPort = 9102

  Catalog = MyCatalog

  Password = "SOME_PASSWORD"          # password for FileDaemon

  File Retention = 62 days   # 62 days

  Job Retention = 365 days            # 365 days

  AutoPrune = yes                     # Prune expired Jobs/Files

}

 

#############

# add client DAPGWUU3

#############

 

Client {

  Name = DAPGWUU3-fd

  Address = 10.10.10.223

  FDPort = 9102

  Catalog = MyCatalog

  Password = "SOME_PASSWORD"          # password for FileDaemon

  File Retention = 62 days   # 62 days

  Job Retention = 365 days            # 365 days

  AutoPrune = yes                     # Prune expired Jobs/Files

}

 

#############

# add client DAPBAUU3

#############

 

Client {

  Name = DAPBAUU3-fd

  Address = 10.10.10.252

  FDPort = 9102

  Catalog = MyCatalog

  Password = "SOME_PASSWORD"          # password for FileDaemon

  File Retention = 62 days   # 62 days

  Job Retention = 365 days            # 365 days

  AutoPrune = yes                     # Prune expired Jobs/Files

}

 

=====================

Bacula-Jobs.conf

=====================

 

Job {

  Name = "Default-RHEL-test"

  FileSet = "RHEL_root_etc_for_test"

  Type = Backup

  Level = Full

  Client = DAPGWUU1-fd

  Messages = Standard

  Storage = LTO-STORAGE

  Pool = Default

  Write Bootstrap = "/var/bacula/defaultRHEL_test.bsr"

  Spool Data = "">

  Spool Attributes = yes

  Priority = 10

  Maximum Concurrent Jobs = 4

}

 

Job {

  Name = JS-DAPGWUU1

  Enabled = yes

  Type = backup

  Bootstrap = /var/bacula/DAPGWUU1.bsr

  Client = DAPGWUU1-fd

  FileSet = RHEL-Root-OPT2

  Pool = Default

  Schedule = Coredev-Full-Inc

  Storage = LTO-STORAGE

  Prune Jobs = yes

  Prune Files = yes

  Prune Volumes = yes

  Spool Data = "">

  Spool Attributes = yes

  Priority = 10

  Messages = Standard

  Maximum Concurrent Jobs = 4

}

 

#######Job for client $X

Job {

  Name = JS-DAPGWUU2

  Enabled = yes

  Type = backup

  Bootstrap = /var/bacula/DAPGWUU2.bsr

  Client = DAPGWUU2-fd

  FileSet = RHEL-Root-OPT2

  Pool = Default

  Schedule = Coredev-Full-Inc

  Storage = LTO-STORAGE

  Prune Jobs = yes

  Prune Files = yes

  Prune Volumes = yes

  Spool Data = "">

  Spool Attributes = yes

  Priority = 10

  Messages = Standard

  Maximum Concurrent Jobs = 4

}

 

#######Job for client $X

Job {

  Name = JS-DAPGWUU3

  Enabled = yes

  Type = backup

  Bootstrap = /var/bacula/DAPGWUU3.bsr

  Client = DAPGWUU3-fd

  FileSet = RHEL-Root-OPT2

  Pool = Default

  Schedule = Coredev-Full-Inc

  Storage = LTO-STORAGE

  Prune Jobs = yes

  Prune Files = yes

  Prune Volumes = yes

  Spool Data = "">

  Spool Attributes = yes

  Priority = 10

  Messages = Standard

  Maximum Concurrent Jobs = 4

}

 

Should you have any questions or concerns please do not hesitate to contact me

Robert Denault
Programmeur-analyste principal II/ Senior Programmer-Analyst II

TELUS Santé et Solutions de paiement
TELUS Health and Payment Solutions
T : (450) 928 6000 x 3740
T : 1 (866) 363 7447 x 3740
robert.denault AT telus DOT com 
telussante.com  telushealth.com

 

http://emergisweb/emergis/communications/promo/THS_bbE_Fre_RGB.gif

Le présent courriel et tout document joint sont destinés uniquement à la personne ou à l’entité à qui il est adressé et peut contenir des renseignements confidentiels. Si vous n’êtes pas le destinataire visé, veuillez nous en informer sans délai et détruire le courriel et les documents ainsi que toute copie qui en aurait été faite. Merci

This message and any accompanying attachments are intended only for the person(s) to whom this message is addressed and may contain privileged, proprietary and/or confidential information. Any unauthorized use, disclosure, reproduction or distribution of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately and permanently delete or destroy this message, including any attachments. Thank you

 

------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
<Prev in Thread] Current Thread [Next in Thread>