Bacula-users

Re: [Bacula-users] bacula can't make use of multi drives

2015-02-11 19:18:27
Subject: Re: [Bacula-users] bacula can't make use of multi drives
From: "Kelley, Jared" <jkelley AT popcap DOT com>
To: "Kelley, Jared" <jkelley AT popcap DOT com>, Süleyman Kuran <skuran AT ayk.gov DOT tr>, "Ana Emília M. Arruda" <emiliaarruda AT gmail DOT com>
Date: Wed, 11 Feb 2015 23:27:47 +0000
bumping this thread because I have another issue I am unable to resolve.
I am running:
bacula 7.0.5 compiled from source
 3 pools.
Multiple jobs per pool.
10 tape drives
1 autochanger for those 10 drives. 
1600 slots

running concurrent jobs works.
I have 41 volumes(tapes)  in the virtual changer with 10 drives.
Bacula is getting confused very quickly on which volume is in which slot when comparing whats in the bacula database, 
as shown by select query, compared to the output of mtx-changer —list all command. See below.
So as these volumes get more and more out of sync my jobs get hung with waiting to mount volume which cascades to exceeding max jobs
as the running jobs start to queue.   I’m wondering if this is a bug due to the number of slots.
The fix is to release all 10 drives, run update slots.  But I am having to do this daily now.  Unacceptable.
When I was setting this up I started with only 10 virtual tapes or volumes and things ran smoothly.
Now I have added a 31 more tapes to  prevent jobs from failing, if all the tapes in a pool become full, and the problem has been exacerbated by 
the increasing the number volumes.
It seems bacula is unable to handle this many slots and as it moves a volume back to the slot from whence it came to pull another for a different job
and it puts the tape(volume) back into a different slot but gets confused and Slot in the database is now wrong for this particular volume.
Does anyone have any experience with this?  And is there a possible configuration fix?  
My max concurrent jobs is 20 for SD and max concurrent jobs per tapedrive  is 1.  This to keep jobs from interleaving data when they are writing concurrently.
I will be happy to submit configuration files if that is helpful.
Thanks

Volumes in red are the volumes bacula is now out of sync with.  Every one of these volumes is currently in a tape-drive.  I have edited the output below for simplicity in comparing mtx-changer command vs mysql select query.

Select VolumeName,Slot from Media order by slot; slot out put from mtx changer --list all
AAAC236886 1   AAAC236886                      
AAAACC5F69 2 AAAACC5F69                      
AAAACE5F6B 3 AAAACE5F6B                      
AAAAD55F70 4 AAAAD55F70                      
AAAB1763B2 5 AAAB1763B2                      
AAAACA5F6F 6 AAAACA5F6F                      
AAAACD5F68 7 AAAACD5F68                      
AAAB0B63AE 8 AAAB0B63AE                      
AAAB1463B1 9 AAAB1463B1                      
AAAACF5F6A 10 AAAACF5F6A                      
AAAAC95F6C 11 AAAAC95F6C                      
AAAB0963AC 12 AAAB0963AC                      
AAAAC85F6D 13 AAAAC85F6D                      
AAAAD45F71 14 AAAAD45F71                      
AAAC2D6888 15 AAAC226887                      
AAACDA687F 16 AAAC216884                      
AAAB0863AD 17 AAAB0863AD                      
AAAC256880 18 AAAC256880                      
AAAB0F63AA 19 AAAB0F63AA                      
AAAC226887 20 AAADBB691E                      
AAAC206885 21 AAAC206885                      
AAAB1563B0 22 AAAB1563B0                      
AAAB1663B3 23 AAAB1663B3                      
AAAB3B609E 24 AAACDA687F                      
AAAB0E63AB 25 AAAB0E63AB                      
AAADB06915 26 AAADB06915                      
AAAB0A63AF 27 AAAB0A63AF                      
AAAC216884 28 AAAB3B609E                      
AAAACB5F6E 29 AAAACB5F6E                      
AAAC246881 30 AAAC246881                      
AAAC276882 31 AAAC276882                      
AAAC266883 32 AAAC2D6888                      
AAADB9691C 33 AAADB9691C                      
AAADBD6918 34 AAADBD6918                      
AAADB8691D 35 AAADB8691D                      
AAADB36916 36 AAADB36916                      
AAADBE691B 37 AAADBE691B                      
AAADBB691E 38 AAAC266883                      
AAADBF691A 39 AAADBF691A                      
AAADBC6919 40 AAADBC6919                      
AAADB26917 41 AAADB26917 

From: <Kelley>, "Kelley, Jared" <jkelley AT popcap DOT com>
Date: Wednesday, December 17, 2014 at 12:36 PM
To: Süleyman Kuran <skuran AT ayk.gov DOT tr>, "\"Ana Emília M. Arruda\"" <emiliaarruda AT gmail DOT com>
Cc: "Bacula-users AT lists.sourceforge DOT net" <bacula-users AT lists.sourceforge DOT net>
Subject: Re: [Bacula-users] bacula can't make use of multi drives

OK, I was able to get this working.
One of the major factors was upgrading my version of bacula from 5.2.6 to 7.0.5.
It appears there is a bug in 5.2.6 in which bacula gets confused with regards to which tape(volume) is in which drive and/or slot.
with 5.2.6, with my config files working, after backups ran for a few days the slots no longer matched in bacula with what mtx-changer listall 
told me from command line and I was getting lots of errors and backups hanging.

After upgrading to 7.0.5 I no longer had this problem and my backups are running smoothly on 10 different drives using 3 different pools.
All is good for me here using multi drive configuration.

Thanks to Ana for extensive help to get this working.


Jared 


From: Süleyman Kuran <skuran AT ayk.gov DOT tr>
Date: Tuesday, November 25, 2014 at 11:15 PM
To: "\"Ana Emília M. Arruda\"" <emiliaarruda AT gmail DOT com>
Cc: "Bacula-users AT lists.sourceforge DOT net" <bacula-users AT lists.sourceforge DOT net>
Subject: Re: [Bacula-users] bacula can't make use of multi drives

Thanks for your reply.

My autochanger device definition in bacula-sd.conf is as follows. I removed unrelated changer commands in Drive resource directives as you suggested, maybe those direcives mess up bacula, I will see what happens.

Autochanger {
  Name = AYKAutochanger
  Device = Drive-1
  Device = Drive-2
  Device = Drive-3
  Device = Drive-4
  Changer Command = "/usr/libexec/bacula/mtx-changer %c %o %S %a %d"
  Changer Device = /dev/sg9
}

Best regards,


On 25-11-2014 18:25, Ana Emília M. Arruda wrote:
Hello,

Do you have an autochanger device definition in your bacula-sd.conf?

Autochanger {
Name = "AYKAutochanger"
Device = Drive-1, Drive-2, Drive-3, Drive-4
Changer Device = /dev/sg9
Changer Command = "/usr/libexec/bacula/mtx-changer %c %o %S %a %d"
}

In your Drives definitions, you don´t need to have Changer Device nor Changer Command defined. The bellow lines should be removed from your drives definitions:

 Changer Command = "/usr/libexec/bacula/mtx-changer %c %o %S %a %d"
#  Changer Device = /dev/changer
  Changer Device = /dev/sg9
  AutoChanger = yes

Best regards,
Ana

On Tue, Nov 25, 2014 at 5:46 AM, Süleyman Kuran <skuran AT ayk.gov DOT tr> wrote:
I still can't find the final key to my puzzle :)

Jobs (from the same pool)  are still waiting the first drive to finish. I followed Ana's advise and set Max Concurrent Jobs = 1 in the Device directive in bacula-sd.conf and other configs in bacula-dir.conf  The following are related parts of config files. Please advice.

Best regards,

bacula-sd.conf
=================================================
...
Storage {                             # definition of myself
  Name = bacula-sd
  SDPort = 9103                  # Director's port     
  WorkingDirectory = "/var/spool/bacula"
  Pid Directory = "/var/run"
  Maximum Concurrent Jobs = 20
}
Device {
  Name = Drive-1
  Drive Index = 0
  Media Type = LTO-5
  Archive Device = /dev/nst0
  AutomaticMount = yes
  AlwaysOpen = yes
  RemovableMedia = yes
  Random Access = no
  Maximum File Size = 10GB
  Changer Command = "/usr/libexec/bacula/mtx-changer %c %o %S %a %d"
#  Changer Device = /dev/changer
  Changer Device = /dev/sg9
  AutoChanger = yes
##  # Enable the Alert command only if you have the mtx package loaded
  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
### If you have smartctl, enable this, it has more info than tapeinfo
  #Alert Command = "sh -c 'smartctl -H -l error %c'" 
  Maximum Concurrent Jobs = 1
}
Device {
  Name = Drive-2
  Drive Index = 1
  Media Type = LTO-5
  Archive Device = /dev/nst1
  AutomaticMount = yes
  AlwaysOpen = yes
  RemovableMedia = yes
  Random Access = no
  Maximum File Size = 10GB
  Changer Command = "/usr/libexec/bacula/mtx-changer %c %o %S %a %d"
#  Changer Device = /dev/changer
  Changer Device = /dev/sg9
  AutoChanger = yes
  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
  Maximum Concurrent Jobs = 1
}
Device {
  Name = Drive-3
  Drive Index = 2
  Media Type = LTO-5
  Archive Device = /dev/nst2
  AutomaticMount = yes
  AlwaysOpen = yes
  RemovableMedia = yes
  Random Access = no
  Maximum File Size = 10GB
  Changer Command = "/usr/libexec/bacula/mtx-changer %c %o %S %a %d"
#  Changer Device = /dev/changer
  Changer Device = /dev/sg9
  AutoChanger = yes
  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
  Maximum Concurrent Jobs = 1
}
Device {
  Name = Drive-4
  Drive Index = 3
  Media Type = LTO-5
  Archive Device = /dev/nst3
  AutomaticMount = yes
  AlwaysOpen = yes
  RemovableMedia = yes
  Random Access = no
  Maximum File Size = 10GB
  Changer Command = "/usr/libexec/bacula/mtx-changer %c %o %S %a %d"
#  Changer Device = /dev/changer
  Changer Device = /dev/sg9
  AutoChanger = yes
  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
  Maximum Concurrent Jobs = 1
}
...
==================================================================
bacula-dir.conf
...
Storage {
  Name = "Fujitsu ETERNUS LT60"
  Address = fqdn.bacula.tld                # N.B. Use a fully qualified name here
  SDPort = 9103
  Password = "mypass"          # password for Storage daemon
  Device = AYKAutochanger
  Media Type = LTO-5                  # must be same as MediaType in Storage daemon
  Autochanger = yes                   # enable for autochanger device
  Maximum Concurrent Jobs = 20
}
Director {                            # define myself
  Name = bacula-dir
  DIRport = 9101                # where we listen for UA connections
  #QueryFile = "/usr/libexec/bacula/query.sql"
  QueryFile = "/etc/bacula/query.sql"
  WorkingDirectory = "/var/spool/bacula"
  PidDirectory = "/var/run"
  Maximum Concurrent Jobs = 20
  Password = "mypasswd"         # Console password
  Messages = Daemon
}
Client {
  Name = bacula-fd
  Address = localhost
  FDPort = 9102
  Catalog = MyCatalog
  Password = "mypasswd"          # password for FileDaemon
  Maximum Concurrent Jobs = 20
}




On 21-11-2014 01:27, Kelley, Jared wrote:
Yep, that was the final key to the puzzle.  SUCCESS!!
Thank you so much for your help with this.
I’m running test jobs now with the new setting and I’m 4 jobs from same pool are writing to 
4 different drives and volumes concurrently.
Again, your help is much appreciated.

Thank you

Jared

From: "Ana Emília M. Arruda" <emiliaarruda AT gmail DOT com>
Date: Thursday, November 20, 2014 at 1:39 PM
To: "Kelley, Jared" <jkelley AT popcap DOT com>
Cc: bacula-users <bacula-users AT lists.sourceforge DOT net>
Subject: Re: [Bacula-users] bacula can't make use of multi drives

Hi Jared,

Try setting 
Maximum Concurrent Jobs = 1
in each device definition. This will spread your 4 jobs in 4 drives of your autochanger device.

Device {

  Name = Drive0 

  Drive Index = 0

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-IPiscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-01-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  


Maximum Concurrent Jobs = 1

}


Best regards,
Ana


On Thu, Nov 20, 2014 at 6:33 PM, Kelley, Jared <jkelley AT popcap DOT com> wrote:
Ok, I’ve done some reconfiguring.   Will paste my new configuration files below as well as bconsole output from show pools, list volumes, status storage and querying volumes per job.  I am running 4 concurrent jobs to the same pool and the writes are still interleaving data to the same volume.
I stopped and started bacula-sd,  dir and fd, ran update slots scan before kicking off these jobs.  Status storage shows the new device names and things are working, aside from the single tape being written to by all 4 jobs.

Still not sure where I’m going wrong with bacula and writing to multiple volumes at the same time from the same pool.  Currently the only way I’ve achieved this with mixed results, i.e. success and failures, is with ‘prefer mounted volumes = no.
:\

############# BEGIN BACULA-SD.CONF  ############

SEA-12176-MAC:backup02_bacula jkelley$ cat bacula-sd.conf 

#

# Default Bacula Storage Daemon Configuration file

#

#  For Bacula release 3.0.1 (30 April 2009) -- debian 5.0.1

#

# You may need to change the name of your tape drive

#   on the "Archive Device" directive in the Device

#   resource.  If you change the Name and/or the 

#   "Media Type" in the Device resource, please ensure

#   that dird.conf has corresponding changes.

#


Storage {                             # definition of myself

  Name = backup02-sd

  SDPort = 9103                  # Director's port      

  WorkingDirectory = "/etc/bacula/working"

  Pid Directory = "/var/run/bacula"

  Maximum Concurrent Jobs = 20

}



# List Directors who are permitted to contact Storage daemon

#

Director {

  Name = backup02-dir

  Password = password

}



## Restricted Director, used by tray-monitor to get the

#   status of the storage daemon

##

Director {

  Name = backup02-mon

  Password = password

  Monitor = yes

}


# Devices supported by this Storage daemon

# To connect, the Director's bacula-dir.conf must have the

#  same Name and MediaType. 

#


Autochanger {

  Name = Autochanger

  Device = Drive0, Drive1, Drive2, Drive3, Drive4, Drive5, Drive6, Drive7, Drive8, Drive9

  Changer Command = "/etc/bacula/scripts/mtx-changer %c %o %S %a %d"

  Changer Device = /dev/sg12

}



​ ​
Device {

  Name = Drive0 

  Drive Index = 0

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-IPiscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-01-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

 

​ ​
Maximum Concurrent Jobs = 20 

}


Device {

  Name = Drive1

  Drive Index = 1

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-IPiscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-02-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

}


Device {

  Name = Drive2

  Drive Index = 2

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-IPiscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-03-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

}


Device {

  Name = Drive3

  Drive Index = 3

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-IPiscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-04-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

}


Device {

  Name = Drive4

  Drive Index = 4

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-IPiscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-05-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

}


Device {

  Name = Drive5

  Drive Index = 5

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-IPiscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-06-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

}


Device {

  Name = Drive6

  Drive Index = 6

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-IPiscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-07-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

}


Device {

  Name = Drive7

  Drive Index = 7

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-IPiscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-08-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

}


Device {

  Name = Drive8

  Drive Index = 8

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-IPiscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-09-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

}


Device {

  Name = Drive9

  Drive Index = 9

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-IPiscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-10-lun-0-nst

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

}



# Send all messages except skipped files back to Director

Messages {

  Name = Standard

  director = backup02-dir = all, !skipped, !restored

}

 

####### END SD.CONF #####


###### BEGIN DIR.CONF ####

# Default Bacula Director Configuration file

#

#  The only thing that MUST be changed is to add one or more

#   file or directory names in the Include directive of the

#   FileSet resource.

#

#  For Bacula release 3.0.1 (30 April 2009) -- debian 5.0.1

#

#  You might also want to change the default email address

#   from root to your address.  See the "mail" and "operator"

#   directives in the Messages resource.

#


###

### Source other config files

###


@/etc/bacula/clients.conf

@/etc/bacula/jobs.conf


###

### Basic director configuration

###


Director {                            # define myself

  Name = backup-dir

  DIRport = 9101                # where we listen for UA connections

  QueryFile = "/etc/bacula/query.sql"

  WorkingDirectory = "/etc/bacula/working"

  PidDirectory = "/var/run/bacula"

  Maximum Concurrent Jobs = 20 

#  Password = password

  Password = password

  Messages = Daemon

  DIRAddress = localhost

}



###

### Storage

###



# Storage device for tape

Storage {

  Name = Autochanger

  Address = backup.               # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = password

  Device = Autochanger

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}



#Storage {

###

###

### Storage pools

###

# Default storage pool

Pool {

  Name = Default

  Pool Type = Backup

  Storage = Autochanger

  Recycle = no                       # Bacula can automatically recycle Volumes

  AutoPrune = yes                     # Prune expired volumes

  Volume Retention = 365 days         # one year

  Maximum Volume Bytes = 2500G

}


###

# social service storage pool

Pool {

  Name = svc

  Pool Type = Backup

  Storage = Autochanger 

  Recycle = no                       # Bacula can automatically recycle Volumes

  AutoPrune = yes                     # Prune expired volumes

  Volume Retention = 365 days         # one year

  Maximum Volume Bytes = 2500G

}


# database service storage pool

Pool {

  Name = database 

  Pool Type = Backup

  Storage = Autochanger

  Recycle = no                       # Bacula can automatically recycle Volumes

  AutoPrune = yes                     # Prune expired volumes

  Volume Retention = 365 days         # one year

  Maximum Volume Bytes = 2500G

}




###

### Backup catalog

###


# Generic catalog service

Catalog {

  Name = MyCatalog

# Uncomment the following line if you want the dbi driver

# dbdriver = "dbi:mysql"; dbaddress = 127.0.0.1; dbport =  

  dbname = "bacula"; dbuser = ""; dbpassword = "password"

}



###

### Logging

###


# Reasonable message delivery -- send most everything to email address

#  and to the console

Messages {

  Name = Standard

#

# NOTE! If you send to two email or more email addresses, you will need

#  to replace the %r in the from field (-f part) with a single valid

#  email address in both the mailcommand and the operatorcommand.

#  What this does is, it sets the email address that emails would display

#  in the FROM field, which is by default the same email as they're being

#  sent to.  However, if you send email to more than one address, then

#  you'll have to set the FROM address manually, to a single address. 

#  for example, a 'no-reply AT mydomain DOT com', is better since that tends to

#  tell (most) people that its coming from an automated source.


#

  mailcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula: %t %e of %c %l\" %r"

  operatorcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula: Intervention needed for %j\" %r"

  mail on error = whodat AT wedat DOT com = all, !skipped            

  operator = whodat AT wedat DOT com= mount

  console = all, !skipped, !saved

#

# WARNING! the following will create a file that you must cycle from

#          time to time as it will grow indefinitely. However, it will

#          also keep all your messages if they scroll off the console.

#

  append = "/etc/bacula/working/log" = all, !skipped


  catalog = all, !skipped, !saved

}



#

# Message delivery for daemon messages (no job).

Messages {

  Name = Daemon

  mailcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula daemon message\" %r"

  mail = @localhost = all, !skipped            

  console = all, !skipped, !saved

  append = "/etc/bacula/working/log" = all, !skipped

}



###

### Console

###


# Restricted console used by tray-monitor to get the status of the director

Console {

  Name = backup-mon

  Password = password

  CommandACL = status, .status

}



###### END DIR.CONF ####

#### SHOW POOLS OUTPUT ####

Pool: name=Default PoolType=Backup

      use_cat=1 use_once=0 cat_files=1

      max_vols=0 auto_prune=1 VolRetention=1 year 

      VolUse=0 secs recycle=0 LabelFormat=*None*

      CleaningPrefix=*None* LabelType=0

      RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0

      MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=2684354560000

      MigTime=0 secs MigHiBytes=0 MigLoBytes=0

      JobRetention=0 secs FileRetention=0 secs

  --> Storage: name=Autochanger address=backup02.wes.popcap.com SDport=9103 MaxJobs=20

      DeviceName=Autochanger MediaType=LTO StorageId=13

Pool: name=svc PoolType=Backup

      use_cat=1 use_once=0 cat_files=1

      max_vols=0 auto_prune=1 VolRetention=1 year 

      VolUse=0 secs recycle=0 LabelFormat=*None*

      CleaningPrefix=*None* LabelType=0

      RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0

      MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=2684354560000

      MigTime=0 secs MigHiBytes=0 MigLoBytes=0

      JobRetention=0 secs FileRetention=0 secs

  --> Storage: name=Autochanger address=backup02.wes.popcap.com SDport=9103 MaxJobs=20

      DeviceName=Autochanger MediaType=LTO StorageId=13

Pool: name=database PoolType=Backup

      use_cat=1 use_once=0 cat_files=1

      max_vols=0 auto_prune=1 VolRetention=1 year 

      VolUse=0 secs recycle=0 LabelFormat=*None*

      CleaningPrefix=*None* LabelType=0

      RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0

      MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=2684354560000

      MigTime=0 secs MigHiBytes=0 MigLoBytes=0

      JobRetention=0 secs FileRetention=0 secs

  --> Storage: name=Autochanger address=backup02.wes.popcap.com SDport=9103 MaxJobs=20

      DeviceName=Autochanger MediaType=LTO StorageId=13



#### OUTPUT FROM QUERY option 15 #####

    15: List Volumes Bacula thinks are in changer

    16: List Volumes likely to need replacement from age or errors

Choose a query (1-16): 15

+---------+------------+-------------+-------------+------+----------+-----------+-----------+

| MediaId | VolumeName | GB          | Storage     | Slot | Pool     | MediaType | VolStatus |

+---------+------------+-------------+-------------+------+----------+-----------+-----------+

|      19 | AAAACE5F6B | 255.5893    | Autochanger |    1 | svc      | LTO       | Append    |

|      26 | AAAACC5F69 | 1657.5554   | Autochanger |    2 | svc      | LTO       | Append    |

|      20 | AAAACF5F6A | 1509.3844   | Autochanger |    3 | database | LTO       | Append    |

|      25 | AAAAD45F71 | 7.1532      | Autochanger |    4 | database | LTO       | Append    |

|      22 | AAAAD55F70 | 900.0329    | Autochanger |    5 | database | LTO       | Append    |

|      21 | AAAAC95F6C | 129.9827    | Autochanger |    6 | database | LTO       | Append    |

|      24 | AAAACB5F6E | 32.4151     | Autochanger |    7 | Default  | LTO       | Append    |

|      28 | AAAB3B609E | 329.3842    | Autochanger |    8 | Default  | LTO       | Append    |

|      23 | AAAACA5F6F | 1602.8880   | Autochanger |    9 | Default  | LTO       | Append    |

|      27 | AAAACD5F68 | 15.8603     | Autochanger |   10 | Default  | LTO       | Append    |

|      18 | AAAAC85F6D | 683.2106    | Autochanger |   11 | Default  | LTO       | Full      |

+---------+------------+-------------+-------------+------+----------+-----------+—————+




###### OUT PUT FROM STATUS STORAGE #####


*status storage

Automatically selected Storage: Autochanger

Connecting to Storage daemon Autochanger at backup02.wes.popcap.com:9103


backup02-sd Version: 5.2.6 (21 February 2012) x86_64-pc-linux-gnu ubuntu 14.04

Daemon started 20-Nov-14 20:53. Jobs: run=4, running=0.

 Heap: heap=270,336 smbytes=361,672 max_bytes=665,104 bufs=248 max_bufs=325

 Sizes: boffset_t=8 size_t=8 int32_t=4 int64_t=8 mode=0,0


Running Jobs:

No Jobs running.

====


Jobs waiting to reserve a drive:

====


Terminated Jobs:

 JobId  Level    Files      Bytes   Status   Finished        Name 

===================================================================

   950  Diff          2    14.14 G  OK       20-Nov-14 10:39 shopdb3-02:database

   948  Diff          2    29.38 G  OK       20-Nov-14 11:12 pushdb1-02:database

   946  Diff          1    31.01 G  OK       20-Nov-14 11:28 ecommdb1-02:database

   959  Diff        202    27.32 G  OK       20-Nov-14 12:41 svc4

   960  Full      3,205    137.3 G  OK       20-Nov-14 19:01 svc5

   966  Diff          1    1.321 G  Cancel   20-Nov-14 20:46 labsdb3-02:database

   970  Diff          1    3.567 G  OK       20-Nov-14 21:20 labsdb3-02:database

   971  Diff          3    4.529 G  OK       20-Nov-14 21:25 labsdb4-02:database

   972  Diff          1    5.152 G  OK       20-Nov-14 21:26 labsdb5-01:database

   973  Diff          1    5.508 G  OK       20-Nov-14 21:26 labsdb6-02:database

====


Device status:

Autochanger "Autochanger" with devices:

   "Drive0" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-01-lun-0-nst)

   "Drive1" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-02-lun-0-nst)

   "Drive2" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-03-lun-0-nst)

   "Drive3" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-04-lun-0-nst)

   "Drive4" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-05-lun-0-nst)

   "Drive5" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-06-lun-0-nst)

   "Drive6" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-07-lun-0-nst)

   "Drive7" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-08-lun-0-nst)

   "Drive8" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-09-lun-0-nst)

   "Drive9" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-10-lun-0-nst)

Device "Drive0" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-01-lun-0-nst) is mounted with:

    Volume:      AAAACF5F6A

    Pool:        database

    Media type:  LTO

    Slot 3 is loaded in drive 0.

    Total Bytes=1,621,836,840,960 Blocks=290,988 Bytes/block=5,573,552

    Positioned at File=1,662 Block=0

Device "Drive1" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-02-lun-0-nst) is not open.

    Drive 1 is not loaded.

Device "Drive2" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-03-lun-0-nst) is not open.

    Drive 2 is not loaded.

Device "Drive3" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-04-lun-0-nst) is not open.

    Drive 3 is not loaded.

Device "Drive4" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-05-lun-0-nst) is not open.

    Drive 4 is not loaded.

Device "Drive5" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-06-lun-0-nst) is not open.

    Drive 5 is not loaded.

Device "Drive6" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-07-lun-0-nst) is not open.

    Drive 6 is not loaded.

Device "Drive7" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-08-lun-0-nst) is not open.

    Drive 7 is not loaded.

Device "Drive8" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-09-lun-0-nst) is not open.

    Drive 8 is not loaded.

Device "Drive9" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-10-lun-0-nst) is not open.

    Drive 9 is not loaded.

====


Used Volume status:

AAAACF5F6A on device "Drive0" (/dev/tape/by-path/ip-10.5.66.22:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-01-lun-0-nst)

    Reader=0 writers=0 devres=0 volinuse=0



###############


##### OUT PUT FROM LIST JOBS ###


|   970 | labsdb3-02:database    | 2014-11-20 20:58:07 | B    | D     |         1 |   3,567,140,654 | T         |

|   971 | labsdb4-02:database    | 2014-11-20 20:58:09 | B    | D     |         3 |   4,529,882,002 | T         |

|   972 | labsdb5-01:database    | 2014-11-20 20:58:16 | B    | D     |         0 |               0 | R         |

|   973 | labsdb6-02:database    | 2014-11-20 20:58:18 | B    | D     |         0 |               0 | R




##### AND FINALLY, querying jobid for volume shows all 4 jobs writing to the same volume at the same time, interleaving as opposed to using available volume in pool ####


Available queries:

     1: List up to 20 places where a File is saved regardless of the directory

     2: List where the most recent copies of a file are saved

     3: List last 20 Full Backups for a Client

     4: List all backups for a Client after a specified time

     5: List all backups for a Client

     6: List Volume Attributes for a selected Volume

     7: List Volumes used by selected JobId

     8: List Volumes to Restore All Files

     9: List Pool Attributes for a selected Pool

    10: List total files/bytes by Job

    11: List total files/bytes by Volume

    12: List Files for a selected JobId

    13: List Jobs stored on a selected MediaId

    14: List Jobs stored for a given Volume name

    15: List Volumes Bacula thinks are in changer

    16: List Volumes likely to need replacement from age or errors

Choose a query (1-16): 7

Enter JobId: 970

+-------+------------+

| JobId | VolumeName |

+-------+------------+

|   970 | AAAACF5F6A |

+-------+------------+

*query

Available queries:

     1: List up to 20 places where a File is saved regardless of the directory

     2: List where the most recent copies of a file are saved

     3: List last 20 Full Backups for a Client

     4: List all backups for a Client after a specified time

     5: List all backups for a Client

     6: List Volume Attributes for a selected Volume

     7: List Volumes used by selected JobId

     8: List Volumes to Restore All Files

     9: List Pool Attributes for a selected Pool

    10: List total files/bytes by Job

    11: List total files/bytes by Volume

    12: List Files for a selected JobId

    13: List Jobs stored on a selected MediaId

    14: List Jobs stored for a given Volume name

    15: List Volumes Bacula thinks are in changer

    16: List Volumes likely to need replacement from age or errors

Choose a query (1-16): 7

Enter JobId: 971

+-------+------------+

| JobId | VolumeName |

+-------+------------+

|   971 | AAAACF5F6A |

+-------+------------+

*query

Available queries:

     1: List up to 20 places where a File is saved regardless of the directory

     2: List where the most recent copies of a file are saved

     3: List last 20 Full Backups for a Client

     4: List all backups for a Client after a specified time

     5: List all backups for a Client

     6: List Volume Attributes for a selected Volume

     7: List Volumes used by selected JobId

     8: List Volumes to Restore All Files

     9: List Pool Attributes for a selected Pool

    10: List total files/bytes by Job

    11: List total files/bytes by Volume

    12: List Files for a selected JobId

    13: List Jobs stored on a selected MediaId

    14: List Jobs stored for a given Volume name

    15: List Volumes Bacula thinks are in changer

    16: List Volumes likely to need replacement from age or errors

Choose a query (1-16): 7

Enter JobId: 972

+-------+------------+

| JobId | VolumeName |

+-------+------------+

|   972 | AAAACF5F6A |

+-------+------------+

*query

Available queries:

     1: List up to 20 places where a File is saved regardless of the directory

     2: List where the most recent copies of a file are saved

     3: List last 20 Full Backups for a Client

     4: List all backups for a Client after a specified time

     5: List all backups for a Client

     6: List Volume Attributes for a selected Volume

     7: List Volumes used by selected JobId

     8: List Volumes to Restore All Files

     9: List Pool Attributes for a selected Pool

    10: List total files/bytes by Job

    11: List total files/bytes by Volume

    12: List Files for a selected JobId

    13: List Jobs stored on a selected MediaId

    14: List Jobs stored for a given Volume name

    15: List Volumes Bacula thinks are in changer

    16: List Volumes likely to need replacement from age or errors

Choose a query (1-16): 7

Enter JobId: 973

+-------+------------+

| JobId | VolumeName |

+-------+------------+

|   973 | AAAACF5F6A |

+-------+------------


From: "Ana Emília M. Arruda" <emiliaarruda AT gmail DOT com>
Date: Wednesday, November 19, 2014 at 5:56 PM
To: "Kelley, Jared" <jkelley AT popcap DOT com>
Cc: bacula-users <bacula-users AT lists.sourceforge DOT net>
Subject: Re: [Bacula-users] bacula can't make use of multi drives

Hi Jared,

In your bacula-sd.conf, you have defined an autochanger device:

Name = Autochanger

  Device = Drive-1, Drive-2, Drive-3, Drive-4, Drive-5, Drive-6, Drive-7, Drive-8, Drive-9, Drive-10

...

}


But, in your bacula-sd.conf, you don't use it. Instead, you have a storage definition for each of your tape devices, like the bellow one for the Drive-1:


Storage {

  Name = Tape1

  Address = backup              # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = “PASSWORD"  # password for SD 

  Device = Drive-1

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}


​ This way, you are not using an autochanger nor a group of devices, but stand alone tape drives.​

If you want to use the autochanger, in your bacula-dir.conf, you should have just one storage definition for your autochanger, like (note that the Device name in your bacula-dir.conf is the same as the Autochanger Name in your bacula-sd.conf):

Storage {

  Name =

Autochanger-sd ​

  Address = backup              # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = “PASSWORD"  # password for SD 

  Device =

​ Autochanger

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}


​ Also, could you please confirm if the bellow definitions in your pools are really working?

Pool {

  Name = Default

  Pool Type = Backup

  Storage = Tape7, Tape8, Tape9, Tape10

...

}


Is it really possible to have more than one storage in the storage directive definition above? Because I really think that the storage directive accepts just one storage entry definition. And if you have defined an autochanger in your bacula-dir.conf, so this should be something like:


Pool {

  Name = Default

  Pool Type = Backup

  Storage = Autochanger-sd

...

}


Best regards,

Ana



On Wed, Nov 19, 2014 at 5:01 PM, Kelley, Jared <jkelley AT popcap DOT com> wrote:
Sure, here is my bacula-sd.conf and bacula-dir.conf.  passwords and addresses edited for security reasons.



# Default Bacula Storage Daemon Configuration file

#

#  For Bacula release 3.0.1 (30 April 2009) -- debian 5.0.1

#

# You may need to change the name of your tape drive

#   on the "Archive Device" directive in the Device

#   resource.  If you change the Name and/or the 

#   "Media Type" in the Device resource, please ensure

#   that dird.conf has corresponding changes.

#


Storage {                             # definition of myself

  Name = backup02-sd

  SDPort = 9103                  # Director's port      

  WorkingDirectory = "/etc/bacula/working"

  Pid Directory = "/var/run/bacula"

  Maximum Concurrent Jobs = 20

}



# List Directors who are permitted to contact Storage daemon

#

Director {

  Name = backup02-dir

  Password = “PASSWORD"

}



## Restricted Director, used by tray-monitor to get the

#   status of the storage daemon

##

Director {

  Name = backup02-mon

  Password = “PASSWORD"

  Monitor = yes

}



Autochanger {

  Name = Autochanger

  Device = Drive-1, Drive-2, Drive-3, Drive-4, Drive-5, Drive-6, Drive-7, Drive-8, Drive-9, Drive-10

  Changer Command = "/etc/bacula/scripts/mtx-changer %c %o %S %a %d"

  Changer Device = /dev/sg12

}



Device {

  Name = Drive-1 

  Drive Index = 0

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-01-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

  Maximum Changer Wait = 10 minutes

}


Device {

  Name = Drive-2

  Drive Index = 1

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-02-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

  Maximum Changer Wait = 10 minutes

}


Device {

  Name = Drive-3

  Drive Index = 2

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-03-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

  Maximum Changer Wait = 10 minutes

}


Device {

  Name = Drive-4

  Drive Index = 3

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-04-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

  Maximum Changer Wait = 10 minutes

}


Device {

  Name = Drive-5

  Drive Index = 4

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-05-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

  Maximum Changer Wait = 10 minutes

}


Device {

  Name = Drive-6

  Drive Index = 5

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-06-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

  Maximum Changer Wait = 10 minutes

}


Device {

  Name = Drive-7

  Drive Index = 6

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-07-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

  Maximum Changer Wait = 10 minutes

}


Device {

  Name = Drive-8

  Drive Index = 7

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-08-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

  Maximum Changer Wait = 10 minutes

}


Device {

  Name = Drive-9

  Drive Index = 8

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-09-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

  Maximum Changer Wait = 10 minutes

}


Device {

  Name = Drive-10

  Drive Index = 9

  Media Type = LTO

  Archive Device = /dev/tape/by-path/ip-:3260-iscsi-iqn.1997-05.com.amazon:sgw-20b85d49-tapedrive-10-lun-0-nst

  AutomaticMount = yes;               # when device opened, read it

  AlwaysOpen = yes;

  RemovableMedia = yes;

  RandomAccess = no;

  LabelMedia = no;

  AutoChanger = yes

  # Enable the Alert command only if you have the mtx package loaded

  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"

  SpoolDirectory = /etc/bacula/spool

  Maximum Concurrent Jobs = 20 

  Maximum Changer Wait = 10 minutes

}



# Send all messages except skipped files back to Director

Messages {

  Name = Standard

  director = backup02-dir = all, !skipped, !restored

}

########### END BACULA-SD.CONF ############

########### BEGIN BACULA-DIR.CONF ##########


Default Bacula Director Configuration file

#

#  The only thing that MUST be changed is to add one or more

#   file or directory names in the Include directive of the

#   FileSet resource.

#

#  For Bacula release 3.0.1 (30 April 2009) -- debian 5.0.1

#

#  You might also want to change the default email address

#   from root to your address.  See the "mail" and "operator"

#   directives in the Messages resource.

#


###

### Source other config files

###


@/etc/bacula/clients.conf

@/etc/bacula/jobs.conf


###

### Basic director configuration

###


Director {                            # define myself

  Name = backup02-dir

  DIRport = 9101                # where we listen for UA connections

  QueryFile = "/etc/bacula/query.sql"

  WorkingDirectory = "/etc/bacula/working"

  PidDirectory = "/var/run/bacula"

  Maximum Concurrent Jobs = 20 

  Password = “PASSWORD"         # Console password

  Messages = Daemon

  DIRAddress = localhost

}



###

### Storage

###



# Storage device for tape

Storage {

  Name = Tape1

  Address = backup              # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = “PASSWORD"  # password for SD 

  Device = Drive-1

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}


Storage {

  Name = Tape2

  Address = backup               # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = “PASSWORD"  # password for SD 

  Device = Drive-2                      # must be same Device in sd

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}


Storage {

  Name = Tape3

  Address = backup               # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = “PASSWORD"  # password for SD 

  Device = Drive-3

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}


Storage {

  Name = Tape4

  Address = backup               # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = “PASSWORD"  # password for SD 

  Device = Drive-4

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}


Storage {

  Name = Tape5

  Address = backup               # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = “PASSWORD"  # password for SD 

  Device = Drive-5

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}


Storage {

  Name = Tape6

  Address = backup               # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = “PASSWORD"  # password for SD 

  Device = Drive-6

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}


Storage {

  Name = Tape7

  Address = backup               # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = “PASSWORD"  # password for SD 

  Device = Drive-7

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}


Storage {

  Name = Tape8

  Address = backup               # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = “PASSWORD"  # password for SD 

  Device = Drive-8

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}


Storage {

  Name = Tape9

  Address = backup               # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = “PASSWORD"  # password for SD 

  Device = Drive-9

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}


Storage {

  Name = Tape10

  Address = backup              # N.B. Use a fully qualified name here, not localhost

  SDPort = 9103

  Password = “PASSWORD"  # password for SD 

  Device = Drive-10

  Media Type = LTO

  Autochanger = yes                   # enable for autochanger device

  Maximum Concurrent Jobs = 20 

}


#Storage {

###

###

### Storage pools

###

# Default storage pool

Pool {

  Name = Default

  Pool Type = Backup

  Storage = Tape7, Tape8, Tape9, Tape10

  Recycle = no                       # Bacula can automatically recycle Volumes

  AutoPrune = yes                     # Prune expired volumes

  Volume Retention = 365 days         # one year

  Maximum Volume Bytes = 2500G

}


###

# social service storage pool

Pool {

  Name = svc

  Pool Type = Backup

  Storage = Tape1, Tape2

  Recycle = no                       # Bacula can automatically recycle Volumes

  AutoPrune = yes                     # Prune expired volumes

  Volume Retention = 365 days         # one year

  Maximum Volume Bytes = 2500G

}


# database service storage pool

Pool {

  Name = database 

  Pool Type = Backup

  Storage = Tape3, Tape4, Tape5, Tape6

  Recycle = no                       # Bacula can automatically recycle Volumes

  AutoPrune = yes                     # Prune expired volumes

  Volume Retention = 365 days         # one year

  Maximum Volume Bytes = 2500G

}


###

### Backup catalog

###


# Generic catalog service

Catalog {

  Name = MyCatalog

# Uncomment the following line if you want the dbi driver

# dbdriver = "dbi:mysql"; dbaddress = 127.0.0.1; dbport =  

  dbname = "bacula"; dbuser = “USER"; dbpassword = “HAHAHA"

}



###

### Logging

###


# Reasonable message delivery -- send most everything to email address

#  and to the console

Messages {

  Name = Standard

#

# NOTE! If you send to two email or more email addresses, you will need

#  to replace the %r in the from field (-f part) with a single valid

#  email address in both the mailcommand and the operatorcommand.

#  What this does is, it sets the email address that emails would display

#  in the FROM field, which is by default the same email as they're being

#  sent to.  However, if you send email to more than one address, then

#  you'll have to set the FROM address manually, to a single address. 

#  for example, a 'no-reply AT mydomain DOT com', is better since that tends to

#  tell (most) people that its coming from an automated source.


#

  mailcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula: %t %e of %c %l\" %r"

  operatorcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula: Intervention needed for %j\" %r"

  mail on error = EMAILADDRESS = all, !skipped            

  operator = EMAILADDRESS= mount

  console = all, !skipped, !saved

#

# WARNING! the following will create a file that you must cycle from

#          time to time as it will grow indefinitely. However, it will

#          also keep all your messages if they scroll off the console.

#

  append = "/etc/bacula/working/log" = all, !skipped


  catalog = all, !skipped, !saved

}



#

# Message delivery for daemon messages (no job).

Messages {

  Name = Daemon

  mailcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula daemon message\" %r"

  mail = root@localhost = all, !skipped            

  console = all, !skipped, !saved

  append = "/etc/bacula/working/log" = all, !skipped

}



###

### Console

###


# Restricted console used by tray-monitor to get the status of the director

Console {

  Name = backup02-mon

  Password = “PASSWORD"

  CommandACL = status, .status

}


From: "Ana Emília M. Arruda" <emiliaarruda AT gmail DOT com>
Date: Wednesday, November 19, 2014 at 11:27 AM
To: "Kelley, Jared" <jkelley AT popcap DOT com>
Cc: "heitor AT bacula.com DOT br" <heitor AT bacula.com DOT br>, Josh Fisher <jfisher AT pvct DOT com>, "bacula-users AT lists.sourceforge DOT net" <bacula-users AT lists.sourceforge DOT net>

Subject: Re: [Bacula-users] bacula can't make use of multi drives

Hi Jared,

Can you post here your storage configurations?

Best regards,
Ana

On Wed, Nov 19, 2014 at 3:56 PM, Kelley, Jared <jkelley AT popcap DOT com> wrote:
For the sake of sharing information I am going to reply to this thread since I am working on the same type of setup and experiencing similar issues.  


I removed the setting ‘prefer mounted volumes = no' and separated my jobs into 3 different pools.  All my jobs are running smoothly now, no failures.  The prefer mounted volumes was the issue.
Now I am experiencing interleaving on a single volume/tape drive with multiple jobs scheduled at the same time, in the same pool, as opposed to bacula using different volumes and different drives for jobs scheduled at the same time for the same pool.

Here is my issue:

I have 10 jobs that kick off  at the same time every day.  Each job takes anywhere from 10 minutes to 2 hours.  The pool behind these jobs has 4 tape drives and 4 volumes(virtual tapes) assigned to it.  The 10 jobs all kick off at the same time and they all write to the same volume, apparently(assuming) interleaving the data.   See my paste below, Ive bolded start times, end times, pools and tapes.  One can see these jobs all completed near each other and ran for over an hour.  All to the same tape/volume, tape3 and volume AAAACF5F6A .   I assume if I had the "prefer mounted volume=no" for these jobs they would use all 4 drives in the pool and write to separate volumes as opposed to interleaving to a single drive and volume.  But based on the manual, reported problems and experienced problems with ‘prefer mounted volumes = no’ I hesitate to use that.  

My question:
Is there another way to get multiple jobs scheduled at the same time to run concurrently using different drives and volumes, as assigned to the pool, versus all writing at the same time and interleaving the data to a single drive and volume even though the pool has 4 drives/volumes assigned to it?    Or is prefer mounted volumes = no the only way to solve this?

any help is greatly appreciated

19-Nov 10:06 backup02-sd JobId 925: Job write elapsed time = 01:01:23, Transfer rate = 1.489 M Bytes/second

19-Nov 10:06 backup02-dir JobId 925: Bacula backup02-dir 5.2.6 (21Feb12):

  Build OS:               x86_64-pc-linux-gnu ubuntu 14.04

  JobId:                  925

  Job:                    labsdb1.2014-11-19_09.05.00_21

  Backup Level:           Full (upgraded from Differential)

  Client:                 "labsdb1" 2.4.4 (28Dec08) x86_64-pc-linux-gnu,debian,lenny/sid

  FileSet:                "Database Backup" 2014-11-07 22:02:50

  Pool:                   "database" (From Job resource)

  Catalog:                "MyCatalog" (From Client resource)

  Storage:                "Tape3" (From Pool resource)

  Scheduled time:         19-Nov-2014 09:05:00

  Start time:             19-Nov-2014 09:05:02

  End time:               19-Nov-2014 10:06:26

  Elapsed time:           1 hour 1 min 24 secs

  Priority:               10

  FD Files Written:       2

  SD Files Written:       2

  FD Bytes Written:       5,486,802,781 (5.486 GB)

  SD Bytes Written:       5,486,802,988 (5.486 GB)

  Rate:                   1489.4 KB/s

  Software Compression:   None

  VSS:                    no

  Encryption:             no

  Accurate:               no

  Volume name(s):         AAAACF5F6A

  Volume Session Id:      155

  Volume Session Time:    1415816791

  Last Volume Bytes:      1,348,311,508,992 (1.348 TB)

  Non-fatal FD errors:    0

  SD Errors:              0

  FD termination status:  OK

  SD termination status:  OK

  Termination:            Backup OK


19-Nov 10:06 backup02-dir JobId 925: Begin pruning Jobs older than 1 year .

19-Nov 10:06 backup02-dir JobId 925: No Jobs found to prune.

19-Nov 10:06 backup02-dir JobId 925: Begin pruning Files.

19-Nov 10:06 backup02-dir JobId 925: No Files found to prune.

19-Nov 10:06 backup02-dir JobId 925: End auto prune.


19-Nov 10:41 backup02-sd JobId 919: Job write elapsed time = 01:36:26, Transfer rate = 2.570 M Bytes/second

19-Nov 10:41 backup02-dir JobId 919: Bacula backup02-dir 5.2.6 (21Feb12):

  Build OS:               x86_64-pc-linux-gnu ubuntu 14.04

  JobId:                  919

  Job:                    shopd1.2014-11-19_09.05.00_15

  Backup Level:           Full (upgraded from Differential)

  Client:                 "shopdb1" 5.0.2 (28Apr10) x86_64-pc-linux-gnu,debian,squeeze/sid

  FileSet:                "varlibDatabase Backup" 2014-11-06 20:20:52

  Pool:                   "database" (From Job resource)

  Catalog:                "MyCatalog" (From Client resource)

  Storage:                "Tape3" (From Pool resource)

  Scheduled time:         19-Nov-2014 09:05:00

  Start time:             19-Nov-2014 09:05:00

  End time:               19-Nov-2014 10:41:26

  Elapsed time:           1 hour 36 mins 26 secs

  Priority:               10

  FD Files Written:       3

  SD Files Written:       3

  FD Bytes Written:       14,875,191,024 (14.87 GB)

  SD Bytes Written:       14,875,192,011 (14.87 GB)

  Rate:                   2570.9 KB/s

  Software Compression:   None

  VSS:                    no

  Encryption:             yes

  Accurate:               no

  Volume name(s):         AAAACF5F6A

  Volume Session Id:      149

  Volume Session Time:    1415816791

  Last Volume Bytes:      1,375,507,574,784 (1.375 TB)

  Non-fatal FD errors:    0

  SD Errors:              0

  FD termination status:  OK

  SD termination status:  OK

  Termination:            Backup OK


19-Nov 10:41 backup02-dir JobId 919: Begin pruning Jobs older than 7 years .

19-Nov 10:41 backup02-dir JobId 919: No Jobs found to prune.

19-Nov 10:41 backup02-dir JobId 919: Begin pruning Files.

19-Nov 10:41 backup02-dir JobId 919: No Files found to prune.

19-Nov 10:41 backup02-dir JobId 919: End auto prune.



19-Nov 11:10 backup02-sd JobId 917: Job write elapsed time = 02:05:32, Transfer rate = 3.894 M Bytes/second

19-Nov 11:10 backup02-dir JobId 917: Bacula backup02-dir 5.2.6 (21Feb12):

  Build OS:               x86_64-pc-linux-gnu ubuntu 14.04

  JobId:                  917

  Job:                    pushdb.2014-11-19_09.05.00_13

  Backup Level:           Full (upgraded from Differential)

  Client:                 "pushdb1" 5.0.2 (28Apr10) x86_64-pc-linux-gnu,debian,6.0.6

  FileSet:                "varlibDatabase Backup" 2014-11-06 20:20:52

  Pool:                   "database" (From Job resource)

  Catalog:                "MyCatalog" (From Client resource)

  Storage:                "Tape3" (From Pool resource)

  Scheduled time:         19-Nov-2014 09:05:00

  Start time:             19-Nov-2014 09:05:00

  End time:               19-Nov-2014 11:10:33

  Elapsed time:           2 hours 5 mins 33 secs

  Priority:               10

  FD Files Written:       2

  SD Files Written:       2

  FD Bytes Written:       29,334,385,684 (29.33 GB)

  SD Bytes Written:       29,334,385,894 (29.33 GB)

  Rate:                   3894.1 KB/s

  Software Compression:   None

  VSS:                    no

  Encryption:             no

  Accurate:               no

  Volume name(s):         AAAACF5F6A

  Volume Session Id:      147

  Volume Session Time:    1415816791

  Last Volume Bytes:      1,397,835,371,520 (1.397 TB)

  Non-fatal FD errors:    1

  SD Errors:              0

  FD termination status:  OK

  SD termination status:  OK

  Termination:            Backup OK 


19-Nov 11:10 backup02-dir JobId 917: Begin pruning Jobs older than 1 year .

19-Nov 11:10 backup02-dir JobId 917: No Jobs found to prune.

19-Nov 11:10 backup02-dir JobId 917: Begin pruning Files.

19-Nov 11:10 backup02-dir JobId 917: No Files found to prune.

19-Nov 11:10 backup02-dir JobId 917: End auto prune.


19-Nov 11:26 backup02-sd JobId 915: Job write elapsed time = 02:21:25, Transfer rate = 3.655 M Bytes/second

19-Nov 11:26 backup02-dir JobId 915: Bacula backup02-dir 5.2.6 (21Feb12):

  Build OS:               x86_64-pc-linux-gnu ubuntu 14.04

  JobId:                  915

  Job:                    ecommdb1.2014-11-19_09.05.00_11

  Backup Level:           Differential, since=2014-11-08 09:05:03

  Client:                 "ecommdb" 2.4.4 (28Dec08) x86_64-pc-linux-gnu,debian,lenny/sid

  FileSet:                "Database Backup" 2014-11-07 22:02:50

  Pool:                   "database" (From Job resource)

  Catalog:                "MyCatalog" (From Client resource)

  Storage:                "Tape3" (From Pool resource)

  Scheduled time:         19-Nov-2014 09:05:00

  Start time:             19-Nov-2014 09:05:00

  End time:               19-Nov-2014 11:26:27

  Elapsed time:           2 hours 21 mins 27 secs

  Priority:               10

  FD Files Written:       1

  SD Files Written:       1

  FD Bytes Written:       31,016,183,168 (31.01 GB)

  SD Bytes Written:       31,016,183,295 (31.01 GB)

  Rate:                   3654.6 KB/s

  Software Compression:   None

  VSS:                    no

  Encryption:             no

  Accurate:               no

  Volume name(s):         AAAACF5F6A

  Volume Session Id:      145

  Volume Session Time:    1415816791

  Last Volume Bytes:      1,407,107,681,280 (1.407 TB)

  Non-fatal FD errors:    0

  SD Errors:              0

  FD termination status:  OK

  SD termination status:  OK

  Termination:            Backup OK



From: "heitor AT bacula.com DOT br" <heitor AT bacula.com DOT br>
Date: Wednesday, November 19, 2014 at 9:03 AM
To: Josh Fisher <jfisher AT pvct DOT com>
Cc: "bacula-users AT lists.sourceforge DOT net" <bacula-users AT lists.sourceforge DOT net>
Subject: Re: [Bacula-users] bacula can't make use of multi drives

I think that the only problem with setting "Prefer Mounted Volumes = no" is that it will result in a lot of moving of a volume back and forth between drives. While it might not be a bug, since it works as advertised, it CAN be a problem. For tape, it will really slow things down because it takes a substantial time to move a tape out of one drive and into another. For disk virtual autochangers, though, the moving between "drives" is simply closing a disk file and then re-opening it, meaning that the delay is very negligible.

Mr. Fisher: It's not just that. Eventually "Prefer Mounted Volumes = no" messes up the autochanger during a random backup and you have to restart storage daemon and sometimes powercycle the tape library.

If there is some other issue, like a race condition that I am not aware of, then that would indeed be a bug in my book. But I have used PreferMountedVolumes=no for disk virtual autochanger without (so far) encountering any such problems, just the swapping between drives. And for disk virtual autochangers, I don't see that as a problem in and of itself.

I don't know if it is already fixed or if this error happens for all autochangers. But last Bacula System Community Storage whitepapers goes in the same way. Having 2 pools is still safer and more reliable way to do multiple volumes writing.

Regards,
=========================================================================
Heitor Medrado de Faria  - LPIC-III | ITIL-F
Faltam poucos dias - Treinamento Telepresencial Bacula: http://www.bacula.com.br/?p=2174
==========================================================================


De: "Josh Fisher" <jfisher AT pvct DOT com>
Para: bacula-users AT lists.sourceforge DOT net
Enviadas: Quarta-feira, 19 de novembro de 2014 14:52:42
Assunto: Re: [Bacula-users] bacula can't make use of multi drives

On 11/19/2014 11:10 AM, heitor AT bacula.com DOT br wrote:
Is it a bug that bacula cannot write to multiple drives for the jobs in the same pool? Will there be a fix. Back in March 2014 Wolfgang Denk wrote "it can" but how?

http://adsm.org/lists/html/Bacula-users/2014-03/msg00212.html

It's not a bug, but just the way it's designed.
"Prefer Mounted Volumes=no" it's a directive that could allow to write multiple volumes within the same pool at the same time, however this is a little buggy and not recommended at this time.

It's not a bug, but it is a little buggy??

I think that the only problem with setting "Prefer Mounted Volumes = no" is that it will result in a lot of moving of a volume back and forth between drives. While it might not be a bug, since it works as advertised, it CAN be a problem. For tape, it will really slow things down because it takes a substantial time to move a tape out of one drive and into another. For disk virtual autochangers, though, the moving between "drives" is simply closing a disk file and then re-opening it, meaning that the delay is very negligible.

If there is some other issue, like a race condition that I am not aware of, then that would indeed be a bug in my book. But I have used PreferMountedVolumes=no for disk virtual autochanger without (so far) encountering any such problems, just the swapping between drives. And for disk virtual autochangers, I don't see that as a problem in and of itself.


Regards,
=========================================================================
Heitor Medrado de Faria  - LPIC-III | ITIL-F
Faltam poucos dias - Treinamento Telepresencial Bacula: http://www.bacula.com.br/?p=2174
==========================================================================



------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users






------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk


_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT nethttps://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users



------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
<Prev in Thread] Current Thread [Next in Thread>