Bacula-users

Re: [Bacula-users] vchanger 1.0.1 says "No slots in changer to scan"

2016-05-05 08:20:30
Subject: Re: [Bacula-users] vchanger 1.0.1 says "No slots in changer to scan"
From: Josh Fisher <jfisher AT pvct DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Thu, 5 May 2016 08:19:40 -0400

Hi Pann. See comments inline below.


On 5/5/2016 4:29 AM, Pann Tolk wrote:

Hello Guys,

I'm new to Bacula.  And it's my first time trying to config Bacula 7.4.0 with vchanger 1.0.1 for my home network.

I have configured Bacula and vchanger following the vchanger 1.0.1 documentation included in the rpm package (vchangerHowto.html)

My hardware setup is as below:

       |-SATA_Port_1 -| 2-slots |    < Multiple 2TB SATA harddisks, each 
 HOST -|              | hotplug |<== < formated with single ext4 partition
       |-SATA_Port_2 -| dock    |    < (with unique UUID) for the respective
                                     < filesystem to be used as magazines.

The above 2-slots hotplug dock will allow me insert or remove 2 physical SATA drives (and therefore, 2 magazines) at a time.

For each magazine, I intend to make each volume file to be 100GB.


Here is my host system software setup:

- Fedora 23

- Bacula 7.4.0 (installed from Fedora repository)
Both Bacula Director and Bacula Storage Daemon are installed on the same host
(above).  It will be used to backup other clients on the network.

- vchanger-1.0.1-1.el7.x86_64.rpm (downloaded from Sourceforge: https://sourceforge.net/projects/vchanger/files/RPM/)



I have the following questions and problem:

1.  With regards to using 2 magazines simultaneously, are the following correct?

/etc/bacula/bacula-sd.conf:
===========================

[root@deneb ~]# cat /etc/bacula/bacula-sd.conf | grep -v  "^[#]"

Storage {                        # definition of myself
  Name = bacula-sd
  SDPort = 9103                  # Director's port
  WorkingDirectory = "/var/spool/bacula"
  Pid Directory = "/var/run"
  Maximum Concurrent Jobs = 20
}

Director {
  Name = bacula-dir
  Password = "xxxxxxxxxxxxxxxx"
}

Autochanger {
  Name = SATA-vchanger01
  Device = SATA-vchanger01-drive-0
  Changer Command = "vchanger %c %o %S %a %d"
  Changer Device = "/etc/vchanger/vchanger01.conf"
}

Device {
  Name = SATA-vchanger01-drive-0
  Drive Index = 0
  Autochanger = yes
  Device Type = File
  Media Type = File
  Removable Media = no
  Random Access = yes
  Maximum Concurrent Jobs = 1
  Archive Device = "/var/spool/vchanger/vchanger01/drive0"
}

Messages {
  Name = Standard
  director = bacula-dir = all
}
[root@deneb ~]#



/etc/vchanger/vchanger01.conf:
==============================

[root@deneb ~]# cat /etc/vchanger/vchanger01.conf | grep -v  "^[#]"

Storage Resource = "vchanger01"
User = bacula
group = tape
work dir = "/var/spool/vchanger/vchanger01"
logfile = "/var/log/vchanger/vchanger01.log"
log level = 3
bconsole = "/usr/sbin/bconsole"
bconsole config = /etc/bacula/bconsole.conf
default pool = "Scratch"

magazine = UUID:1fde15be-83a4-4b67-accc-378d5fa10ea7
magazine = UUID:412060d2-7be4-4719-8089-2ee6cfd28624
magazine = UUID:20d986c1-9391-467e-9f73-232b53fba377
magazine = UUID:fa48ed8d-2e63-4c27-8670-c943822d2cd0




2.  I followed the vchanger documentation (vchangerHowto.html) to setup vchanger.

I have successfully configured udev and autofs.  udev is able to mount the filesystems when I insert the drives.  And, autofs is able to mount it to the mountpoint /mnt/vchanger


You should not configure for both autofs and udev at the same time, but rather one or the other.

I have also successfully created volumes in the first 2 magazines.


[root@deneb ~]# vchanger /etc/vchanger/vchanger01.conf listmags
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
0:18:1:/mnt/vchanger/1fde15be-83a4-4b67-accc-378d5fa10ea7
1:18:19:/mnt/vchanger/412060d2-7be4-4719-8089-2ee6cfd28624
2:::
3:::

[root@deneb ~]# vchanger /etc/vchanger/vchanger01.conf list
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
device-enumerator: scan all dirs
  device-enumerator: scanning /sys/bus
  device-enumerator: scanning /sys/class
1:vchanger01_0_0
2:vchanger01_0_1
3:vchanger01_0_10
4:vchanger01_0_11
5:vchanger01_0_12
6:vchanger01_0_13
7:vchanger01_0_14
8:vchanger01_0_15
9:vchanger01_0_16
10:vchanger01_0_17
11:vchanger01_0_2
12:vchanger01_0_3
13:vchanger01_0_4
14:vchanger01_0_5
15:vchanger01_0_6
16:vchanger01_0_7
17:vchanger01_0_8
18:vchanger01_0_9
19:vchanger01_1_0
20:vchanger01_1_1
21:vchanger01_1_10
22:vchanger01_1_11
23:vchanger01_1_12
24:vchanger01_1_13
25:vchanger01_1_14
26:vchanger01_1_15
27:vchanger01_1_16
28:vchanger01_1_17
29:vchanger01_1_2
30:vchanger01_1_3
31:vchanger01_1_4
32:vchanger01_1_5
33:vchanger01_1_6
34:vchanger01_1_7
35:vchanger01_1_8
36:vchanger01_1_9
37:


[root@deneb ~]#  echo "status slots storage=vchanger01 drive=0" | bconsole
Connecting to Director deneb.pt.local:9101
1000 OK: 102 bacula-dir Version: 7.4.0 (16 January 2016)
Enter a period to cancel a command.
status slots storage=vchanger01 drive=0
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
Connecting to Storage daemon vchanger01 at deneb.pt.local:9103 ...
3306 Issuing autochanger "slots" command.
Device "SATA-vchanger01" has 0 slots.
No slots in changer to scan.



Questions:
==========
2a.  Why did vchanger list and listmags spits out "device-enumerator" lines?

Those are generated by systemd. My best guess is that they are triggered by vchanger's use of the libudev function udev_enumerate_scan_devices() to scan for mounted filesystems by UUID. But systemd is a total enigma to me, so I am not sure why this would be the case.


2b.  bconsole throws out the lines below when I issue "status slots".  Did I misconfigured something?

Device "SATA-vchanger01" has 0 slots.
No slots in changer to scan.


This is very likely one of two things (or both); a race condition bug in vchanger or vchanger is incompatible with the installed version of libudev..

The vchanger bug is due to vchanger automatically issuing 'update slots' commands via bconsole when it detects a change in the volume files it has access to. Vchanger is invoking bconsole in a child process. The child bconsole process executes a 'update slots' command that in turn causes bacula-sd to invoke a few other instances of vchanger with the SLOTS and LIST commands, which is how bacula-sd queries autochangers to obtain the number of slots and current volume barcodes in those slots. Vchanger uses a lockfile to serialize the simultaneous instances, but some bug in vchanger is causing a race for the lockfile (I think).

The simple workaround to determine if the vchanger bug is causing your problem, is to change the bconsole="/usr/sbin/bconsole" line in vchanger.conf to bconsole="". This will prevent vchanger from issuing bconsole commands at all. Of course you will then have to issue the update slots command in bconsole manually whenever anything changes with the magazine drives. You may need to manually delete any leftover stale .lock files in the vchanger work directory.

Another possibility is that there has been some change in recent releases of libudev that causes vchanger to not find filesystem mountpoints by UUID. That would perhaps explain the messages from systemd. The workaround for that is to make sure the drives are mounted, either by udev or manually, and then specify the magazines as a mountpoint path in vchanger.conf, rather than as a UUID. When specified as a full path, vchanger just assumes the volumes are in that directory and does not call any libudev functions.



2c.  Also, when I issue "status storage" in bconsole, I get following:

*status storage
...
...
Device status:
Autochanger "SATA-vchanger01" with devices:
   "SATA-vchanger01-drive-0" (/var/spool/vchanger/vchanger01/drive0)

Device file: "SATA-vchanger01-drive-0" (/var/spool/vchanger/vchanger01/drive0) is not open.
    Drive 0 is not loaded.




I have searched Google and still not able to find out why / what's wrong with my setup.  Appreciate if someone can help me out.

Thanks in advance.
PanTor





=====================================================================
Below are my settings and config files
=====================================================================

--------------------------------
SELINUX Setting
--------------------------------
[root@deneb ~]# /usr/sbin/sestatus
SELinux status:                 disabled


[root@deneb ~]# cat /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

-----------------------------------
Bacula Director and SD Running As
-----------------------------------

[root@deneb ~]# ps -aux | grep bacula
bacula   12628  0.0  0.5 365132 11124 ?        Ssl  14:10   0:00 /usr/sbin/bacula-dir -f -c /etc/bacula/bacula-dir.conf -u bacula -g bacula
bacula   12640  0.0  0.3 327868  7764 ?        Ssl  14:10   0:00 /usr/sbin/bacula-sd -f -c /etc/bacula/bacula-sd.conf -u bacula -g tape

-----------------------------------
Config for Bacula
-----------------------------------

[root@deneb ~]# ls -l /etc/bacula/
total 36
-rw-r--r-- 1 bacula bacula 9419 May  3 13:56 bacula-dir.conf
-rw-r--r-- 1 bacula bacula 8944 May  5 09:58 bacula-sd.conf
-rw-r--r-- 1 bacula bacula  254 Apr 26 11:54 bconsole.conf
-rw-r----- 1 bacula bacula  312 Jan 26 04:26 query.sql

--------------------------------
/etc/bacula/bacula-dir.conf
--------------------------------
[root@deneb ~]# cat /etc/bacula/bacula-dir.conf | grep -v  "^[#]"

Director {                            # define myself
  Name = bacula-dir
  DIRport = 9101                # where we listen for UA connections
  QueryFile = "/etc/bacula/query.sql"
  WorkingDirectory = "/var/spool/bacula"
  PidDirectory = "/var/run"
  Maximum Concurrent Jobs = 20
  Password = "xxxxxxxxxxxxxxxx"         # Console password
  Messages = Daemon
}

JobDefs {
  Name = "DefaultJob"
  Type = Backup
  Level = Incremental
  Client = bacula-fd
  FileSet = "Full Set"
  Schedule = "WeeklyCycle"
  Storage = vchanger01
  Messages = Standard
  Pool = File
  SpoolAttributes = yes
  Priority = 10
  Write Bootstrap = "/var/spool/bacula/%c.bsr"
}

Job {
  Name = "BackupClient1"
  JobDefs = "DefaultJob"
}

Job {
  Name = "BackupCatalog"
  JobDefs = "DefaultJob"
  Level = Full
  FileSet="Catalog"
  Schedule = "WeeklyCycleAfterBackup"
  # This creates an ASCII copy of the catalog
  # Arguments to make_catalog_backup.pl are:
  #  make_catalog_backup.pl <catalog-name>
  RunBeforeJob = "/usr/libexec/bacula/make_catalog_backup.pl MyCatalog"
  # This deletes the copy of the catalog
  RunAfterJob  = "/usr/libexec/bacula/delete_catalog_backup"
  Write Bootstrap = "/var/spool/bacula/%n.bsr"
  Priority = 11                   # run after main backup
}

Job {
  Name = "RestoreFiles"
  Type = Restore
  Client=bacula-fd
  FileSet="Full Set"
  Storage = vchanger01
  Pool = File
  Messages = Standard
  Where = /tmp/bacula-restores
}

FileSet {
  Name = "Full Set"
  Include {
    Options {
      signature = MD5
    }
     File = /usr/sbin
  }

  Exclude {
    File = /var/spool/bacula
    File = /tmp
    File = /proc
    File = /tmp
    File = /sys
    File = /.journal
    File = /.fsck
  }
}

Schedule {
  Name = "WeeklyCycle"
  Run = Full 1st sun at 23:05
  Run = Differential 2nd-5th sun at 23:05
  Run = Incremental mon-sat at 23:05
}

Schedule {
  Name = "WeeklyCycleAfterBackup"
  Run = Full sun-sat at 23:10
}

FileSet {
  Name = "Catalog"
  Include {
    Options {
      signature = MD5
    }
    File = "/var/spool/bacula/bacula.sql"
  }
}

Client {
  Name = bacula-fd
  Address = episilon.pt.local
  FDPort = 9102
  Catalog = MyCatalog
  Password = "xxxxxxxxxxxxxxxx"          # password for FileDaemon
  File Retention = 60 days            # 60 days
  Job Retention = 6 months            # six months
  AutoPrune = yes                     # Prune expired Jobs/Files
}

Storage {
  Name = vchanger01
  Address = deneb.pt.local        # N.B. Use a fully qualified name here
  SDPort = 9103
  Password = "xxxxxxxxxxxxxxxx"
  Device = SATA-vchanger01
  Media Type = File
  Autochanger = yes
  Maximum Concurrent Jobs = 10        # run up to 10 jobs a the same time
}

Catalog {
  Name = MyCatalog
  dbname = "bacula"; dbuser = "bacula"; dbpassword = "xxxxxxxxxxxxxxxx"
}

Messages {
  Name = Standard
  mailcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula: %t %e of %c %l\" %r"
  operatorcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula: Intervention needed for %j\" %r"
  mail = root@localhost = all, !skipped
  operator = root@localhost = mount
  console = all, !skipped, !saved
  append = "/var/log/bacula/bacula.log" = all, !skipped
  catalog = all
}

Messages {
  Name = Daemon
  mailcommand = "/usr/sbin/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula daemon message\" %r"
  mail = root@localhost = all, !skipped
  console = all, !skipped, !saved
  append = "/var/log/bacula/bacula.log" = all, !skipped
}

Pool {
  Name = Default
  Pool Type = Backup
  Recycle = yes                       # Bacula can automatically recycle Volumes
  AutoPrune = yes                     # Prune expired volumes
  Volume Retention = 365 days         # one year
  Maximum Volume Bytes = 100G         # Limit Volume size to something reasonable
  Maximum Volumes = 100               # Limit number of Volumes in Pool
}

Pool {
  Name = File
  Pool Type = Backup
  Recycle = yes                       # Bacula can automatically recycle Volumes
  AutoPrune = yes                     # Prune expired volumes
  Volume Retention = 365 days         # one year
  Maximum Volume Bytes = 100G         # Limit Volume size to something reasonable
  Maximum Volumes = 100               # Limit number of Volumes in Pool
  Label Format = "Vol-"               # Auto label
}

Pool {
  Name = Scratch
  Pool Type = Backup
}


--------------------------------
/etc/bacula/bacula-sd.conf
--------------------------------

[root@deneb ~]# cat /etc/bacula/bacula-sd.conf | grep -v  "^[#]"

Storage {                        # definition of myself
  Name = bacula-sd
  SDPort = 9103                  # Director's port
  WorkingDirectory = "/var/spool/bacula"
  Pid Directory = "/var/run"
  Maximum Concurrent Jobs = 20
}

Director {
  Name = bacula-dir
  Password = "xxxxxxxxxxxxxxxx"
}

Autochanger {
  Name = SATA-vchanger01
  Device = SATA-vchanger01-drive-0
  Changer Command = "vchanger %c %o %S %a %d"
  Changer Device = "/etc/vchanger/vchanger01.conf"
}

Device {
  Name = SATA-vchanger01-drive-0
  Drive Index = 0
  Autochanger = yes
  Device Type = File
  Media Type = File
  Removable Media = no
  Random Access = yes
  Maximum Concurrent Jobs = 1
  Archive Device = "/var/spool/vchanger/vchanger01/drive0"
}

Messages {
  Name = Standard
  director = bacula-dir = all
}


--------------------------------
/etc/bacula/bconsole.conf
--------------------------------

[root@deneb ~]# cat /etc/bacula/bconsole.conf | grep -v  "^[#]"

Director {
  Name = bacula-dir
  DIRport = 9101
  address = deneb.pt.local
  Password = "xxxxxxxxxxxxxxxx"


--------------------------------
/etc/vchanger/vchanger01.conf
--------------------------------

[root@deneb ~]# cat /etc/vchanger/vchanger01.conf | grep -v  "^[#]"

Storage Resource = "vchanger01"
User = bacula
group = tape
work dir = "/var/spool/vchanger/vchanger01"
logfile = "/var/log/vchanger/vchanger01.log"
log level = 3
bconsole = "/usr/sbin/bconsole"
bconsole config = /etc/bacula/bconsole.conf
default pool = "Scratch"

magazine = UUID:1fde15be-83a4-4b67-accc-378d5fa10ea7
magazine = UUID:412060d2-7be4-4719-8089-2ee6cfd28624
magazine = UUID:20d986c1-9391-467e-9f73-232b53fba377
magazine = UUID:fa48ed8d-2e63-4c27-8670-c943822d2cd0


--------------------------------
Files Ownership and Permissions
--------------------------------
[root@deneb ~]# ls -l /mnt/vchanger/
total 8
drwxr-x--- 3 bacula tape 4096 May  4 19:34 1fde15be-83a4-4b67-accc-378d5fa10ea7
drwxr-x--- 3 bacula tape 4096 May  4 19:34 412060d2-7be4-4719-8089-2ee6cfd28624

[root@deneb ~]# ls -l /mnt/vchanger/1fde15be-83a4-4b67-accc-378d5fa10ea7/
total 16
drwxr-x--- 2 bacula tape 16384 Apr 27 13:57 lost+found
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_0
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_1
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_10
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_11
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_12
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_13
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_14
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_15
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_16
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_17
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_2
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_3
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_4
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_5
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_6
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_7
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_8
-rw-r--r-- 1 bacula tape     0 May  4 19:34 vchanger01_0_9

[root@deneb ~]# ls -l /etc/vchanger
total 4
-rw-r--r-- 1 root root 3754 May  5 10:12 vchanger01.conf

[root@deneb ~]# ls -l /etc/vchanger/vchanger01.conf
-rw-r--r-- 1 root root 3754 May  5 10:12 /etc/vchanger/vchanger01.conf

[root@deneb ~]# ls -l /var/spool/bacula
total 8
-rw-r----- 1 bacula bacula 196 May  5 14:10 bacula-dir.9101.state
-rw------- 1 bacula bacula   0 May  4 14:22 bacula-dir.conmsg
-rw-r----- 1 bacula tape   196 May  5 14:10 bacula-sd.9103.state

[root@deneb ~]# ls -l /var/spool/vchanger
total 4
drwxr-x--- 2 bacula tape 4096 May  5 15:17 vchanger01

[root@deneb ~]# ls -l /var/spool/vchanger/vchanger01
total 12
-rw-r----- 1 bacula tape 47 May  5 15:17 bay_state-0
-rw-r----- 1 bacula tape 48 May  5 15:17 bay_state-1
-rw-r----- 1 bacula tape 17 May  5 15:17 dynamic.conf





------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z


_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users