Bacula-users

Re: [Bacula-users] Configuration reload for bacula-sd

2014-11-12 14:18:42
Subject: Re: [Bacula-users] Configuration reload for bacula-sd
From: Josh Fisher <jfisher AT pvct DOT com>
To: Dmitri Maziuk <dmaziuk AT bmrb.wisc DOT edu>
Date: Wed, 12 Nov 2014 14:11:36 -0500
On 11/8/2014 1:35 PM, Dmitri Maziuk wrote:
> On 11/8/2014 5:36 AM, Kern Sibbald wrote:
>
>
>> By the way, in case you have not noticed, there are three white
>> papers posted on the bacula.org web site, two of them, if I am not
>> mistaken, document Virtual Autochangers as implemented in the
>> community version (slightly different from the Enterprise version).
> I notice now, thanks.
>
> CommunityDiskBackup.pdf has the same old problem in 2.1 Grouping Storage
> Devices: as usual, it talks multiple devices and shows 2 Device
> definitions with Archive Device = /disk in both. Personally I don't know
> how to mount multiple physical devices in the same /disk at the same
> time and keep them separate too.
>
> Same goes for 2.2 Virtual Autochanger: it doesn't let you write to
> multiple (disk) devices, it lets bacula treat a single Archive Device as
> "multipe devices" -- presumably for concurrent access.

I think there is some confusion about this due to "multiple devices" 
being an ambiguous term. There is a difference between "multiple devices 
that are only attached one at a time" and "multiple devices that may be 
attached all at the same time". For disk storage, including virtual disk 
autochanger, the "device" is a directory, and in almost all cases it is 
in fact the mount point of a filesystem partition. The virtual 
autochanger described in 2.2 will work with any filesystem mounted at 
/disk, so multiple disk drive partitions CAN be used, but ONLY one at a 
time. This then works similarly to tape. When the disk drive partition 
gets full, you must unmount the current disk partition and mount the 
next one. Note that the "device" is really a filesystem. The filesystem 
itself could be spread across multiple physical disk drives, but that is 
another story.

> Same goes for section 3 (Multiple Devices) in CommunityDiskBackupDesign.pdf
>
> At least that's what I read in there. Maybe I'm not reading it right:
> the years have not been kind and my leet reading comprehension skillz
> ain't what they used to be.
>
> The rest of it is very useful but not germane to the topic of using
> multiple hot-swappable disks for storage.

There are really (at least) two separate issues going on here. One is 
utilizing multiple filesystems simultaneously. Another is using 
hot-swappable disk drives. I'll start with using hot-swappable drives.

Bacula has thus far not given a great deal of consideration to 
hot-swappable disk drives. Nevertheless, the hooks are there for 
mounting and unmounting as needed via the Requires Mount, Mount Command, 
and associated directives in the Device resource config. There are 
caveats, the most obvious being that the SD must either run as root or 
else have sudo configured to allow the SD user to mount the 
hot-swappable drives. The not so obvious caveat is that in order to use 
hot-swappable drives with a disk Device you MUST use Requires Mount, 
particularly if you are using automatic volume labeling. Otherwise, if 
no filesystem is mounted at the Archive Device path, then Bacula will 
create a volume file on the root filesystem. With Requires Mount = yes, 
Bacula will block waiting for a mount request if no filesystem is 
mounted at the Mount Point path. Yes, you could mount the hot-swappable 
disks using udev and then run SD as a normal user, but if you do so, 
then Bacula will write to the root filesystem when no hot-swappable 
drive is currently mounted.

The other issue, using multiple simultaneously mounted filesystems, is 
not so clear. Each Device resource associated with an autochanger can 
specify a different Archive Device path. To do so, it is also necessary 
to use a different Media Type for each filesystem. Bacula expects to be 
able to load any volume of a particular Media Type into any Device 
having that same Media Type. A volume file located in one Device's 
Archive Device directory cannot be "loaded" into another Device with a 
different Archive Device directory, and so each filesystem must have its 
own Media Type. In my testing, the SD seems to support this well. What 
is not so clear to me is how to then define the associated Storage 
resource in the Director. As far as I can tell, a Storage resource can 
only use one Media Type, so a SD Autochanger resource using multiple 
Media Types does not seem to be usable as a single Storage resource in Dir.

> This is all great but so far the only known config I've actually seen
> working is a vchanger. I'd consider buying it off and rolling it into
> bacula proper so instead of much handwaving and allegations of "maybe
> documented somewhere" features you'd have a definitive and working
> answer for retentive assholes like me.

As the author of vchanger, I can tell you a little about it. First, it 
was developed to use multiple USB drives as a virtual autochanger at the 
time of Bacula version 1.39.x in 2006. It is open source (GPL v2) and 
available on SourceForge, so no "buying it off" is needed.

At the time vchanger was developed Bacula did not have the virtual 
autochanger code. My suggestion is to use the native virtual autochanger 
if you are backing up to fixed disk or else one removable disk at a 
time. Otherwise, use vchanger for multiple simultaneously attached 
removable disks. There will never be automatic volume labeling in the 
sense of using Label Media=yes, but there is automatic barcode generation.

It just so happens that I have recently had some time to update 
vchanger. I will hopefully be pushing it to SourceForge in the next week 
or so as version 1.0.0. It has been working without issues for me for a 
few weeks with USB drives, but the documentation updates are not yet 
complete. The new version eliminates the artificial limitations on the 
number of virtual drives and number of slots on a USB drive. The 
"magazine" filesystem now contain nothing but volume files and all state 
info and symlinks are kept in a per-autochanger subdirectory of 
/var/spool/vchanger. It also adds udev support and integration with 
Bacula via bconsole commands to automate the Update Slots and Label 
Barcodes commands, making swapping USB drives a true plug-n-play 
operation. My reasons for updating after all this time are I guess 
somewhat selfish. It worked for me, but now I have a need for more 
functionality for use with RDX drives.


> Dima
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users