Bacula-users

Re: [Bacula-users] bacula-sd file storage question

2017-01-10 10:59:50
Subject: Re: [Bacula-users] bacula-sd file storage question
From: Josh Fisher <jfisher AT pvct DOT com>
To: Timo Neuvonen <timo-news AT tee-en DOT net>, bacula-users AT lists.sourceforge DOT net
Date: Tue, 10 Jan 2017 10:59:32 -0500
On 1/10/2017 4:49 AM, Timo Neuvonen wrote:
>
> I'm not trying to achieve anything special, I'm rather trying to avoid doing
> anything unnecessarily complicated.
> I just want to set up (and  to fully understand my set-up) a simple but
> working file storage device for a small business / home office environments,
> with no more than just a few concurrent jobs.
> By now, I've used Bacula for a decade with tape drives, but only now I'm
> about to switch over to disk-based backup.
>
>
> What confused me is that the example conf files supplied with current Bacula
> releases don't give an example of the very basic disk-based setup, but just
> show how to setup a virtual disk autochanger.
> The whitepapers that Kern referred gave me an impression that the virtual
> autochanger setup is primarily for high-end applications where may be
> thousands of concurrent jobs, and where it is essential to have a more
> precise control on how concurrent jobs simultaneously use several volumes.
> I hope I got this right?
>
> So I guess that within my small sandbox I'll be fine with just one file
> storage device :-)

More precisely, that is 'one storage device at a time'. For example, 
with RDX or USB drives, or hot-swap SAS/SATA, you may have several 
devices that are used for backup volume storage, but only one at a time 
can be used with s particular SD-defined Device resource. Of course, for 
fixed disk, it truly is a single storage device. I would not recommend 
using fixed disk, unless it is used as an intermediate backup storage in 
a disk-disk-tape type setup. I guess RAIDed fixed disk might be OK, but 
it is important that there be at least two copies of backups on separate 
media to protect from media failure.

The simplest approach is just a single Device resource in 
bacula-sd.conf, such as:

Device {
   Name = FileStorage
   Media Type = File
   Archive Device = /mnt/backup
   LabelMedia = yes;
   Random Access = Yes;
   AutomaticMount = yes;
   RemovableMedia = no;
   AlwaysOpen = no;
}

Let's say you want to use USB drives. You would then mount, whether 
manually or through udev rules, one USB drive at a time at /mnt/backup. 
This will allow an unlimited number of volume files by using an 
unlimited number of USB drives, one at a time. However, since they can 
only be mounted one at a time, only a subset of those volumes can be 
made available at any one time. If a job needs volumes from more than 
one USB drive, then manually intervention will be needed.

Only one SD Device resource means that only one volume may be used at a 
time. It is still possible to run concurrent jobs, however those 
concurrent jobs will interlace their data onto the same volume file. 
That can make restores slower, but other than that is not too much of an 
issue. It is also possible to turn on data spooling, which will spool 
concurrent jobs to a temporary spool directory on fixed disk, then 
despool the jobs one at a time to the USB drive, eliminating the 
interlacing. It depends on the media being used. Generally, USB drives 
are far too slow to support concurrent jobs, and it is just as fast to 
run one job at a time in the first place.

For faster storage devices like RDX, hot-swap SATA, or if using Virtual 
Full jobs, or if the ability to restore while other backups are 
occurring is needed, then it is necessary to utilize more than one 
volume file concurrently. That is where the virtual autochanger comes 
in. The autochanger allows a using multiple Device resources as a single 
logical resource known as a Autochanger resource. With the virtual 
autochanger, there is still only one mountpoint, so one physical device. 
Each Device resource associated with the Autochanger resource defines 
the ArchiveDevice to the same directory path, and so reads/writes volume 
files in the same (shared) mountpoint directory. But since there are 
multiple Device resources, and each can have a volume file open, the 
virtual autochanger is a single logical resource that can have more than 
one volume file open at a time.

To use tape terminology, a virtual autochanger is similar to a tape 
autoloader with a library consisting of a single tape caddie containing 
available tapes (the volume files in the ArchiveDevice directory) and 
multiple tape drives (the Device resources associated with the 
Autochanger resource). Each drive can hold one tape, and a particular 
tape can only be loaded into one drive at a time. Each physical disk is 
a tape caddie containing volumes, and only one caddie at a time can be 
loaded into the library. The simpler single Device resource (no 
autochanger) approach is like a tape autoloader with a single-caddie 
library and having only a single tape drive. Any tape in the caddie can 
be used, but only one at a time.

Then there is vchanger, which uses a different method of implementing a 
virtual autochanger. It also uses an Autochanger resource with any 
number of associated Device resources, but each Device resource has a 
different ArchiveDevice that points to a dynamically generated symlink. 
This allows the use of more than one mountpoint, so multiple physical 
disks at a time. Again with tape terminology, it simulates a tape 
autoloader with multiple drives, like the built-in virtual autochanger, 
but with a multi-caddie tape library instead of a single caddie tape 
library.

And last, but not least, a future version of Bacula will support a Cloud 
storage resource, which I think should allow read/write to an unlimited 
number of volumes stored in cloud storage, such as MyCloud.


>
>
> Kern and Dimitri, thank you for the answers!
>
> Regards,
> Timo
>
>
>
> ------------------------------------------------------------------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today. http://sdm.link/xeonphi
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users