Hello,
I think what you are asking for is probably a much bigger issue than you might
imagine. Hopefully what follows will give a rough overview the issues.
The scheme you are suggesting has a number of things that worry me:
1. As a general principle, it is not a good idea to be cycling through storage
daemons seeing if they work or not, which is what I think you are proposing
unless I misunderstood it.
2. Under the current design, the Director *must* first connect to the SD then
connect to the FD. This is required to prepare the SD for accepting a
communication from the FD. Changing this would require a major design review
as it involves the low level details of ensuring security, and before
undertaking such a change, I would really need to be convinced of the need to
make such a change ...
3. I am not convinced that any given bootstrap file should contain a reference
to more than one SD or what it would mean.
4. Any change in how the Storage daemon is specified needs to be carefully
thought out, because the current specification allows for multiple storage
devices (within one storage daemon) to be specified -- this is already
implemented and I believe it works, but it is not documented, and it also
permits specifying multiple storage daemons, but the code for this is
probably not complete and not working. (see below for more "specifics").
5. I have always planned to have SD <-> SD communications. For example a
migration job that migrates from one SD to another. Any addition of Storage=
in bsr files would need to be carefully designed not to interfere with this.
If this feature is properly implemented, it might even be capable of giving
you what you want -- in fact, it goes more in the direction that David Boyes
has been advocating for some time to have a sort of Virtual Storage Manager
or Virtual Volume Manager interposed between the Director and the Storage
daemon(s).
On items that don't worry me:
1. I don't see any particular problem with adding Storage=xx to the bootstrap
file. IMO, it probably will not help at all for the problem you are trying
to solve since the Dir already knows during most restores which SD is needed,
but it could serve to make a bootstrap file more self-contained for disaster
recovery where no catalog may be available.
Bottom line:
I don't have any problem supporting multiple storage daemons and would welcome
anyone who wants to work on it, but before beginning anything, we need a very
detailed careful design that looks at the current capabilities (including the
undocumented features) and shows how we are going to evolve those gracefully
into implementation of multiple storage daemons.
To be a bit more specific, one must consider within say a Pool or a Job
resource what:
storage = File, DLT-Changer, ...
means
and what
storage = File
storage = DLT-Changer
...
means. Currently the semantics are totally different and as I mentioned, not
documented. Even I am not sure exactly what works and doesn't without
carefully examining the code.
Then one would need to make sure that any extension to a bsr file is
consistent with generalizing Bacula's ability to deal with multipe devices
and or Storage daemons -- I believe that the issue of dealing with multiple
devices is far more important since Bacula can already deal with multiple
storage daemons providing one follows a few simple, but not too constraining
rules (principally that a given Job name be associated with only one Storage
daemon).
Another thing that would be helpful is to specify on a much higher level what
feature you are trying to implement. You've proposed a somewhat low level
solution to a problem of using Bacula in a way that is either not possible or
hasn't been fully implemented, and by starting from the high end, we might be
able to make it work by simplely completing some of the features that have
been planned and partially implemented but not completed for many years now.
Regards,
Kern
On Thursday 12 March 2009 17:30:21 Graham Keeling wrote:
> Hello,
> I may implement this myself, after people have commented.
> Thanks,
> Graham.
>
>
> Item n: Restore from volumes on multiple storage daemons
>
> Origin: Graham Keeling (graham AT equiinet DOT com)
>
> Date: 12 March 2009
>
> Status: Proposing
>
> What: The ability to restore from volumes held by multiple storage daemons
> would be very useful.
>
> Why: It is useful to be able to backup to any number of different storage
> daemons. For example, your first storage daemon may run out of space, so
> you switch to your second and carry on. Bacula will currently let you do
> this. However, once you come to restore, bacula cannot cope when volumes on
> different storage daemons are required.
>
> Notes: The director knows that more than one storage daemon is needed, as
> bconsole outputs something like the following table.
>
> The job will require the following
> Volume(s) Storage(s) SD Device(s)
> ===========================================================================
>
> backup-0001 Disk 1 Disk 1.0
> backup-0002 Disk 2 Disk 2.0
>
> However, the bootstrap file that it creates gets sent to the first storage
> daemon only, which then stalls for a long time, 'waiting for a mount
> request' for the volume that it doesn't have.
> The bootstrap file contains no knowledge of the storage daemon.
>
> I think that restoring from multiple storage daemons can be achieved as
> follows:
>
>
> Add 'Storage=' lines to the bootstrap file.
>
> Currently, the director connects to the sd first, and then the fd.
> Since we need to go through the storages one by one, it should connect to
> the fd first.
>
> The director then reads the bootstrap file, looking for the storages.
> The existing mechanism for authorising and restoring from a bootstrap file
> is then reused once for each storage, but the director sends only the
> relevant section of the bootstrap file to the fd.
> After each storage, the fd tells the director that it has finished. Once
> the director finishes reading the bootstrap file, it tells the fd to end
> the restore.
> The fd then runs the 'run after' scripts.
>
>
>
> With the idea above, the sd code is not changed.
>
> The director migrate, vbackup, and verify code also suffer from the same
> problem to do with reading from multiple storage daemons.
> However, once the restore is fixed, I believe that it would be very simple
> to update those parts too, as they do not need to contact a file daemon.
>
>
> ---------------------------------------------------------------------------
>--- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> _______________________________________________
> Bacula-devel mailing list
> Bacula-devel AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-devel
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
