Bacula-users

Re: [Bacula-users] [Bacula-devel] Minor SD Feature Request

2008-05-11 05:38:06
Subject: Re: [Bacula-users] [Bacula-devel] Minor SD Feature Request
From: Kern Sibbald <kern AT sibbald DOT com>
To: bacula-devel AT lists.sourceforge DOT net
Date: Sun, 11 May 2008 11:37:53 +0200
Hello,

OK, thanks. You have confirmed what I suspected.  In effect, this is really a 
support problem - I suspect you not fully understanding how Bacula works and 
its limitations (explained below).

First, without some additional design and coding, it is not possible for 
Bacula to snoop around on the autochanger for an available slot in which to 
unload a volume.  The autochanger may have hundreds of slots, with only a few 
available for Bacula, and currently there is no way to tell Bacula that 
it "owns" slots n-m  (this could be a future enhancement).  As a consequence, 
with the current design, Bacula must always unload a volume into the slot 
from which it came.

Second, from the above you should have gathered that if you manually load a 
volume into a slot where Bacula has loaded a volume from that slot into a 
drive, at some point everything is going to fail as you are seeing. 

When you change something in the autochanger the preferred way of doing so is:

-- first unmount all drives that Bacula has mounted
-- change the autochanger volumes
-- do an update slots
-- finally remount the drives with Bacula.  

It is possible to rearrange the volumes in the autochanger without unloading 
all the drives providing that Bacula doesn't want to load/unload any volume 
while you are changing things in in the autochanger.  I strongly recommend 
against doing this, but it is possible in a situation where Bacula is running 
a job.  Doing so is not without risks though.

If you don't follow these simple rules, Bacula will sooner or later fail, and 
probably the worst case is if you load a volume into a slot where there is a 
volume in one of the drives.

I do believe that we could improve how Bacula handles Volumes found in Slots 
where they are not expected, and I will look at that, but for the moment, 
having Bacula unload a volume into a different slot than from where it came 
is a much bigger project that if well designed and accepted would be a 
feature after the next major release (3.0.0).

Summary:
- I cannot accept your Feature Request as formulated without additional design 
work so that it won't break shared autochangers.

- You can resolve your problems by implementing improved sysadmin procedures.

Regards,

Kern

PS: When unmounting, you do specify an Autochanger, but since autochangers may 
have multiple drives, you must specify which drive of the autochanger.  If 
you have only one drive, entering a return at the question is all that is 
necessary to do the right thing.

On Saturday 10 May 2008 19:22:47 Blake Dunlap wrote:
> > Hello Blake,
> >
> > One part of Bacula that I would like to improve just a bit (not too much
> > coding for the moment) for the next release is the information returned
> > for
> > Autochangers.  Currently, it seems to me that the sysadmin has very
> > little information about the actual state of the autochanger via the
> > console interface.  Although your suggestion seems to be a bit more than
> > simple reporting of the status, I am interested in it.  The problem is
> > that I don't
> > understand what you are asking for well enough to possibly implement
> > something.
> >
> > Could you be much more explicit with what you want, perhaps giving an
> > explicit
> > example of what happens now and what you would like to see happen.  Don't
> > forget that at the current time, Bacula has no concept of changing the
> > slot -- for example, when a Volume is loaded by Bacula from Slot 2 into
> > the
> > drive, it *must* be returned to the same Slot.  Changing this behavior is
> > a
> > project that would require significant design and thought and is probably
> > not
> > something we would want to implement in the near future.
> >
> > On the other hand, I think there is a lot of need and possibility for
> > making
> > Bacula much smarter at automatically recognizing that a Volume is in a
> > different Slot from what is written in the database.  Currently such
> > volumes
> > are marked in error (if I remember right), but we could consider simply
> > correcting the info in the database.
> >
> > Best regards,
> >
> > Kern
>
> It is the last paragraph that I am mostly looking at dealing with. Let me
> give our situation in depth and I think that will explain what I am looking
> for.
>
> We have a 2 drive auto-changer and run 4 pools of backups (Incremental,
> OnSiteFull, OffsiteFull, and OnsiteMonthly). We run two sets of backups for
> clients, an offsite backup that runs every Friday night (due to the lack of
> copy pools etc), and the OnSite backups which occur every night
> incremental, except Saturday night which is a full (the pool is overridden
> to Monthly the first sat of a month). Anyway we rotate the Offsite tapes
> every Tuesday, and supposedly there is an update slots run with all drives
> released at the conclusion of the procedure which should update the
> database as to the current state of the auto-changer.
>
> Now that the back story is established, what has been extremely frustrating
> is that a decent percentage of the time, something occurs which places the
> tapes out of sync, and come Saturday night (the first night a drive would
> have to swap) the auto-changer fails to load a new tape it is looking for
> in the OnsiteFull pool, due to the tape that was in the drive failing to
> unload due to a slot full condition. Bacula now requests user intervention
> loading the tape, and the drive is marked unloaded (because the error
> didn't occur during an unload event, but a load event, which makes it a
> pain to determine what tape is actually loaded in the drive currently). To
> fix this, one must run an update slots, then look back in the logs to
> figure out what tape failed to unload, then "load" that tape into the
> drive, and Bacula will then realize the drive is usable again, and then
> proceed as normal. Of course due to the times we run backups, this has to
> occur in the middle of the night, or pot entially the next day which
> impacts backups, and the general network.
>
> I believe this is an error condition that could reasonably be dealt with
> programmatically instead of requiring user intervention (An automatic slot
> refresh before unloading tapes / loading tapes (with an assumed lifetime
> validity of say 10 minutes to reduce occurrences) would be one solution).
>
> Let me know if I need to add anything further, as I tried to be as detailed
> as possible in this response, as compared to the quick summary of the
> actual feature request. From a user prospective, I do agree that
> auto-changer support feels more tacked on than anything (for example, the
> requiring to specify a drive instead of an auto-changer when doing an
> update slots command) and would love to see improvements in that regard.
>
> -Blake
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/java
>one _______________________________________________
> Bacula-devel mailing list
> Bacula-devel AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-devel



-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users