Bacula-users

Re: [Bacula-users] [Bacula-devel] Virtual backup

2008-09-04 12:27:13
Subject: Re: [Bacula-users] [Bacula-devel] Virtual backup
From: Kern Sibbald <kern AT sibbald DOT com>
To: Josh Fisher <jfisher AT pvct DOT com>
Date: Thu, 4 Sep 2008 18:25:53 +0200
On Thursday 04 September 2008 17:01:55 Josh Fisher wrote:
> In find_next_volume_for_append() in next_vol.c, there is a check to make
> sure the volume selected has not expired. Would it be possible to also
> add a check to make sure the volume is not in jcr->VolList? Assuming the
> list of input volumes for a virtual full job is in jcr->VolList and has
> been built by the time the next appendable volume is searched for, of
> course. The idea being to simply exclude any input volumes from being
> selected as an output volume.

Yes, that is a very good suggestion for eliminating the problems on the Volume 
level. Thanks.  

There is still the annoying issue that currently the SD cannot switch the 
output device once it is chosen, but I think your idea at least ensures we 
won't be trying to write to any volume that is going to be read.  It will, of 
course, require a bit more code, but the needed code should be relatively 
trivial to implement.

There is still an issue that to make it work right, Bacula will need to be 
able to read from and write to volumes in the same pool and ensure that the 
read device and the write device are not the same.  However, I think those 
are also possible to implement relatively simply.

Thanks for the suggestion.  I'll see if it is possible to make it work :-)

Kern

>
> Kern Sibbald wrote:
> > Hello,
> >
> > As many of you know Virtual Backup (consolidation, synthetic full, ...)
> > is a new feature that is implemented in the development trunk and
> > scheduled to be released later this year.  It essentially copies what
> > would be a "full current" restore to a new Volume thus creating an
> > virtual backup that can serve as a Full backup.  This has a lot of
> > advantages, particularly for sites with full backups that run long times
> > or for remote sites where the time to transmit a full backup is
> > excessive.
> >
> > The Virtual Backup feature works much like Migration and Copy.  It reads
> > from the required Volumes and writes to a Volume specified in the pool as
> > "Next Pool".  This ensures that the read and write Volumes are different.
> >
> > Everything seems to work fine with the Virtual Backup.  However, thinking
> > about longer term operations, it has occurred to me that when you want to
> > make a second Virtual Backup things will become very complicated.  First,
> > the Virtual backup will want to read the previous Virtual backup volume,
> > and then if that Volume is not full, it will want to write to the same
> > Volume.  Even if the volume is full, you will be in a situation where the
> > Job will want to read and write to volumes in the same pool.  In all
> > those cases, there is no guarantee that there will not be a deadlock
> > situation (actually Bacula currently cancels any job attempting to read
> > and write from the same Storage device).
> >
> > I am not 100% sure what to do to resolve this issue.  I suppose one could
> > run a Migration job to "move" the Virtual Backup back to the Pool from
> > which it originally came, then the next Virtual Backup would work fine
> > (the same as the first one), but that sounds a bit kludgie.
> >
> > If anyone has any suggestions, I would appreciate to hear them.  However,
> > suggestions that require implementing significant amounts of code or
> > complex new algorithms such as deadlock detection won't be very helpful
> > since there is no time left to do such implementations between now and
> > release time.  In addition, deadlock detection won't help, what we really
> > need is deadlock resolution, and that is an even more difficult subject.
> >
> > Best regards,
> >
> > Kern
> >
> >
> > -------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Moblin Your Move Developer's
> > challenge Build the coolest Linux based applications with Moblin SDK &
> > win great prizes Grand prize is a trip for two to an Open Source event
> > anywhere in the world
> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________
> > Bacula-devel mailing list
> > Bacula-devel AT lists.sourceforge DOT net
> > https://lists.sourceforge.net/lists/listinfo/bacula-devel



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users