Bacula-users

[Bacula-users] Rif: Re: Rif: Virtual backup

2008-09-04 03:31:13
Subject: [Bacula-users] Rif: Re: Rif: Virtual backup
From: Ferdinando Pasqualetti <fpasqual AT ccci DOT it>
To: Kern Sibbald <kern AT sibbald DOT com>
Date: Thu, 4 Sep 2008 09:30:54 +0200
bacula-users-bounces AT lists.sourceforge DOT net wrote on 03/09/2008 18.49.14:

> On Wednesday 03 September 2008 14:50:56 Ferdinando Pasqualetti wrote:
> > Hello Kern,
> > thanks one more time for your work.
> > I am trying to contribute to this discussion, even if I am sure that 
my
> > knowledge is too weak.
> > In my opinion you should allow to set a Next Pool also in a Virtual 
Backup
> > pool (maybe also for Migration or Copy pool). This could be managed in 
a
> > number of ways, i.e. you could set the Next Pool of the third pool to 
the
> > second one, thus alternating the destination pools of the virtual 
backup.
> > After the backup virtualization the source pools could be purged to 
make
> > room for the next run.
> 
> There might indeed be some way to make this work, but I cannot 
> clearly see how 
> it would work automatically for multiple virtual backups. 
> 
> It would be very useful if you could show an explicit example with 
volumes 
> (with specific names) in specific pools and how multiple virtual backups 

> could work without eventually getting into a deadlock where Bacula wants 
to 
> read and write the same volume or on the same device.

Maybe my ideas are note completely clear, but in my opinion Virtual Backup 
should be defined starting from the Job definition of a real Backup and 
should be only of level Full or Differential. So in the Job definition for 
a virtual backup the source Job name should be stated or, maybe better, 
the virtual backup directly defined in a Run for that job.
This should lead to the relevant job IDs to be scanned and to their pool 
names. Once done the virtual backup should become a standard backup for 
that job name (either Full or Differential).
In order to be consistent with copy and migration the destination pool 
should be taken from Next Pool of the pool used in the first job ID (IMO 
it would be clearer if the destination pool would be directly stated, also 
in copy and migration; I believe that you did not use this choice because 
in order to guarantee different input and output pool).
So, trying a definition for a real situation we could have this scenario:

Job {
  Name = Save
  Level = Full
  ......
  Pool = FullPool
  ......
}
Pool {
  Name = FullPool
  Pool Type = Backup
  Next Pool = VirtPool1
  Label Format = "..."
  .....
}
Pool {
  Name = FullPool
  Pool Type = Backup
  Next Pool = VirtPool1
  Label Format = "..."
  .....
}
Pool {
  Name = IncPool
  Pool Type = Backup
  Next Pool = VirtPool1
  Label Format = "..."
  .....
}
Pool {
  Name = VirtPool1
  Pool Type = Backup
  Next Pool = VirtPool2
  Label Format = "..."
  .....
}
Pool {
  Name = VirtPool2
  Pool Type = Backup
  Next Pool = VirtPool1
  Label Format = "..."
  .....
}
Schedule {
  Name = SaveSched
  Run = Full 24-12 at 20:00
  Run = Incremental Pool = IncPool mon-fri at 22:00
  Run = Differential Type = virtual sat at 2:00
  Run = Full Type = virtual last sat at 8:00
}

We could avoid the use of VirtPool2, setting directly FullPool as NextPool 
of VirtPool1, but maybe it could be useful to have different media for 
real and virtual backup.

Sorry if something in this construct is completely wrong, I just tried to 
contribute.

Best regards,
 
 
--------------------------------------------------------------------------
Ferdinando Pasqualetti
G.T.Dati srl
Tel. 0557310862 - 3356172731 - Fax 055720143


> 
> Regards,
> 
> Kern
> 
> > Best regards,
> > 
--------------------------------------------------------------------------
> > Ferdinando Pasqualetti
> > G.T.Dati srl
> > Tel. 0557310862 - 3356172731 - Fax 055720143
> >
> > bacula-users-bounces AT lists.sourceforge DOT net wrote on 03/09/2008 
11.37.57:
> > > Hello,
> > >
> > > As many of you know Virtual Backup (consolidation, synthetic full, 
...)
> >
> > is a
> >
> > > new feature that is implemented in the development trunk and 
scheduled
> >
> > to be
> >
> > > released later this year.  It essentially copies what would be a 
"full
> > > current" restore to a new Volume thus creating an virtual backup 
that
> >
> > can
> >
> > > serve as a Full backup.  This has a lot of advantages, particularly 
for
> >
> > sites
> >
> > > with full backups that run long times or for remote sites where the 
time
> >
> > to
> >
> > > transmit a full backup is excessive.
> > >
> > > The Virtual Backup feature works much like Migration and Copy.  It 
reads
> >
> > from
> >
> > > the required Volumes and writes to a Volume specified in the pool as
> >
> > "Next
> >
> > > Pool".  This ensures that the read and write Volumes are different.
> > >
> > > Everything seems to work fine with the Virtual Backup.  However,
> >
> > thinking
> >
> > > about longer term operations, it has occurred to me that when you 
want
> >
> > to
> >
> > > make a second Virtual Backup things will become very complicated. 
First,
> >
> > the
> >
> > > Virtual backup will want to read the previous Virtual backup 
volume,and
> >
> > then
> >
> > > if that Volume is not full, it will want to write to the same 
Volume.
> >
> > Even
> >
> > > if the volume is full, you will be in a situation where the Job will
> >
> > want to
> >
> > > read and write to volumes in the same pool.  In all those cases, 
there
> >
> > is no
> >
> > > guarantee that there will not be a deadlock situation (actually 
Bacula
> > > currently cancels any job attempting to read and write from the same
> >
> > Storage
> >
> > > device).
> > >
> > > I am not 100% sure what to do to resolve this issue.  I suppose one
> >
> > could run
> >
> > > a Migration job to "move" the Virtual Backup back to the Pool from 
which
> >
> > it
> >
> > > originally came, then the next Virtual Backup would work fine (the 
same
> >
> > as
> >
> > > the first one), but that sounds a bit kludgie.
> > >
> > > If anyone has any suggestions, I would appreciate to hear them. 
However,
> > >
> > > suggestions that require implementing significant amounts of code or
> >
> > complex
> >
> > > new algorithms such as deadlock detection won't be very helpful 
since
> >
> > there
> >
> > > is no time left to do such implementations between now and release 
time.
> >
> >  In
> >
> > > addition, deadlock detection won't help, what we really need is 
deadlock
> > >
> > > resolution, and that is an even more difficult subject.
> > >
> > > Best regards,
> > >
> > > Kern
> >
> > 
-------------------------------------------------------------------------
> >
> > > This SF.Net email is sponsored by the Moblin Your Move Developer's
> >
> > challenge
> >
> > > Build the coolest Linux based applications with Moblin SDK & win 
great
> >
> > prizes
> >
> > > Grand prize is a trip for two to an Open Source event anywhere in 
the
> >
> > world
> >
> > > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > > _______________________________________________
> > > Bacula-users mailing list
> > > Bacula-users AT lists.sourceforge DOT net
> > > https://lists.sourceforge.net/lists/listinfo/bacula-users
> 
> 
> 
> 
-------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's 
challenge
> Build the coolest Linux based applications with Moblin SDK & win great 
prizes
> Grand prize is a trip for two to an Open Source event anywhere in the 
world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users