Bacula-users

Re: [Bacula-users] [Bacula-devel] Copy jobs in Bacula version 3.0.0

2008-12-16 10:52:13
Subject: Re: [Bacula-users] [Bacula-devel] Copy jobs in Bacula version 3.0.0
From: Josh Fisher <jfisher AT pvct DOT com>
To: Kern Sibbald <kern AT sibbald DOT com>
Date: Tue, 16 Dec 2008 10:49:55 -0500
Kern Sibbald wrote:
> Hello,
>
> I've been discussing with Eric how we might handle Copy jobs in our 
> development version.  Currently, Copy jobs are implemented, and they work 
> much like Migration jobs (share 99% of the code).  The difference is that 
> Migration jobs purge the original backup job and keep only the Migrated data. 
>  
> With a Copy Job, the original backup job remains and there is a 
> second "identical" job that contains the copied data.  The only difference 
> between the original and the Copy job is that they will be in different 
> Pools.
>
> Now this poses a few problems for doing restores such as:
>
> 1. It is possible that a simple restore will choose JobIds from both the 
> original and the Copy Job.
>
> 2. There is no easy mechanism for the user to select whether he/she wants to 
> restore from the original backup or the Copy (or Copies).
>
> So for the moment, the situation is not really satisfactory (one of the 
> reasons the code is not yet released).
>
> We have a number of ideas for different ways to solve the above problems, 
> many 
> have already been discussed on the mailing lists, and we will probably 
> implement a number of the ideas put forward, either before or after the 
> release (depending on the time we have and the complexity of the proposal -- 
> e.g. using the Location table and Costs ...).
>
> A few things seem obvious:
>   

Maybe, but I am still trying to understand. :)  My thought is that a 
Copy job is just that, a copy of a real Backup job, or in other words, a 
"backup of a backup", as for example an offsite backup. So I am inclined 
to think that to "restore" from a Copy job is to restore the Backup job, 
rather than the client machine. In this scenario, nothing at all would 
change about the client restore job. Instead, a restore from a Copy job 
would restore the original Backup job. A client would always be restored 
from an "original" Backup job, though the "original" may actually have 
been restored from a Copy. My reasoning is that a Copy job would never 
be needed unless the original Backup job was somehow lost, in which case 
there would be no need to ask the operator which to restore from. 
Perhaps my reasoning is flawed, though, because I'm not sure I "get it".

> 1. Any restore where Bacula automatically selects the jobs to be restored 
> (e.g. a restore to the current state -- #5 on the restore prompt menu) should 
> be done by default using the original backups.
>
> 2. If a job has been copied, Bacula should probably display an information 
> message during the restore that indicates that the JobIds to be used have 
> Copies.
>
> 3. The restore command should allow the user to select any Copy job or jobs.
>
> I am proposing to change the current way that Copy jobs are handled.  Below 
> is 
> partial output from a list of Jobs run (a Copy regression test):
>
> +-------+--------------+---------------------+------+-------+----------+-----------+-----------+
> | JobId | Name         | StartTime           | Type | Level | JobFiles | 
> +-------+--------------+---------------------+------+-------+----------+-----------+-----------+
> | 1     | CopyJobSave  | 2008-12-15 20:38:28 | B    | F     | 7020     | 
> | 5     | CopyJobSave  | 2008-12-15 20:38:28 | B    | F     | 7020     | 
> | 2     | CopyJobSave  | 2008-12-15 20:38:32 | B    | I      | 999     | 
> | 6     | CopyJobSave  | 2008-12-15 20:38:32 | B    | I      | 999     | 
> | 3     | copy-job     | 2008-12-15 20:41:05 | C    | F     | 0        | 
> | 4     | copy-job     | 2008-12-15 20:41:50 | C    | F     | 0        | 
> | 8     | RestoreFiles | 2008-12-15 20:42:39 | R    | F     | 7020     | 
> +-------+--------------+---------------------+------+-------+----------+-----------+-----------+
>
> JobIds 1 and 2 are original full backups. 
> JobIds 5 and 6 are copies of the original JobIds 1 and 2 respectively.
> JobIds 3 and 4 are the jobs that ran the Copy jobs.
>
> Notice that JobIds 5 and 6 look identical to their original jobs 1 and 2 
> despite the fact that they were actually run at the time of job 3 and 4.
>
> I propose that we modify the Jobs to look like the following:
>
> +-------+--------------+---------------------+------+-------+----------+-----------+-----------+
> | JobId | Name         | StartTime           | Type | Level | JobFiles | 
> +-------+--------------+---------------------+------+-------+----------+-----------+-----------+
> | 1     | CopyJobSave  | 2008-12-15 20:38:28 | B    | F     | 7020     | 
> | 5     | CopyJobSave  | 2008-12-15 20:38:28 | C    | F     | 7020     | 
> | 2     | CopyJobSave  | 2008-12-15 20:38:32 | B    | I      | 999     | 
> | 6     | CopyJobSave  | 2008-12-15 20:38:32 | C    | I      | 999     | 
> | 3     | copy-job     | 2008-12-15 20:41:05 | c    | F     | 0        | 
> | 4     | copy-job     | 2008-12-15 20:41:50 | c    | F     | 0        | 
> | 8     | RestoreFiles | 2008-12-15 20:42:39 | R    | F     | 7020     | 
> +-------+--------------+---------------------+------+-------+----------+-----------+-----------+
>
> Now here, JobIds 5 and 6 no longer appear to Bacula like they are Backup 
> jobs, 
> rather they are marked as Type=C (i.e. a Copy job), and so they will never be 
> considered for restoration by default.  I've give the Copy control jobs the 
> Type 'c' to distinguish them from the real copy -- they exist just to record 
> the actual time the copy was made.
>
> Now by modifying the restore code in Bacula, we should be able to provide 
> features that allow the user to know that there is a Copy of JobId 1 (i.e. 5) 
> and a copy of JobId 2 (i.e. 6) and allow him/her to choose which copy to use.
>
> In my opinion, this would simplify future handling of Copy jobs allowing a 
> lot 
> more flexibility and avoiding confusion.  However, it will cause some minor 
> changes (nothing serious, I believe) for users already using the new code.
>
> A restore might go like the following:
>
> It might go something like this:
>
> restore 
> (chose option 5 -- current for full restore)
>
> The following JobIds are selected for restore:
>
>  1, 2
>
> These JobIds have copies as follows:
>
> JobId      Copy JobId    Media Type    Date   Location ....
> ====================================
> 1            5                 Disk   ...
> 2            6                 Disk
>
>
> To use a copy, select JobId, Copy JobId
> 2,6
>
> The following JobIds have been selected for restore:
> 1,6
>
>
> ===
>
> A more complicated example might be:
>
> restore 
> (chose option 5 -- current for full restore)
>
> The following JobIds are selected for restore:
>
> 1, 2, 3, 4
>
> These JobIds have copies as follows:
>
> JobId      Copy JobId    Media Type    Date   Location ....
> ====================================
> 1            5                 Tape   ...
>               7                 Disk  (second copy of job 1)
> 2            6                 Disk
> 3            8                 Tape
> 4            None
>
>
> To use a copy, select JobId, Copy JobId
> 1,7 2,6
>
> The following JobIds have been selected for restore:
> 7,6,3,4
>
>
> Note, instead of selecting pairs (original Job, Copy), since all the jobids 
> are unique, you could simply just type in the desired jobids.  For the above 
> example: 7,6,3,4. 
>
> Any comments?
>
> Best regards,
>
> Kern
>
> ------------------------------------------------------------------------------
> SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
> The future of the web can't happen without you.  Join us at MIX09 to help
> pave the way to the Next Web now. Learn more and register at
> http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>   

------------------------------------------------------------------------------
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
The future of the web can't happen without you.  Join us at MIX09 to help
pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users