Bacula-users

[Bacula-users] Copy jobs in Bacula version 3.0.0

2008-12-15 15:55:21
Subject: [Bacula-users] Copy jobs in Bacula version 3.0.0
From: Kern Sibbald <kern AT sibbald DOT com>
To: "bacula-devel" <bacula-devel AT lists.sourceforge DOT net>
Date: Mon, 15 Dec 2008 21:52:39 +0100
Hello,

I've been discussing with Eric how we might handle Copy jobs in our 
development version.  Currently, Copy jobs are implemented, and they work 
much like Migration jobs (share 99% of the code).  The difference is that 
Migration jobs purge the original backup job and keep only the Migrated data.  
With a Copy Job, the original backup job remains and there is a 
second "identical" job that contains the copied data.  The only difference 
between the original and the Copy job is that they will be in different 
Pools.

Now this poses a few problems for doing restores such as:

1. It is possible that a simple restore will choose JobIds from both the 
original and the Copy Job.

2. There is no easy mechanism for the user to select whether he/she wants to 
restore from the original backup or the Copy (or Copies).

So for the moment, the situation is not really satisfactory (one of the 
reasons the code is not yet released).

We have a number of ideas for different ways to solve the above problems, many 
have already been discussed on the mailing lists, and we will probably 
implement a number of the ideas put forward, either before or after the 
release (depending on the time we have and the complexity of the proposal -- 
e.g. using the Location table and Costs ...).

A few things seem obvious:

1. Any restore where Bacula automatically selects the jobs to be restored 
(e.g. a restore to the current state -- #5 on the restore prompt menu) should 
be done by default using the original backups.

2. If a job has been copied, Bacula should probably display an information 
message during the restore that indicates that the JobIds to be used have 
Copies.

3. The restore command should allow the user to select any Copy job or jobs.

I am proposing to change the current way that Copy jobs are handled.  Below is 
partial output from a list of Jobs run (a Copy regression test):

+-------+--------------+---------------------+------+-------+----------+-----------+-----------+
| JobId | Name         | StartTime           | Type | Level | JobFiles | 
+-------+--------------+---------------------+------+-------+----------+-----------+-----------+
| 1     | CopyJobSave  | 2008-12-15 20:38:28 | B    | F     | 7020     | 
| 5     | CopyJobSave  | 2008-12-15 20:38:28 | B    | F     | 7020     | 
| 2     | CopyJobSave  | 2008-12-15 20:38:32 | B    | I      | 999     | 
| 6     | CopyJobSave  | 2008-12-15 20:38:32 | B    | I      | 999     | 
| 3     | copy-job     | 2008-12-15 20:41:05 | C    | F     | 0        | 
| 4     | copy-job     | 2008-12-15 20:41:50 | C    | F     | 0        | 
| 8     | RestoreFiles | 2008-12-15 20:42:39 | R    | F     | 7020     | 
+-------+--------------+---------------------+------+-------+----------+-----------+-----------+

JobIds 1 and 2 are original full backups. 
JobIds 5 and 6 are copies of the original JobIds 1 and 2 respectively.
JobIds 3 and 4 are the jobs that ran the Copy jobs.

Notice that JobIds 5 and 6 look identical to their original jobs 1 and 2 
despite the fact that they were actually run at the time of job 3 and 4.

I propose that we modify the Jobs to look like the following:

+-------+--------------+---------------------+------+-------+----------+-----------+-----------+
| JobId | Name         | StartTime           | Type | Level | JobFiles | 
+-------+--------------+---------------------+------+-------+----------+-----------+-----------+
| 1     | CopyJobSave  | 2008-12-15 20:38:28 | B    | F     | 7020     | 
| 5     | CopyJobSave  | 2008-12-15 20:38:28 | C    | F     | 7020     | 
| 2     | CopyJobSave  | 2008-12-15 20:38:32 | B    | I      | 999     | 
| 6     | CopyJobSave  | 2008-12-15 20:38:32 | C    | I      | 999     | 
| 3     | copy-job     | 2008-12-15 20:41:05 | c    | F     | 0        | 
| 4     | copy-job     | 2008-12-15 20:41:50 | c    | F     | 0        | 
| 8     | RestoreFiles | 2008-12-15 20:42:39 | R    | F     | 7020     | 
+-------+--------------+---------------------+------+-------+----------+-----------+-----------+

Now here, JobIds 5 and 6 no longer appear to Bacula like they are Backup jobs, 
rather they are marked as Type=C (i.e. a Copy job), and so they will never be 
considered for restoration by default.  I've give the Copy control jobs the 
Type 'c' to distinguish them from the real copy -- they exist just to record 
the actual time the copy was made.

Now by modifying the restore code in Bacula, we should be able to provide 
features that allow the user to know that there is a Copy of JobId 1 (i.e. 5) 
and a copy of JobId 2 (i.e. 6) and allow him/her to choose which copy to use.

In my opinion, this would simplify future handling of Copy jobs allowing a lot 
more flexibility and avoiding confusion.  However, it will cause some minor 
changes (nothing serious, I believe) for users already using the new code.

A restore might go like the following:

It might go something like this:

restore 
(chose option 5 -- current for full restore)

The following JobIds are selected for restore:

 1, 2

These JobIds have copies as follows:

JobId      Copy JobId    Media Type    Date   Location ....
====================================
1            5                 Disk   ...
2            6                 Disk


To use a copy, select JobId, Copy JobId
2,6

The following JobIds have been selected for restore:
1,6


===

A more complicated example might be:

restore 
(chose option 5 -- current for full restore)

The following JobIds are selected for restore:

1, 2, 3, 4

These JobIds have copies as follows:

JobId      Copy JobId    Media Type    Date   Location ....
====================================
1            5                 Tape   ...
              7                 Disk  (second copy of job 1)
2            6                 Disk
3            8                 Tape
4            None


To use a copy, select JobId, Copy JobId
1,7 2,6

The following JobIds have been selected for restore:
7,6,3,4


Note, instead of selecting pairs (original Job, Copy), since all the jobids 
are unique, you could simply just type in the desired jobids.  For the above 
example: 7,6,3,4. 

Any comments?

Best regards,

Kern

------------------------------------------------------------------------------
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
The future of the web can't happen without you.  Join us at MIX09 to help
pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users