Bacula-users

[Bacula-users] VirtualFull backup. No previous Jobs found.

2009-11-02 22:03:59
Subject: [Bacula-users] VirtualFull backup. No previous Jobs found.
From: Jim Barber <jim.barber AT ddihealth DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Tue, 03 Nov 2009 11:00:56 +0800
Hi.

I have just started using Bacula for the first time.
I am using version 3.0.2 as packaged for Debian testing.

     vc-dir Version: 3.0.2 (18 July 2009) i486-pc-linux-gnu debian squeeze/sid

The director and storage daemon are installed on to the same host with a Dell 
TL2000 tape library attached.
This tape library has one autochanger servicing two drives.

I am backing up a mix of Linux and Windows clients, that after doing a full 
backup of each, fills a little over 5 LTO-3 tapes.
I only have about 50 GB of free disk space, so I can't use a backup to disk and 
then to tape strategy, so I am using disk as spool.
I've assigned 20 GB spool space to each drive in the tape library.

The plan I am hoping to achieve is as follows:

1. Incremental backups of all clients nightly, Monday to Friday.
2. Produce a VirtualFull backup after the incremental backup on Friday to 
consolidate the week's incremental backups into a full backup.
3. Copy the newly created full backup to another set of tapes that can be taken 
off-site.

My reasoning is that the clients will only ever need to do one full backup 
(which takes over a day for all of them to complete).
 From then on, they will only need to do incremental backups which regularly 
get consolidated on the backup server into full backups.
This should keep backup times short at all times and minimise the amount of 
data crossing the network.

I have separate pools defined for each of these steps all referring the to same 
tape library.
I also have a scratch pool where I've placed all of my tapes so that the above 
pools can pick from it as required.

So far step 1 is working out fine, but I'm having issues with step 2.
I haven't tried step 3 yet since it depends on a working step 2.

I've run incremental jobs a number of times and now want to create the 
VirtualFull backup.
If I start the VirtualFull job manually it shows:

     *run
     Automatically selected Catalog: MyCatalog
     Using Catalog "MyCatalog"
     A job name must be specified.
     The defined Job resources are:
          1: BackupBuildatron
          2: BackupDavros
          3: BackupDc1
          4: BackupDc2
          5: BackupFreddy
          6: BackupMail
          7: BackupSpirateam
          8: BackupShadow
          9: BackupVc
         10: BackupWiki
         11: BackupWikiHcn
         12: FullBackup
         13: CatalogBackup
         14: OffsiteBackup
         15: RestoreFiles
     Select Job resource (1-15): 12
     Run Backup job
     JobName:  FullBackup
     Level:    VirtualFull
     Client:   vc-fd
     FileSet:  None
     Pool:     Default (From Job resource)
     Storage:  TL2000 (From Storage from Pool's NextPool resource)
     When:     2009-11-03 09:40:01
     Priority: 11
     OK to run? (yes/mod/no): yes
     Job queued. JobId=67

The job immediately terminates and the messages are as follows:

     03-Nov 09:40 vc-dir JobId 67: Start Virtual Backup JobId 67, 
Job=FullBackup.2009-11-03_09.40.04_02
     03-Nov 09:40 vc-dir JobId 67: Fatal error: No previous Jobs found.
     03-Nov 09:40 vc-dir JobId 67: Error: Bacula vc-dir 3.0.2 (18Jul09): 
01-Jan-1970 08:00:00
       Build OS:               i486-pc-linux-gnu debian squeeze/sid
       JobId:                  67
       Job:                    FullBackup.2009-11-03_09.40.04_02
       Backup Level:           Virtual Full
       Client:                 "vc-fd" 3.0.2 (18Jul09) 
i486-pc-linux-gnu,debian,squeeze/sid
       FileSet:                "None" 2009-11-02 20:50:14
       Pool:                   "FullPool" (From Job Pool's NextPool resource)
       Catalog:                "MyCatalog" (From Client resource)
       Storage:                "TL2000" (From Storage from Pool's NextPool 
resource)
       Scheduled time:         03-Nov-2009 09:40:01
       Start time:             01-Jan-1970 08:00:00
       End time:               01-Jan-1970 08:00:00
       Elapsed time:           0 secs
       Priority:               11
       SD Files Written:       0
       SD Bytes Written:       0 (0 B)
       Rate:                   0.0 KB/s
       Volume name(s):
       Volume Session Id:      0
       Volume Session Time:    0
       Last Volume Bytes:      0 (0 B)
       SD Errors:              0
       SD termination status:
       Termination:            *** Backup Error ***

The definitions in my /etc/bacula/bacula-sd.conf file follows.
Passwords are blanked out.

     Storage {
       Name = vc-sd
       SDPort = 9103
       WorkingDirectory = "/var/lib/bacula"
       Pid Directory = "/var/run/bacula"
       Maximum Concurrent Jobs = 20
     }

     Director {
       Name = vc-dir
       Password = ""
     }

     Director {
       Name = vc-mon
       Password = ""
       Monitor = yes
     }

     Autochanger {
       Name = TL2000
       Device = Drive-0
       Device = Drive-1
       Changer Command = "/etc/bacula/scripts/mtx-changer %c %o %S %a %d"
       Changer Device = /dev/sg5
     }

     Device {
       Name = Drive-0
       Drive Index = 0
       Media Type = LTO
       Archive Device = /dev/nst0
       RemovableMedia = yes;
       RandomAccess = no;
       Maximum File Size = 4GB
       AutoChanger = yes
       Maximum Job Spool Size = 20g
       Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
     }

     Device {
       Name = Drive-1
       Drive Index = 1
       Media Type = LTO
       Archive Device = /dev/nst1
       RemovableMedia = yes;
       RandomAccess = no;
       Maximum File Size = 4GB
       AutoChanger = yes
       Maximum Job Spool Size = 20g
       Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
     }

     Messages {
       Name = Standard
       director = vc-dir = all
     }

The definitions in my /etc/bacula/bacula-dir.conf file follows.
Passwords are blanked out.
Clients and their backups jobs are left out except for the local host 
(vc.ddihealth.com).
Only FileSets pertaining to the local host are shown.
Catalog, Console, and Messages entries are also left out as they are 
unimportant (and unchanged from defaults).
I've left a comment where ever I've removed something.

     Director{
       Name = vc-dir
       DIRport = 9101
       QueryFile = "/etc/bacula/scripts/query.sql"
       WorkingDirectory = "/var/lib/bacula"
       PidDirectory = "/var/run/bacula"
       Maximum Concurrent Jobs = 2
       Password = ""
       Messages = Daemon
     }

     JobDefs{
       Name = "DefaultJob"
       Type = Backup
       Level = Incremental
       Client = vc-fd
       FileSet = "LinuxSet"
       Schedule = "DailyBackupSchedule"
       Messages = Standard
       Pool = Default
       Priority = 10
       # Bootstrap file will be named after the job.
       Write Bootstrap = "/var/lib/bacula/%n.bsr"
       Spool Data = yes
       # I may need to enable the following, based on I'll never get real full 
backups, so might miss moved files without it.
     # Accurate = yes
     }

     Job{
       Name = "BackupVc"
       JobDefs = "DefaultJob"
       Client = vc-fd
     }

     ### Jobs for other Linux and Windows clients removed...

     # Consolidate incremental backups into a full backup.
     Job{
       Name = "FullBackup"
       Type = Backup
       Level = VirtualFull
       Client = vc-fd
       FileSet = "None"
       Schedule = "FullBackupSchedule"
       Messages = Standard
       # The pool to consolidate from.
       # The pool's 'Next Pool' directive defines where the VirtualFull backup 
gets created.
       Pool = Default
       Accurate = yes
       # Run after all client incremental backups are done.
       Priority = 11
     }

     # Backup the catalog database.
     Job{
       Name = "CatalogBackup"
       JobDefs = "DefaultJob"
       Level = Full
       FileSet = "Catalog"
       Schedule = "CatalogBackupSchedule"
       Pool = FullPool
       RunBeforeJob = "/etc/bacula/scripts/make_catalog_backup bacula bacula"
       RunAfterJob  = "/etc/bacula/scripts/delete_catalog_backup"
       # Run after client incremental backups and virtual full backups (if run) 
are complete.
       Priority = 12
     }

     # Copy the full backups over to tapes for off-site storage.
     Job{
       Name = "OffsiteBackup"
       Type = Copy
       Level = Full
       Client = vc-fd
       FileSet = "None"
       Schedule = "OffsiteBackupSchedule"
       Messages = Standard
       # The pool to copy from (which has Next Pool defined for where to copy 
to).
       Pool = FullPool
       Selection Type = PoolUncopiedJobs
       Allow Duplicate Jobs = yes
       Allow Higher Duplicates = no
       # Run after the catalog backup has been done.
       Priority = 13
     }

     # Incremental backups on the week days.
     Schedule{
       Name = "DailyBackupSchedule"
       Run = mon-fri at 22:05
     }

     # This schedule is to consolidate the incremental backups into a full 
backup.
     Schedule{
       Name = "FullBackupSchedule"
       Run = fri at 22:10
     }

     # Backup the catalog.
     Schedule{
       Name = "CatalogBackupSchedule"
       Run = mon-fri at 22:15
     }

     # This schedule is to create off-site tapes.
     Schedule{
       Name = "OffsiteBackupSchedule"
       Run = fri at 22:20
     }

     Client{
       Name = vc-fd
       Address = localhost
       Catalog = MyCatalog
       Password = ""
       File Retention = 3 months
       Job Retention = 3 months
     }

     ### Other client definitions for Linux and Windows clients removed...

     Storage{
       Name = TL2000
       Address = vc.ddihealth.com
       Password = ""
       Device = TL2000
       Media Type = LTO
       Autochanger = yes
       # Allow two jobs to this tape library so that we can utilise both drives.
       Maximum Concurrent Jobs = 2
     }

     FileSet{
       Name = "LinuxSet"
       Include {
          @/etc/bacula/fileset-linux-exclude.conf
          File = /
       }
       Exclude {
         File = /dev
         File = /lib/init/rw
         File = /proc
         File = /sys
         File = /var/lib/bacula
         File = /.journal
         File = /.fsck
       }
     }

     ### Windows FileSets removed.

     # Fake FileSet definition for jobs that don't use the FileSet field, but 
still need it declared.
     FileSet{
       Name = "None"
       Include {
         Options {
           signature = MD5
         }
       }
     }

     ### Catalog and Messages definitions removed...

     # Incremental backups go here.
     Pool{
       Name = Default
       Pool Type = Backup
       Recycle = yes
       AutoPrune = yes
       Volume Retention = 13 days
       Cleaning Prefix = "CLN"
       Maximum Volumes = 22
       Storage = TL2000
       Scratch Pool = Scratch
       Recycle Pool = Scratch
       # The location where the virtual full backups will go.
       Next Pool = FullPool
     }

     # VirtualFull and Catalog backups go here.
     Pool{
       Name = FullPool
       Pool Type = Backup
       Recycle = yes
       AutoPrune = yes
       Volume Retention = 35 days
       Cleaning Prefix = "CLN"
       Maximum Volumes = 22
       Storage = TL2000
       Scratch Pool = Scratch
       Recycle Pool = Scratch
       # The location where the copies go for offsite backups.
       Next Pool = CopyPool
     }

     # Offsite tapes get written here.
     Pool{
       Name = CopyPool
       Pool Type = Backup
       Recycle = yes
       AutoPrune = yes
       Volume Retention = 6 days
       Cleaning Prefix = "CLN"
       Maximum Volumes = 22
       Storage = TL2000
       Scratch Pool = Scratch
       Recycle Pool = Scratch
     }

     Pool{
       Name = Scratch
       Pool Type = Backup
       Recycle Pool = Scratch
     }

     ### Console directive removed...

Can anyone see why the VirtualFull backup can't find the previous backup jobs?

The "FullBackup" job is referring to the "Default" pool where the incremental 
backups have been going.
The first ever incremental backups to the "Default" pool created Full backups 
as expected.
The "Default" pool has "Next Pool" defined to be "FullPool".
I figured it should be able to find all the backups on the "Default" pool.

I've also tried just using a single "Default" pool with "Next Pool" pointing to 
itself.
But I think that may possibly violate the Copy job I want to achieve later?
However I had exactly the same result with the VirtualFull backup trying this 
method as well.

Thanks,

-- 
----------
Jim Barber
DDI Health

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users