Bacula-users

Re: [Bacula-users] Other than Bacula

2014-05-26 18:56:43
Subject: Re: [Bacula-users] Other than Bacula
From: Phil Stracchino <phils AT caerllewys DOT net>
To: bacula-users AT lists.sourceforge DOT net
Date: Mon, 26 May 2014 18:54:04 -0400
On 05/26/14 17:46, Rickinfl wrote:
> Here is my bacula-dir file. I would really like to get Bacula to 
> work. Just need some help getting it setup right then I'll be off
> running. Thanks for the help just so frustrated fighting this for
> almost a year now. For now I just delete the pool and recreate it
> after it fills so I can keep using Bacula.>


Well, the first ting that comes to notice is that your maximum volume
size is set to 3.7 terabytes, with 100 volumes allowed.  Is that
intentional?  You're telling Bacula it can use up to 370TB of disk
space.  Do you *have* 370TB of free disk space?


> # Default pool definition
> Pool {
>   Name = Default
>   Pool Type = Backup
>   Recycle = yes                       # Bacula can automatically recycle 
> Volumes
>   AutoPrune = yes                     # Prune expired volumes
>   Volume Retention = 45  days         # one year
> }
> 
> # File Pool definition
> Pool {
>   Name = File
>   Pool Type = Backup
>   Recycle = yes
>   AutoPrune = yes
>   Volume Retention = 30 days
>   Maximum Volume Bytes = 3700G
>   Maximum Volumes = 100               # Limit number of Volumes in Pool
>   LabelFormat = Backup
> }

The following is a disk file pool configuration that I find works very
well for me:


Pool {
  Name = Scratch
  Storage = babylon4-file
  Pool Type = Backup
}

Pool {
  Name = Full-Disk
  Storage = babylon4-file
  Pool Type = Backup
  Recycle = no
  Recycle Oldest Volume = no
  Recycle Current Volume = no
  AutoPrune = yes                     # Prune expired volumes
  Volume Retention = 365d
  File Retention = 365d
  Maximum Volume Jobs = 0
  Volume Use Duration = 23h
  Label Format =
"FULL-$Year${Month:p/2/0/r}${Day:p/2/0/r}-${Hour:p/2/0/r}:${Minute:p/2/0/r}"
  RecyclePool = Scratch
}

Pool {
  Name = Diff-Disk
  Storage = babylon4-file
  Pool Type = Backup
  Recycle = no
  Recycle Oldest Volume = no
  Recycle Current Volume = no
  AutoPrune = yes
  Volume Retention = 2 months
  File Retention = 2 months
  Maximum Volume Jobs = 0
  Volume Use Duration = 23h
  Label Format =
"DIFF-$Year${Month:p/2/0/r}${Day:p/2/0/r}-${Hour:p/2/0/r}:${Minute:p/2/0/r}"
  RecyclePool = Scratch
}

Pool {
  Name = Incr-Disk
  Storage = babylon4-file
  Pool Type = Backup
  Recycle = no
  Recycle Oldest Volume = no
  Recycle Current Volume = no
  AutoPrune = yes
  Volume Retention = 1 month
  File Retention = 1 month
  Maximum Volume Jobs = 0
  Volume Use Duration = 23h
  Label Format =
"INCR-$Year${Month:p/2/0/r}${Day:p/2/0/r}-${Hour:p/2/0/r}:${Minute:p/2/0/r}"
  RecyclePool = Scratch
}


Notice several things here:

1.  I am not restricting the size of volumes, I'm restricting their use
duration instead.  I'm keeping volumes SMALL, not letting them grow
almost indefinitely.  The duration is set such that each volume will be
used for only one day's jobs, with a new volume in each needed Pool
being created each day.  This means that I can delete old volumes as the
jobs on them expire and they are no longer needed.

2.  Volumes are set up to be auto-created as needed, auto-labelled, and
auto-pruned.  Each Pool has a different retention time, depending on
whether it holds monthly Full backups, weekly Differentials, or daily
Incrementals.  Incremental jobs are pruned after a month, differentials
after two months, full backups after a year.  When old volumes that no
longer contain any active jobs are purged, they are recycled into the
Scratch pool, where a weekly admin job sweeps them up and deletes them.
 This is my admin job; you may prefer to do it differently.


#!/usr/bin/perl
use strict;
use Getopt::Long;
use IPC::Open2;
use IO::Handle;


my $bconsole = '/usr/sbin/bconsole';
my (%opts, @purged, $pid);

GetOptions(\%opts,
           'verbose|v',
           'test');

my ($IN, $OUT) = (IO::Handle->new(), IO::Handle->new());

$pid = open2($OUT, $IN, $bconsole);

if (scalar (@purged = check_volumes()))
{
    printf("Bacula reports the following purged volumes:\n\t%s\n",
           join("\n\t", @purged)) if ($opts{verbose});
    my $deleted = delete_volumes(@purged);
    print "$deleted volumes deleted.\n" if ($opts{verbose});
}
elsif ($opts{verbose})
{
    print "No purged volumes found to delete.\n";
}

print $IN "exit\n";
waitpid($pid, 0);

exit (0);


sub check_volumes
{
    my $dividers = 0;
    my (@purged, @row);

    print $IN "list volumes pool=Scratch\n";
    for (;;)
    {
        my $resp = <$OUT>;
        last if ($resp =~ /No results to list./);
        $dividers++ if ($resp =~ /^[\+\-]+$/);
        last if ($dividers == 3);
        @row = split(/\s+/, $resp);
        push (@purged, $row[3]) if ($row[5] eq 'Purged');
    }

    return (@purged);
}


sub delete_volumes
{
    my $volume_dir = '/spool/bacula/';
    my $count = 0;

    foreach my $vol (@_)
    {
        my $l;
        my $file = $volume_dir.$vol;

        print "Deleting volume $vol from catalog ... " if ($opts{verbose});
        print $IN "delete volume=$vol yes\n";
        $l = <$OUT>;
        $l = <$OUT>;
        print "Done.\nDeleting volume $file from disk ... " if
($opts{verbose});
        if (-f $file)
        {
            $count++;
            unlink ($file);
        }
        print "Done.\n" if ($opts{verbose});
    }

    return ($count);
}


Another way that you could manage your disk volumes is to have a fixed
set of volumes in each Pool, use them in rotation, and use the
ActionOnPurge = TRUNCATE option.  This will truncate volumes to 0 bytes
when they are purged, instead of leaving them their original size
occupying the full disk space of whatever contents they once held.

Unless you are planning to burn your disk volumes to removable media
such as DVD or Blu-Ray disk after backup, setting a maximum volume size
and allowing your volumes to be used until they reach that size is
almost always the wrong way to do it.



-- 
  Phil Stracchino
  Babylon Communications
  phils AT caerllewys DOT net
  phil AT co.ordinate DOT org
  Landline: 603.293.8485

------------------------------------------------------------------------------
The best possible search technologies are now affordable for all companies.
Download your FREE open source Enterprise Search Engine today!
Our experts will assist you in its installation for $59/mo, no commitment.
Test it for FREE on our Cloud platform anytime!
http://pubads.g.doubleclick.net/gampad/clk?id=145328191&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users