[Bacula-users] My auto scheduler for bacula.

Hi.
I run bacula since long, and is amazing how stable it run year after year, but
the problem is that I work with other things and have no time to fiddle with it,
so I made some scripts so it do some sort of auto scheduler.

The problem:

I have only 80 AIT2 tapes in the jukebox, and it takes some skill and time to 
change tapes. (in case of restore anyway)

I have filesystems with 700GB size, so I split it in multiple jobs.

I need longer retention period then possible with one Full every month on all 
jobs.


So, I stopped to run level Full jobs, and let a script run a Level full if I 
have less then 2 Level Full on tape, but only one at time.

That way they spread over time by itself, and is now running in perfect balance.

My scripts is not perfect, and they need some changes to run on other machines,
but not much.

But if I share it , maybe someone have use for the idea, or can make it better.


1.  The script that identify jobs that have less then 2 Level Full.

lastfull.report:
#!/bin/sh
#
# Print out name and date of the last full backup of each Job.
# /Birger.Blixt AT ericsson DOT com
#

SQL="/opt/as/mysql/0/bin/mysql -r  --skip-column-names bacula"

jobnames() {
$SQL << EOF
select distinct Name from Job;
EOF
}

lastfull() {
$SQL << EOF
select max(date(EndTime)) from Job where Name="$1" and Level="F" and 
JobStatus="T";
EOF
}

minfull() {
$SQL << EOF
select min(date(EndTime)) from Job where Name="$1" and Level="F" and 
JobStatus="T";
EOF
}

numberfull() {
$SQL << EOF
select count(JobID) from Job where Name="$1" and Level="F" and JobStatus="T";
EOF
}

for i in `jobnames`
do
         name=$i
         number=`numberfull $i`
         last=`lastfull $i`
         oldest=`minfull $i`
         if [ $number -lt 2 ]; then
            msg="WARNING"
         else
            msg="OK"
         fi
         printf "%14s has %2s level 0 backups, last=%10s oldest=%10s %7s\n" 
$name $number $last $oldest $msg
done

2.  I clean the db from canceled jobs, and dec,inc jobs based on deleted Full 
jobs.
It will also clean current jobs, so be careful.

dbcleanup:
#!/bin/sh
#
# Clean up database bacula from inc and dec jobs based on deleted full backup.
# Also clean canceled and failed jobs from the database
# /Birger.Blixt AT ericsson DOT com
#
cd /opt/local/etc
echo status dir | ./bconsole | grep "No Jobs running" >/dev/null
if [ $? -eq 0 ]; then
   echo Cleaning up the bacula database from errors and bogus records
else
   echo we have jobs running, try later.
   exit 2
fi

SQL="/extra/mysql/0/bin/mysql -r  --skip-column-names bacula"

#Fix this to better define an error/canceled/bogus job
errorjobs() {
$SQL << EOF
select JobId from Job where JobStatus!="T";
EOF
}

errornumjobs() {
$SQL << EOF
select count(JobId) from Job where JobStatus!="T";
EOF
}

deletejobid() {
$SQL << EOF
select JobId,Level,JobStatus,EndTime,Job from Job where JobId=$1;
delete from File where JobId=$1;
delete from Job where JobId=$1;
delete from JobMedia where JobId=$1;
EOF
}

jobnames() {
$SQL << EOF
select distinct Name from Job;
EOF
}

firstfull() {
$SQL << EOF
select min(JobId) from Job where Name="$1" and Level="F" and JobStatus="T";
EOF
}

beforefullnum() {
$SQL << EOF
select count(JobId) from Job where JobId<$2 and Name="$1";
EOF
}

beforefulljobs() {
$SQL << EOF
select JobId from Job where JobId<$2 and Name="$1";
EOF
}

for n in `jobnames`
do
         name=$n
         fullid=`firstfull $name`
         if [ $fullid -lt 1 ]; then
            echo "No Level 0 backup found for job $name, deleting all $name 
jobs."
            fullid=999999999
         fi
         delnum=`beforefullnum $name $fullid`
         if [ $delnum -lt 1 ]; then
            echo Job $name start with JobId $fullid level FULL
         else
            echo Deleting all jobs on $name before JobId $fullid
            for jobid in `beforefulljobs  $name $fullid`
            do
               deletejobid $jobid
            done
         fi
done
delerrnum=`errornumjobs`
if [ $delerrnum -lt 1 ]; then
  echo No error jobs to delete
else
    echo Deleting $delerrnum failed or canceled jobs from the database
    for jobid in `errorjobs`
    do
       deletejobid $jobid
    done
fi

3. A script that can start a Level Full job.
just ./run will display all jobs , ./run jobname will run an inc.
./run jobname Full will run a full, this must be changed dependong on what
bconsole ask for, not so nice maybe.

run:
#!/bin/bash
#
# Run a backup from a script
#



if [ $# -lt 1 ]; then
   echo run | ./bconsole | grep "[0-9]: " |grep -vi restore| cut -f2 -d":"
fi

if [ $# -eq 1 ]; then

./bconsole << EOF
run $1
yes
EOF
fi

if [ $# -eq 2 ]; then
if [ $2 == "Full" ]; then
./bconsole << EOF
run $1
mod
1
2
yes
EOF
else
echo "Only supported argument 2 is \"Full\""
fi
fi

4: A script run from cron that do the job.
I run it once per day on a time where everything normally is quiet.
autofull:
#!/bin/bash
# Run a level 0 backup if less then 2 remains.

cd /opt/local/etc
./dbcleanup
if [ $? -eq 2 ]; then
   #Backup is running, try later
   exit 2
fi

name=$(./lastfull.report | grep WARNING | head -1 | awk '{print $1}')

if  [ -z $name ]; then
    echo No Full backup needed
else
    ./run $name Full
fi

Last, to make sql to work as root without -u root -p , one need the file 
~root/.my.cnf  (not readable for group/others)
[client]
host = localhost
user    = root
password       = 'dbapasswd'

If you do sudo su , make sure to change $HOME so mysql don't look after your 
$HOME/.my.cnf (or use sudo su -)

I was going to make it finish and add more safe checks, but since it just work
so far it has just been left as it is :-/

/BB

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users