Bacula-users

Re: [Bacula-users] How to set up large database backup

2008-11-26 20:20:34
Subject: Re: [Bacula-users] How to set up large database backup
From: Kjetil Torgrim Homme <kjetilho AT linpro DOT no>
To: bacula-users AT lists.sourceforge DOT net
Date: Thu, 27 Nov 2008 02:17:16 +0100
I thought I'd get back to the original question :-)

"David Jurke" <David.Jurke AT tnzi DOT com> writes:

> The problem I have is with our large (expected to grow to several
> terabytes) [Oracle] server. I’m told by the DBAs that the size and
> amount of activity on this database is such that putting the whole
> database into hot backup mode for the several hours it takes to back
> it up is a Bad Idea, it generates far too many log(?) files. The
> method they recommend is to put a single tablespace into backup
> mode, back that up, put it back to normal, repeat for each
> tablespace.  The backup takes the same time, but doesn’t generate
> anything like the amount of log(?) files.
> [...]
> One way I can think to do it is if I can dive into Bacula and get
> hold of the “back up this file” program, and write a little script
> to do the database hot-backup change, back up that file (across the
> network via Bacula, straight to tape), change it back, rinse and
> repeat. It would need to be the Bacula component, not something like
> dd or tar (did I mention these are all Linux boxes), because it’s
> going to be sharing a tape, in fact interleaved with, all the other
> backups. Pointers, anyone?

I think you need to use a FIFO.  actually, many FIFO's, one for each
tablespace.

essentially, you set up each FIFO ready to go like this in a
pre-script:

  mkfifo /var/backup/tablespace1
  cat /var/oracle/tablespace1.data > /var/backup/tablespace1 &

cat will block, waiting for someone to open the FIFO and consume
bytes.  so, when Bacula gets around to this file, cat will wake up and
transfer data.

you'll want to replace cat with a program which sets the correct locks
in Oracle, reads the data and dumps it into the FIFO, then releases
the locks.

the problem in your case is that you don't want to do the locking
until Bacula starts reading, and this calls for a little systems
programming.  here's a rough outline: you'll need to dynamically
determine the list of tablespaces somehow, and you'll have to supply
the code to do the actual dumping :-)


#! /usr/bin/perl -w

use strict;
use POSIX;

my @tablespaces = ("ts01", "ts02", "exp01", "bal01", "bal02");
my $dumpdir = "/var/backup";

unless (-d $dumpdir) {
    mkdir($dumpdir)
        or die "$dumpdir: $!";
}

my %children;

for my $ts (@tablespaces) {
    my $pid = fork();
    if ($pid == 0) {
        # the child
        mkfifo("$dumpdir/$ts", 0600)
            or die "$dumpdir/$ts: $!";
        open(my $fifo, ">", "$dumpdir/$ts")
            or die "$dumpdir/$ts: $!";
        # open will block until there's a reader
        print STDERR "Dumping $ts\n";

        # ... do the work here.  silly example just copies /etc/hosts
        open(my $src, "/etc/hosts");
        while (<$src>) {
            print $fifo $_;
        }
        close($src);

        close($fifo)
            or warn "$ts: Closing FIFO failed: $!";
        unlink("$dumpdir/$ts")
            or warn "$dumpdir/$ts: $!";
        exit(1);
    } elsif ($pid > 0) {
        $children{$pid} = $ts;
    } else {
        warn "$ts: fork failed: $!";
    }
}

# Now we have started one process per tablespace in the background.
#
# You may want to skip this last bit, since the pre-script needs to
# exit before Bacula starts the backup.  If you include it for better
# logging, you need to make sure the pre-script itself is started in
# the background.

my $pid;
do {
    $pid = waitpid(-1, 0);
    if ($pid > 0 && $? != 0) {
        print STDERR "$children{$pid} (pid $pid) exited with code ", $?>>8, 
"\n";
    }
} while ($pid > 0);

-- 
hope this helps,  | Redpill  _
Kjetil T. Homme   | Linpro  (_)


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users