Olivier Nicole a écrit :

In order to backup a Zope database, I would like to use star with pre-post processes to stop & restart the server. I've read the doc about how to use wrapper but i'm still in difficulty.

Jon has already told you all about it. At the end I add some comments
on backuping Zope. But lets go through the list a second time:

- customized debian/rules (the equivalent of the configure) to change --with-gnutar=/bin/tar \ for --with-gnutar=/bin/mywrapperscript \

OK (with the note from Jon about putting scripts into /bin)

- rebuilt the package and installed it on the client to be backed up (should I install it on the server too ?).

OK, no need on the server. Now on the client, Amanda thins that GNU
tar is your sript. So your script should behave as GNU tar.

- customized wrapper example script and put it on client /bin directory.

OK. Did you make sure the script is working?

You should take a look at an amandad debug file (generally in
/tmp/amanda) for tar. The file should be called
runtar.<timestamp>.debug and it contains the tar command used by
amanda. Try to run that command, replacing tar by your wrapper. Your
wrapper should accept all the options and arguments that GNU tar
accepts.

You must have a skelton script working even before you recompile
Amanda on the client...

ERROR: www-dev: [GNUTAR program not available]


Here is your script I've customized:

#!/usr/bin/perl -w

# The list of services to stop before the backup and start afterward
@services=("S20zope2.7");

# How long we keep the logs? (in sec.)
$old=24*3600*3; # 3 days

use Getopt::Long qw(:config pass_through);
open LOG2, ">>/tmp/amanda-wrapper.log";
print LOG2 "Start\n";

my @saveopts = @ARGV;
GetOptions (
       'create' => \$create,
       'directory=s' => \$dir,
       'file=s' => \$file,
   'listed-incremental=s' => \$incremental,
);

@ARGV = @saveopts;

my $postproc = 0;

if ($create  &&  $dir eq '/var/lib/zope2.7' &&  $file ne '/dev/null') {
   $incr=0;
   print LOG2 "plop";
   if ($incremental=~/www-dev_var_lib_zope2.7(\d).new/) {
   $incr=1 if $1>0;
   print LOG2 "plopy";
   }
   &preproc($incr);
   $postproc = 1;
#}

system @ARGV;
print LOG2 "@ARGV\n";
my $rc = $? >> 8;

if ($postproc) {
   &postproc($incr);
}

print LOG2 "End\n";
close(LOG2);
exit $rc;

sub postproc {
   my $incr=$_[0];
# The post-processing: restart the services.

   print LOG "Tared\nStarting services\n";
   close LOG;
   foreach $service (@services) {

   }
   open LOG, ">>$logfile";
   print LOG "Started\n";
   close LOG;
}

sub preproc {
   my $incr=$_[0];
# pre-processing: open the log file, kill old log files,
# stop the services listed

   # find the log file
   @t=localtime;

   $logfile="/var/log/amanda/runtar.$time.debug";
   open LOG, ">$logfile";

# the name of the user running Amanda
   ($l, $p, $u, $g)=getpwnam "backup";
   $l=$p; # remove a warning
   chown $u, $g, "/var/log/amanda/runtar.$time.debug";

   # remove the old /var/log/amanda/amanda-wrapper logs
   opendir DIR, "/var/log/amanda";
   $now=time;
   while ($dir=readdir DIR) {
   next unless $dir=~/^runtar.*$/;
   @s=stat "/var/log/amanda/$dir";
   unlink "/var/log/amanda/$dir" if $s[10]< $now-$old;
   }
   close DIR;

   print LOG "Stopping services\n";
   close LOG;

   foreach $service (@services) {

   }
   open LOG, ">>$logfile";
   print LOG "Stopped\nRunning GNU tar...";
}

Any suggestion ?

- create a custom dumptype to specify program "STAR" instead of "GNUTAR" (may be should I specify "MYWRAPPERSCRIPT" ?)

No need. The dumptype uses GNU tar, the client tries to use GNU tar at
the location you defined at compile time, but that location is really
your script. Amanda does not know about the script.

- modified my disklist to specify the custom dumptype to use

Ditto.

That's really the basic of a wrapper: replace one program (GNU tar) by
another program (your script) without telling anyone. And for all the
one using it, your wrapper is behaving exactelly as the original
program is, only it is doing some extra functions.

How to backup Zope:

I was working on the problem few months ago. There is a problem
because Zope stores all hits data in a single file, that grows huge
(several GB). As there is a single file, it is likely to be changing
all the time, and because it is big, it may take many minutes to
tar. So we can be assured that tar will never succeed to grab the file
at a stable time.

The backup is a compromise between disk space and availability of
Zope:

1) you can stop Zope while you are doing the backup. That's the
   solution I choosed. I include the script bellow.

or

2) Zope includes a tool that allows you to do a snapshot of the big
   file, while Zope is still running. You do a snapshot and backup the
   snapshot instead of the original file. But of course you duplicate
   the big file on your disk.

   The snapshot tool looks promissing because it includes a way to do
   incremental snapshots. But there is in fact no way to decide what
   is the level of snapshot you make. You just make an incremental
   relative to the last snapshot (the last one being incremental or
   full). This does notplay well with the phylosophy of Amanda that
   does estiomantes at level 0, 1 and N+1 before it decides what level
   it will use for actuall backup. Maybe that could be soved with
   server estimates...

There is a third solution that would be very promissing: modifying the
snapshot tool to send the data to stdout instead of storing it to a
file: Amanda ignores the structure of the data it receives from GNU
tar (the total size of the data may be important though). From Amanda
point of view, it is just a set of data, that will be compressed and
stored on tape. Of course if the data is not in tar format, restore
will have to be done manually, but when it comes to that level, I
suppose it is OK.

Best regards,

Olivier

Thanks a lot,

Thomas

#!/usr/bin/perl -w

# For FreeBSD... Additional services are started by scripts in # /usr/local/etc/rc.d; before running tar, this script will stop the # services listed and restart them after. It makes logs in /tmp/amanda

# The script is unclean: it includes a lot of pathnames hard coded.
#
# The list of services to stop before the backup and start afterward
# Those services are in /usr/local/etc/rc.d
# @services=("zope286");
@services=("zope286", "zope28");

# How long we keep the logs? (in sec.)
$old=24*3600*3; # 3 days

use Getopt::Long qw(:config pass_through);

my @saveopts = @ARGV;
GetOptions (
        'create' => \$create,
        'directory=s' => \$dir,
        'file=s' => \$file,
        'listed-incremental=s' => \$incremental,
);
@ARGV = @saveopts;

my $postproc = 0;

### the DLE we apply the trick for is for the machine ufo1000 and the disk /web
if ($create  &&  $dir eq '/web' &&  $file ne '/dev/null') {
    $incr=0;
    if ($incremental=~/ufo1000_web_(\d).new/) {
        $incr=1 if $1>0;
    }
    &preproc($incr);
    $postproc = 1;
}

# call the real GNU tar command
unshift(@ARGV, "/usr/local/bin/tar");
system @ARGV;

my $rc = $? >> 8;

if ($postproc) {
    &postproc($incr);
}

exit $rc;

sub postproc {
    my $incr=$_[0];
# The post-processing: restart the services.

    print LOG "Tared\nStarting services\n";
    close LOG;
    foreach $service (@services) {
        system "PATH=/usr/bin:/bin; export PATH; /usr/local/etc/rc.d/$service.sh start 
>>$logfile 2>&1"
            if -f "/usr/local/etc/rc.d/$service.sh";
    }
    open LOG, ">>$logfile";
    print LOG "Started\n";
    close LOG;
}

sub preproc {
    my $incr=$_[0];

# pre-processing: open the log file, kill old log files, # stop the services listed

    # find the log file
    @t=localtime;

$time=sprintf("%04d%02d%02d%02d%02d%02d", $t[5]+1900, $t[4]+1, $t[3], $t[2], $t[1], $t[0]);

    $logfile="/tmp/amanda/myruntar.$time.debug";
    open LOG, ">$logfile";

# the name of the user running Amanda
    ($l, $p, $u, $g)=getpwnam "amanda";
    $l=$p; # remove a warning
    chown $u, $g, "/tmp/amanda/myruntar.$time.debug";

    # remove the old tmp/amanda/myruntar logs
    opendir DIR, "/tmp/amanda";
    $now=time;
    while ($dir=readdir DIR) {
        next unless $dir=~/^myruntar.*$/;
        @s=stat "/tmp/amanda/$dir";
        unlink "/tmp/amanda/$dir" if $s[10]< $now-$old;
    }
    close DIR;

    print LOG "Stopping services\n";
    close LOG;

    foreach $service (@services) {
        system "/usr/local/etc/rc.d/$service.sh stop >>/tmp/amanda/myruntar.$time.debug 
2>&1"
            if -f "/usr/local/etc/rc.d/$service.sh";
    }
    open LOG, ">>$logfile";
    print LOG "Stopped\nRunning GNU tar... ";
}