Amanda-Users

amanda and zfs

2006-03-08 06:33:19
Subject: amanda and zfs
From: Anthony Worrall <anthony.worrall AT reading.ac DOT uk>
To: amanda-users AT amanda DOT org
Date: Wed, 08 Mar 2006 11:29:22 +0000
Hi

I have playing about with Sun's new filesystem ZFS.

One of the things I wanted to do was make sure I could get amanda to
backup zfs.
The way the backup utility works with zfs is based on snapshots. You can
either
do a full bakup of a filesystem or the difference between two snapshots.
The restore process is an all or nothing recovery of the snapshot. There
is no
way to recover a single file from a backup or to index the snapshot.

I have hacked a script to replace ufsdump which will manage the
snapshots and
appropriately respond to amanda's sendsize and sendbackup requests which
is included below.
It requires that the amanda user has a role that can run the zfs
command. Also the
zfs DLE's should have indexing turned off.

Here are some observations about zfs and amanda. ZFS encourages the use
of more
smaller filesystems at the home directory or even software package level
as it is easy
to grow and manage filesystems. This may have an impact on amanda in a
couple of ways.

1. Amanda seems to be more likely to do a full backup of the filesystem
since it is
   small. However with many small filesystems the total amount of data
backed up may 
   end up being larger. Currently I have only a handfull of filestesm
with a few megabytes
   each and amanda has only request fullbackups

2. With many filesystems managing the DLE's becomes more complex. One
idea maybe to 
   haves the DLE's for a client stored on the client. Amanda would then
need to get
   the DLEs during the estimate phase or when doing a check of the
client. This 
   would also be useful in the case where the management of the client
is separated 
   from the backup management.

I guess some of this issues may be addressed in 2.5 

Cheers

Anthony Worrall

#!/bin/pfksh
#need to use pfksh so that zfs can be run

ZFS=/usr/sbin/zfs

#Get the last argument
for FS in $@;
do
 echo $FS > /dev/null
done

#Check if the filesystem is zfs or ufs
res=`$ZFS list | /usr/xpg4/bin/egrep "${FS}$"`
if [ $? -eq 0 ]; then
  DEV=`echo $res | awk '{ print $1}'`
  ESTIMATE=0
  LEVEL="unknown"
  UPDATE=0

  case $1 in
    *0*) LEVEL=0;;
    *1*) LEVEL=1;;
    *2*) LEVEL=2;;
    *3*) LEVEL=3;;
    *4*) LEVEL=4;;
    *5*) LEVEL=5;;
    *6*) LEVEL=6;;
    *7*) LEVEL=7;;
    *8*) LEVEL=8;;
    *9*) LEVEL=9;;
    *) echo level NOT specfied; exit -1;;
  esac

  case $1 in
    *S*) ESTIMATE=1;;
  esac

  case $1 in
    *u*) UPDATE=1;;
  esac

  $ZFS list -H -t snapshot | /usr/xpg4/bin/grep -q ${DEV}@0
  if [ $? -eq 0 ]; then
    $ZFS destroy ${DEV}@0
  fi
  $ZFS snapshot ${DEV}@0

  # make sure all the snapshots up to $LEVEL exist
  n=1
  while [ $n -le $LEVEL ];
  do
    $ZFS list -H -t snapshot | /usr/xpg4/bin/grep -q ${DEV}@$n
    if [ $? -ne 0 ]; then
      LEVEL=$(( $n - 1 ))
      break
    fi
    n=$(($n+1))
  done

  full=`$ZFS list -H -o refer ${DEV}@0`
#convert returned size into kilobytes
  case $full in
    *K) full=`echo $full | sed -e 's/K//'`;;
    *M) full=`echo $full | sed -e 's/M//'`; full=$(( $full * 1024 ));;
    *G) full=`echo $full | sed -e 's/G//'`; full=$(( $full * 1024 *
1024));;
  esac
  if [ $LEVEL -gt 0 ]; then
    incr=`$ZFS list -H -o refer ${DEV}@${LEVEL}`
    case $full in
      *K) full=`echo $full | sed -e 's/K//'`;;
      *M) full=`echo $full | sed -e 's/M//'`; full=$(( $full * 1024 ));;
      *G) full=`echo $full | sed -e 's/G//'`; full=$(( $full * 1024 *
1024));;
    esac
    size=$(( $full - $incr ))
  else
    size=$full
  fi
  size=$(( $size + 16 ))
  if [ $ESTIMATE == 1 ]; then
#    echo "doing Estimate $DEV at level $LEVEL" >&2
    size=$(( $size * 1024 ))
    echo $size
    $ZFS destroy ${DEV}@0
  else
#    echo "Dumping $DEV at level $LEVEL" >&2
    if [ $LEVEL -eq 0 ]; then
      $ZFS backup ${DEV}@0
    else
      $ZFS backup -i ${DEV}@${LEVEL} ${DEV}@0
    fi
    block=$(( $size * 2 ))
    MB=$(( $size / 1024 ))
    echo "DUMP: $block blocks (${MB}MB)" >&2
    if [ $UPDATE -eq 1 ]; then
      for snap in `$ZFS list -H -t snapshot | awk '{print $1}' | egrep
${DEV}@"[1-9]"`; do
        n=`echo $snap | cut -f2 -d@`
        if [ $n -gt $LEVEL ]; then
           $ZFS destroy $snap
        fi
      done
      n=$(( $LEVEL + 1 ))
      $ZFS rename ${DEV}@0 ${DEV}@${n}
    else
      $ZFS destroy ${DEV}@0
    fi
  fi
else
  /usr/lib/fs/ufs/ufsdump $*
fi










<Prev in Thread] Current Thread [Next in Thread>