cleaning up snapdiff snapshots on netapp

[LieutenantLefse]

I've just started to use the Netapp snapdiff features (TSM 6.3 on linux x86_64 with an OnTAP 8.0 filer). It's fantastic for my multimillion inode filesystems. But I've noticed that if there is any failure in the backup, TSM does not clean up the base snapshot - it just hangs around until I manually remove it.

I can, and will, write a script to clean these up, but I thought I'd check here first to see if I was missing some option or simple solution.

 

[LieutenantLefse]

Here's a little bash script that checks for excess snapdiff snapshots, and echos out the command to delete them. It works on the assumption that there should never be more than two "TSM_" snapshots per volume at a time (or one, if you can ensure there are no snapdiffs in progress). Remove the "echo" to have it actually go ahead and delete the snapshots, if you like to live dangerously.

#!/bin/bash
# snapdiff_clean.sh

# Connects to the netapp FILERS via ssh and, for each volume searches for snapshot containing TSM_.
# If there are more than two, the excess snapdiff snapshots are removed.
# Changes in the output of ONTAP's vol and snap commands will likely break this script.
# Tested with ONTAP 8.0.2.

# the names of the filers
FILERS="toaster1 toaster2"

# set this to 1 if you are certain this script is run when no snapdiffs are in progress
MAXSNAP=2

for FILER in $FILERS; do
  VOLS=  
  VOLS=`ssh $FILER vol status -l | grep -v Volume | awk '{print $1}'`
  for VOL in $VOLS; do
    SNAPDIFFS=
    SNAPDIFFS=`ssh $FILER snap list -n $VOL | grep TSM_ | awk '{print $NF}'`
    count=0  
    for SNAPDIFF in $SNAPDIFFS; do
      let count=count+1
      if [ $count -gt $MAXSNAP ]; then
        echo ssh $FILER snap delete $VOL $SNAPDIFF
      fi
    done
  done
done

 

[arbo]

Hello!

I'm having the same problem with occasional backups failing, and snapshots not being deleted.
What I'm more concerned about, besides the snapshots not being deleted, is files not being backed up.

Lets say some files on the NAS are changed, and during the backup the SnapDiff reports to TSM that they need to be backed up. If the backup fails in the middle, some of these files might not be backed up. If the same files are not changed anymore, and I run another SnapDiff backup, a new differential snapshot will be created. The question is which base snapshot will be used to compare the new differential snapshot with?
Will it be the snapshot created during the failed backup, or the snapshot created during the last successful backup?
From what I understand, unless the base snapshot used is the one created during the last successful backup, changed files will not be reported to TSM as eligible for backup...

 

[Harry_Redl]

Hi,

it is recommended to run "full incremental" (without snapdiff option) once a while - exactly because of reason above (applies to journal backups as well).

Harry

 

[arbo]

Hi Harry,

I agree with you, it is important to do a full backup once in a while, but once in a while could be quite long between each full backup. It's not that easy to make full backups for NAS shares with tens of millions of files....
Even if I do a full backup every month, I could still lose data during the month.
That's why I trying to understand how the base\differential snapshot relations work in a case a backup fails..

 

[Harry_Redl]

Hi,

SnapDiff backup stores the name of the used snapshot in the TSM so the next run compares the new snapshot to the stored one. The name should be stored AFTER successful backup. So it should work well.
But anything can go wrong (original snapshot is deleted on the filer etc.) - so again - full incremental backup once a while is a good idea (even if it takes its time).

See this:
http://www-01.ibm.com/support/docview.wss?uid=swg27015881

Harry

 

[arbo]

Meaning that the base snapshot used will always be the one created at the end of the last successful backup.
Right?

 

[Harry_Redl]

Hi,

snapshot is created at the BEGINNING of the backup and its name is stored at the END of successful backup. Then it is used as the base for the next run.

Harry