The enclosed script (and a 1-line cmd.exe helper) cleanly and
automatically sets up shadow copies, mounts them, and launches the
rsync daemon without requiring any special configuration or changes to
your existing (non-shadow) rsyncd.conf script. It also cleanly unwinds
all the above when you are done rsyncing your files.
As a result, you should now be able to back up any (and all) files
that are locked under Windows such as registry files (e.g. ntuser.dat)
and databases (e.g. Outlook).
The script can easily be called directly from $Conf{DumpPreUserCmd}
and $Conf{DumpPostUserCmd} -- see the script for examples in
the comments.
The script itself (which uses some hairy recursion to get around
limitations in cygwin ssh, vshadow, and dosdev) is heavily documented.
Enjoy and if you use it, please send me feedback, bug reports,
enhancements, etc...
Jeff
-----------------------------------------------------------------------------------------
#!/bin/bash
# Mountrsyncshadows
# Copyright Jeffrey J. Kosowsky December 2008
# Version 0.2
# Description: automagically sets up (and takes down) shadow copies,
# shadow mounts, and the rsyncd daemon to allow you to seamlessly
# rsync to shadow copies of all the active modules in your current
# rsyncd.conf file WITHOUT any configuration changes or added settings
# to your existing setup non-shadow copy rsyncd setup. In particular,
# the program determines which shadows to create based upon the
# modules in your existing /etc/rsyncd.conf combined with what mounts
# actually exist.
# Usage:
# mountrsynchadows
# Sets up shadow copies, mounts shadows, and launches rsync
# mountrsyncshadows -d
# Terminates rsync, kills shadow copies, unmounts, and cleans up
#
#
# If using as pre/post commands in BackupPC, try something like:
#$Conf{DumpPreUserCmd} =
# '$sshPath -q -x -l kosowsky $host bash -c
"/usr/local/bin/mountrsyncshadows; sleep 10"';
#
# $Conf{DumpPostUserCmd} =
# '$sshPath -q -x -l kosowsky $host bash -c
"/usr/local/bin/vshadow-scripts/mountrsyncshadows -d"';
#
# where the sleep was just added to give rsyncd time to stabilize on a
# slow machine. Also, clearly shadow copies CANNOT be restored to
# since by definition they are read-only
#
# WARNING: This is beta software which has only been tested to work so
# far on my own XP sp2 setup in a limited number of situation. In
# particular, it may not work on other Windows version since the
# nature of shadow copy (and vshadow and dosdev) in particular has
# changed radically for Windows versions both prior to and post
# WinXP. Given the nature of shadow copy, it *should* (theoretically)
# not be capable of doing anything destructive to the source disk, so
# it's likely that the worse that would happen is that your backups
# would fail (or potentially be incomplete/corrupted). Feedback, bug
# reports, and enhancements are always welcome!!!
#
# NOTE: the companion 1-liner 'shadowwait.cmd' file MUST be in the
# same directory Also, both files need to be in the Administrators
# group and with group permissions +rx
#
# REQUIRES:
# vshadow.exe
# Available from Microsoft downloads (make sure you get the
# right version for your architecture!)
#
http://www.microsoft.com/downloads/details.aspx?FamilyID=0B4F56E4-0CCC-4626-826A-ED2C4C95C871&displaylang=en
# dosdev.exe
# Again available from Microsoft downloads (as part of their
# reporting tools). Also, be careful of the version since
# different verssions seem to have different command line
# interfaces. I obtained my copy from:
# http://www.ltr-data.se/files/dosdev.zip
#
# NOTE: The program recurses through this routine 3 times because of
# the limitations first of ssh and then of vshadow. An additional
# '0th' pass is used to terminate the shadow copy & clean up.
# In particular, cygwin ssh logins lack the authority to create shadow
# copies or set up dos devices since it is not a true authenticated
# user login. On the other hand, vshadow is quite crippled in XP since
# it cannot create shadow copies nor can they be easily
# mounted. Dosdev is necessary because cygwin does not (yet) allow
# direct mounting of shadow copy devices since they lie in the kernel
# namespace.
#
# Since the program recurses both under the login user and under the
# SYSTEM user (for ssh), you need to be attentive to permissions. The
# program tries to detect and log such errors, but in particular:
# vshadow, dosdev, this bash script (and its cmd.exe companion), the
# log & lock file, and the shadowdir should all be readable (and where
# appropriate writable & executable) by the Administrators group. Note also that
# the drive letter string ALL_DRIVELETTS can be restricted if you are
# likely to be mounting/unmounting temporary devices (like cds, dvds)
# requiring a fixed drive letter assignment during the rsync time. You
# do not need to restrict it for fixed drives (e.g. C:) since they are
# automatically avoided if present at the time of shadow creation.
#
# The recursion operates as follows:
# First, since the ssh login lacks the authority to create shadow
# copies or to set up dos devices, the program schedules an 'at' job
# (for the next minute) to recurse again through this routine. (If not
# launched from 'ssh' then for parallelism, the recursion is directly
# launched with cmd.exe).
#
# Then, 'vshadow' calls the routine a third time once the shadow copy
# has been set up to commplete the post-processing, including setting
# up device letters and mounting them and then starting the rsyncd
# daemon on an appropriately modified version of the original
# rsyncd.conf. This shadow -exec script needs a a 1-line helper bash
# cmd script since the -exec script can't take arguments and must be
# either a .cmd or .exe file. The shadow exec script (3rd pass) also
# works to keep the shadow copy alive (mimicking persistence) by
# launching the rsync daemon with the no-detach option.
#
# Finally, when the transfer is completed and rsyncd is killed, the
# whole recursion unwinds and everything gets cleaned up
# appropriately.
# Note that pass #2 which sets up the shadow copies (which is also
# visible as the 'at' job if under ssh) and pass #3 (which is the
# script that mounts the shadow copies and launches rsync) remain
# alive until rsyncd is terminated. Pass #1 which kicks off the
# recursion hangs around until either a setup error is detected or
# until the setup is completed with the launch of the rsync daemon, at
# which time it returns and signals the result to the calling
# process. In particular, this initial pass returns '0' on success and a
# positive error code [1-8] on error.
#
# Indeed, the easiest/cleanest/best way to unwind the recursions and
# cause everything to reset gracefully is to kill the final 'rsync'
# process for which everything else ends up waiting. The '-d' option
# first tries to clean up gracefully like this but if it fails then it
# uses brute force to kill rsyncd, delete mounts, shutdown shadow
# copies, and remove lock files.
#
# Given the complexity of the multiple recursions, the program has
# been instrumented with plentiful (but optional) logging turned on by
# setting the $LOG variable to the log file you want to use. Indeed,
# the logging is fun and instructive to watch ;)
#
# By setting $STRICT you can ensure that shadow creation will abort if
# an rsync process or shadow copy is already running. Otherwise, it
# will kill existing shadows and rsync processes. It's probably a good
# idea to keep it set unless you are sure that you won't be colliding
# with another shadow copy process. The downside is that if something
# is stuck, then the routine will abort (and optionally log the
# result) rather than forcing a clean up.
##############################################################################
##### Define variables
RSYNCCONF=/etc/rsyncd.conf # Your original (non-shadow) version of rsyncd.conf
RSYNCPID=/var/run/rsyncd.pid # Temporary file to which shadow version is written
RSYNC=/usr/bin/rsync #Use full path so you can kill it too
PATH=/bin:/usr/bin:/usr/local/bin:/c/WINDOWS/System32
DOSDEV=dosdev
VSHADOW=vshadow
SHADOWDIR=/tmp/shadowdir #Directory where shadow mounts are set up
#MUST not be any dir where other things are mounted (e.g., / or /cygdrive)
RSYNCCONF_SHADOW=$SHADOWDIR/rsyncd.shadow.conf #Location for modified version
#of rsyncd.conf (auto-generated)
LOCK=$SHADOWDIR/shadowlock
SHADOWWAITCMD=shadowwait.cmd #Name of 1-liner helper script
#Must be in same directory as this script
SHADOWWAITCMD=$(cygpath -w $( (cd -P $(dirname $0) && pwd ))/$SHADOWWAITCMD)
STRICT=1 #Set if you want strict checking before starting
LOG=/tmp/crap #Set if you want logging (advisable at least to start)
TIMEOUT=300 #Time in seconds 1st pass waits for setting up before timing out
##############################################################################
##### Define some helper functions
# Relaunches current script (or optionally the command given in $@)
# via 'at' to get around $USERNAME=SYSTEM problem under ssh login
# where the shell lacks permsisions to run commmands like vshadow or
# dosdev
function at_relaunch () {
local h m s command
if [ $# -eq 0 ] ; then
command=$( (cd -P $(dirname $0) && pwd ))/$(basename $0)
else
command=$@
fi
set -- $(date +"%H %M %S")
let "h=$((10#$1))" "m=$((10#$2))+1" "s=$((10#$3))" #Advance minutes by 1
#Explicitly use base 10 so that 08 and 09 not interpreted as bad octal
[ $s -gt 55 ] && let "m +=1"
[ $m -ge 60 ] && let "m = m % 60" "h += 1"
let "h = h % 24"
at $h:$m $(cygpath -w $(which bash.exe)) -c \"$command\" > /dev/null
}
# Populate SHADOWMOUNT with the drive letters (including colon) that are
# both present in rsyncd.conf and that are actually mounted
function get_shadowmounts () {
#Extract the drive letter from rsyncd.conf, converting to uppercase
local RSYNCMOUNTS=$(sed -ne "s%^[ \t]*path[ \t]*=[
\t]*/\(cygdrive/\)\?\([A-Za-z]\)\([ \t]*$\|[ \t]*#\|/\).*%\u\2%p" $RSYNCCONF|
sort | uniq | sed -e "s|$|:|")
SHADOWMOUNT=()
for mount in $RSYNCMOUNTS ; do #Eliminate mounts that are not actually
present
[ -d $mount ] && SHADOWMOUNT=("${SHADOWMOUNT[@]}" "$mount")
done
}
# Populate DRIVELETS with *free* drive letters starting from Z
function get_driveletters () {
local ALL_DRIVELETS="Z Y X W V U T S R Q P O N M L K J I H G F E D C B
A"
DRIVELETS=()
for letter in $ALL_DRIVELETS ; do
mount | egrep -q "^$letter:" \
|| DRIVELETS=("${DRIVELETS[@]}" "$letter")
done
}
# Kill (any) rsync process
function kill_rsync () {
local PID
PID=$(ps -e | sed -ne "s|^[^0-9]*\([0-9]\+\).* $RSYNC|\1|p")
[ -z "$PID" ] || kill -HUP $PID 2>/dev/null
sleep 1
PID=$(ps -e | sed -ne "s|^[^0-9]*\([0-9]\+\).* $RSYNC|\1|p")
[ -z "$PID" ] || kill -KILL $PID 2>/dev/null
[ -r "$RSYNCPID" ] && rm -f $RSYNCPID
}
# Unmount shadow mounts and remove drive letters
function remove_shadowmounts () {
local shadowmounts letter
local shadowmounts=$(mount | sed -ne "s|^\([A-Za-z]\): on
$SHADOWDIR/[A-Za-z] type.*|\1|p")
for letter in $shadowmounts ; do
umount -s $SHADOWDIR/$letter 2> /dev/null #Unmount
$DOSDEV "${letter}:" /D 2> /dev/nuul
# Note dosdev doesn't work from ssh due to permissions and probably
# not necessary after shadows killed, but good hygiene
done
}
#Clean up
function clean_up () {
( echo Y | $VSHADOW -da ) > /dev/null 2>&1 #Delete shadow copies and
as a
#positive side effect kills shadow process and spawned routines
kill_rsync
remove_shadowmounts
rm -f $LOCK $RSYNCCONF_SHADOW
[ -z "$LOG" ] || echo "..Finished clean_up... $PASS" >> $LOG
}
##############################################################################
[ -z $LOG ] ||echo -e "\n[$(date +"%d/%m/%y %H:%M:%S")]
[$PPID|$$|$1][$USER|$USERNAME|$(id -un)|$(id -run)] $0" >> $LOG #Log and
timestamp each pass
#NOTE: PPID=1 when coming in from 'at' or from 'cmd.exe
# We use this fact to prevent looping and determine where we are in
#the recursion
# Check permissions each time through since (potentially) different user context
if ! (which $VSHADOW && which $DOSDEV && [ -x $SHADOWWAITCMD ] ) > /dev/null
2>&1 ; then
[ -z "$LOG" ] || echo -e "ERROR: Path or permission errors...\n" >> $LOG
[ $PPID -eq 1 ] && echo -1 >| $LOCK #Signal to calling routine
exit 1 # Exit with error if vshadow or dosdev not in path
# or if SHADOWWAITCMD not executable
fi
### PASS=0: Clean_up/termination (just a single pass independent of the others)
if [ "$1" = "-d" ] ; then # Terminate & clean up
PASS="[Pass #0]"
[ -z "$LOG" ] || echo "***Terminating and cleaning up shadows, mounts,
and locks...$PASS" >> $LOG
FORCE=1
if [ -e $LOCK ] ; then
SCRIPTPID=$(cat $LOCK)
if [ ! -z $SCRIPTPID -a $SCRIPTPID -gt 0 ] ; then
[ -z "$LOG" ] || echo -e "--Attempting to clean up
gracefully...$PASS" >> $LOG
ps -e | grep -q "^[^0-9]*$SCRIPTPID.*/usr/bin/bash$" &&
kill -HUP $SCRIPTPID
sleep 1
ps -e | grep -q "^[^0-9]*$SCRIPTPID.*/usr/bin/bash$" &&
kill -KILL $SCRIPTPID
sleep 5 #Wait for unwinding recursion to clean up by itself...
[ -e $LOCK ] || unset FORCE #Successfully cleaned up by itself
fi
fi
if [ ! -z "$FORCE" ] ; then
[ -z "$LOG" ] || echo -e "--Forcing clean up...$PASS" >> $LOG
clean_up #Didn't clean up fully so force full cleanup
fi
[ -z "$LOG" ] || echo -e "--Termination completed...$PASS\n" >> $LOG
exit 0
### PASS=1: Initial time through
elif [ $PPID -ne 1 ] ; then #Launch initial recursion & wait for setup to
complete
PASS="[Pass #1]"
[ -z "$LOG" ] || echo "***Initial pass through...$PASS" >> $LOG
#First do some tests...
if [ ! -z "$STRICT" ] ; then
if [ -e $LOCK ] ; then #Lock file exists
[ -z "$LOG" ] || echo -e "ERROR: Lock file
exists...$PASS\n" >> $LOG
exit 2
elif [ -e $RSYNCPID ] || ps -e | grep -q " $RSYNC$" ; then
#Rsync running
[ -z "$LOG" ] || echo -e "ERROR: Rsync already
running...$PASS\n" >> $LOG
exit 3
elif ! vshadow -q | grep -q "There are no shadow copies in the
system" ; then
[ -z "$LOG" ] || echo -e "ERROR: Active shadow copies
exist...$PASS\n" >> $LOG
exit 4
fi
fi
clean_up # Clean up no matter what - may be partially redundant but
good hygeine
#Check permissions on files we call. Touch lock file. Set up SHADOWDIR
#and check their permissions
if ! ( stat -c %A $0 | grep -q "^....r.x" && \
[ $(stat -c %G $0 ) = "Administrators" ] && \
stat -c %A $SHADOWWAITCMD | grep -q "^....r.x" && \
[ $(stat -c %G $SHADOWWAITCMD) = "Administrators" ] && \
mkdir -m 775 -p $SHADOWDIR && \
touch $LOCK && \
chown $USER.Administrators $SHADOWDIR $LOCK $LOG && \
chmod g+rw $LOCK $LOG
) ; then
[ -z "$LOG" ] || echo -e "ERROR: Permissions error with '$0' or
'$SHADOWWAITCMD' or '$SHADOWDIR' or '$LOCK' or '$LOG'....$PASS\n" >> $LOG
exit 5
fi
if [ ! -z "$SSH_CLIENT" ] ; then # Set only if coming in via SSH
#Relaunch using 'at' if you come in via ssh ($USERNAME=SYSTEM)
#because then you won't have privileges to run vshadow and dosdev
[ -z "$LOG" ] || echo "--Relaunching as SYSTEM user via 'at'
(may need to wait up to a minute)...$PASS" >> $LOG
at_relaunch #Recurse
else #Just relaunch directly (for consistency with recursion above)
[ -z "$LOG" ] || echo "--Relaunching via 'cmd.exe'...$PASS" >>
$LOG
cmd.exe /C $(cygpath -w $(which bash.exe)) -c "$0" &
fi
[ -z "$LOG" ] || echo "--Initial pass going into background waiting for
setup to complete...$PASS" >> $LOG
#Check lock file for error or completion unless times-out first
TIMEOUT=$(($(date +%s) + $TIMEOUT))
while [ $(date +%s) -lt $TIMEOUT ] ; do #Wait for setup completion or
timeout
[ -s "$LOCK" ] && break
sleep 1
done
if [ -s "$LOCK" ] ; then
RETURN=$((-$(cat $LOCK)))
else
RETURN=8
fi
if [ $RETURN -lt 0 ] ; then
RETURN=0
else # Didn't complete successfully
clean_up
fi
exit $RETURN
### PASS=3: Called by -exec command of vshadow
elif [ $PPID -eq 1 -a "$1" = "-e" ] ; then #Mount shadows & launch rsync
PASS="[Pass #3]"
[ -z "$LOG" ] || echo "**Running exec command spawned by
vshadow...$PASS" >> $LOG
get_shadowmounts
#Create paired array of the drive letter and the windows shadow path
SHADOWPAIRS=( `$VSHADOW -q | sed -ne "s/\( *- Original Volume
name:.*\[\([A-Z]\):.*\)\| *- Shadow copy device name: */\2/p"` )
NUMSHADOWS=$((${#SHADOWPAIRS[@]}/2))
if [ $NUMSHADOWS -ne ${#SHADOWMOUNT[@]} ] ; then
[ -z "$LOG"] || echo -e "ERROR: Shadows created $NUMSHADOWS not
equal to shadows requested (${#SHADOWMOUNT[@]})...$PASS\n" >> $LOG
echo -6 >| $LOCK
exit 6
fi
get_driveletters
[ -z "$LOG" ] || echo "drive letters=${DRIVELETS[@]} $PASS" >> $LOG
if [ $NUMSHADOWS -gt ${#DRIVELETS[@]} ] ; then
[ -z "$LOG" ] || echo -e "ERROR: Not enough free drive
letters...$PASS\n" >> $LOG
echo -7 >| $LOCK
exit 7
fi
cp -f $RSYNCCONF $RSYNCCONF_SHADOW
for (( index=0 ; index < $NUMSHADOWS; index+=1 )) ; do
# Create drive letter corresponding to shadow mount
$DOSDEV "${DRIVELETS[$index]}:" "${SHADOWPAIRS[2*$index+1]}"
# Mount drive letter
mount -f -s "${DRIVELETS[$index]}:"
$SHADOWDIR/${DRIVELETS[$index]}
[ -z "$LOG" ] || echo
"${DRIVELETS[$index]}:->${SHADOWPAIRS[2*$index]}: ${SHADOWPAIRS[2*$index+1]}"
>> $LOG
[ -z "$LOG" ] || (echo -en "\t" ; mount | grep
"^${DRIVELETS[$index]}:") >> $LOG
sed -i -e "s%^\([ \t]*path[ \t]*=[
\t]*\)/\(cygdrive/\)\?${SHADOWPAIRS[2*$index]}\([ \t]*$\|[
\t]*#\|/\)%\1$SHADOWDIR/${DRIVELETS[$index]}\3%i" $RSYNCCONF_SHADOW
done
kill_rsync #Should already be dead but kill again just in case...
[ -z "$LOG" ] || echo "--Starting rsyncd daemon and waiting for file
transfer to complete...$PASS" >> $LOG
echo $$ >| $LOCK #Put PID in lockfile & signal that setup successfull
rsync --daemon --no-detach --config=$RSYNCCONF_SHADOW
[ -z "$LOG" ] || echo "--Rsync terminated: exiting & returing control
to PASS=1...$PASS" >> $LOG
exit 0
### PASS=2: Called by at or cmd.exe from initial recursion
else #Determine shadow mounts and launch vshadow
PASS="[Pass #2]"
[ -z "$LOG" ] || echo "**Running inital shadow setup...$PASS" >> $LOG
get_shadowmounts
( echo Y | $VSHADOW -da ) > /dev/null 2>&1 #Delete old shadow copies
# This should be redundant, but just in case...
[ -z "$LOG" ] || echo "--Starting shadow copy and waiting for it to
terminate...$PASS" >> $LOG
$VSHADOW -exec="$SHADOWWAITCMD" ${SHADOWMOUNT[@]} > /dev/null 2>&1
[ -z "$LOG" ] || echo "--Vshadow terminated: cleaning up...$PASS" >>
$LOG
#NOTE: vshadow doesn't finish until script $SHADOWWAITCMD script
completed
clean_up # Should already be cleaned up by unwinding of recursion, but
# no harm in cleaning more...
[ -z "$LOG" ] || echo -e "--Done! Clean up completed...$PASS\n" >> $LOG
exit 0
fi
exit 9 #Shouldn't get here...
-----------------------------------------------------------------------------------------------
@echo OFF
REM Shadowwait.cmd
REM Copyright Jeffrey J. Kosowsky December 2008
REM Version 0.2
REM This cmd.exe script file should be in the same directory as the
REM bash script: mountrsyncshadows
C:\cygwin\bin\bash.exe -c "$(cygpath.exe '%~p0\mountrsyncshadows') -e"
------------------------------------------------------------------------------
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
The future of the web can't happen without you. Join us at MIX09 to help
pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
