Chris Bennett wrote at about 08:35:31 +1030 on Wednesday, November 25, 2009:
> Hi Jeff,
>
> > I'm very interested in this topic since I have been spending a lot of
> > time developing my "shadowmountrsync" routines which attempt to
> > automate the backup part of your approach outlined above. So far I
> > have robustly implemented startup of shadows and backup of acls (using
> > subinacl and/or getfacl).
>
> Great to hear from you - I was going to mention your script in my
> first post, as it's a fantastic contribution to the problem I'm trying
> to solve. The funny thing is I spent the better part of a weekend
> working out how it all works _before_ I found your script, but what
> you've put together has been a good reference for me nonetheless.
>
Thanks for the kind words...
And I too am a newbie on Windows -- and frustrated by the complexity
of what is required to get a full metal restore. Between the
complexity of ntfs (with so many extra non-POSIX features) and the
many special Windows files (like the registry hives), it is so much
harder than Linux where you can do something as easy as 'cp -a /' or
'rsync' or 'tar' or any of dozens of other ways to get a full backup.
Even worse, I haven't seen any good documentation of what needs to be
done to back up an NTFS share and/or what needs to be done to properly
back up the special Windows files that lie top of it.
As an aside, note that in addition to the things you mention, there
are elements of the NTFS filesystem itself (even before you get to the
Windows OS) that are not properly copied with 'rsync' (and hence also
not with BackupPC). This includes things like reparse points (of which
there are several varieties that differ across OS versions) which are
only partially implemented in cygwin and may not be backed up and/or
restored properly. Secondly, there are non-POSIX elements like NTFS
alternate data streams (which are pretty cool but not consistent with
POSIX so intentionally not supported by cygwin -- I had a thread over
on cygwin on this).
So, the bottom line is that a 100% reliable under all possible
circumstances bare-metal backup using *nix-like tools may be almost
impossible. That being said much of the above weirdness is not used if
at all on standard Windows installations (for example junctions (a
type of reparse point) were not really used prior to Vista and the
version used in Vista is reportedly supported at least to some degree
in cygwin). Alternate data streams are not even supported by Windows
explorer and are not used in any common applications (though they are
a great friend to rootkit writers since they can hide streams in files
that look zero length).
>
> > 1. What is 'fileacl' and how is it better/worse than subinacl?
> > (which seems to backup all the ACL info (vs. the limited getfacl
> > capabilities)
>
> Good question - it's a tool I found in 2007 when I last had a look at
> doing more then file-level backups with backuppc.
>
> http://www.gbordier.com/gbtools/fileacl.asp
>
> I had not heard/seen of subinacl or getfacl until I looked at your
> work (hint: I'm not a Windows admin.. :) )
>
> I'm willing to use whatever works, and fileacl claims save/restore
> acl's comprehensively.
>
> I've tried both fileacl and subinacl with the same results.
>
I too only heard about 'subinacl' after first implementing 'getfacl'
and then finding access weirdness that I couldn't address with
'getfacl' which led to me realize that there are other attributes out
there not covered by 'getfacl' -- I had several posts on this on the
cygwin list. getfacl only covers the POSIX acl's. This deficiency led
me to 'subinacl'
I would recommend 'subinacl' since it is supported by Microsoft and
seems to get out all the ACL details (and also has some nice options
to make batch changes for things like moving systems, changing
machine/domain names, and changing SIDs).
One thing to be careful about is that 'subinacl' like vshadow and
dosdev can't be run directly from SSH (where USERNAME=SYSTEM). The
problem is insidious in that 'subinacl' seems to run fine without
error but all file ownership is replaced by:
/owner =nt authority\system
Interestingly, 'getfacl' was hacked by the cygwin folks to get around
that limitation so it works even directly from SSH though as I noted
above it misses the non-POSIX acl details.
This requires a slight change from the posted version of my
'shadowmountrsync' script to make sure that 'subinacl' is run after
the 'at' recursion. I hacked the change on my personal copy but have
not had the time to do it right on the posted script (and there is a
subtlety for the case where you ssh but don't want to use shadows)
> I'm wondering what isn't restored by a file-level restore, permission
> restore, and then a fixboot / fixmbr. Or whether it's something to do
> with the change of 'hard disks' (in my case VMWare vmdk files).
I seem to recall that changing disks requires some fiddling.
> Or
> whether the on-disk block location of some files is stored somewhere
> and that breaks on the restore. Coming from a Linux background, I
> don't know why it has to be so hard to understand how to perform a
> full system recovery without forking out $$.. :)
Me too -- it's amazing how hard it seems to be just to understand what
needs to be done for a full backup let alone implement and test
it. And in my opinion such complexity and obscurity just makes
mistakes and bugs more likely even if you do finally implement it all.
>
> > 2. What does the "ntbackup of systemstate" add?
> > If I understand that better, I would be happy to add it as an
> > additional option to shadowmountrsync.
>
> Well, I don't know the details as it's all a little abstract, but
> reading:
>
> http://technet.microsoft.com/en-us/library/cc785306(WS.10).aspx
>
> gives some hints on what it covers, and I'm just used to seeing it be
> done with other backup tools. For some of it's components backed up
> (e.g. AD), I think additional massaging is done on backup, and when
> restored, additional unrolling/replaying of files onto the system
> instead of just file-level recovery. But I could be way off there.
I read about ntbackup of systemstate a while back. But I seem to
recall that one of the key things it backs up are the registry hives
which I do with shadow copy. So I wonder what the 'systemstate' adds
to a full shadow copy (plus full subinacl acl backup). Specifically,
the page you reference says:
Windows XP Professional: The System State data includes only the
registry, COM+ Class Registration database, files under Windows
File Protection, and boot files."
Note that the COM+ Class Registration database is stored under:
C:\Windows\Registration
which seems to be backed up correctly under my shadow script.
So, I'm not sure what if anything it adds (at least in the non-Domain
controller non-Cluster setting).
> > Please keep me informed on your progress on getting bare-metal
> > restores to work. I am very interested in getting this all working.
>
> My major aim is to do a backup/restore out of band without recovering
> a base-os installed. Barring that, I want to work out what the
> minimum amount of work is required before I can restore from my
> backuppc files / permission dump and have the system functioning as it
> was at the time of backup.
That would be my goal too.
> I hope someone out there with the right Windows knowledge can provide
> some additional answers to the questions we are asking.
I hope so but I wouldn't hold my breath.
One alternative that I have been thinking about (but have neither the
time nor knowledge to do) would be to adapt 'ntfsclone' with the
'--metadata' and '--save-image' flags. The first flag just backs up
the NTFS metada but not the file data and the second flag helps save
image space. According to the manpage when compressed, the metada is
typically 1-8MB. Customarily, it is used for forensics, but I wonder
whether it could be combined with the raw file data from BackupPC to
yield a complete byte-level accurate backup.
One problem with ntfsclone is that as written, you need to run it from
Linux and mount the drive... but conceptually the idea of having
another program to back up the metada byte-by-byte while using the
pooling of BackupPC to save the file data sounds like the ideal. In
particular, accurate cloning of the metadata would make sure that you
captured all the subtleties of the filesystem while pooling maximizes
the efficiency of the data backup.
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
