BackupPC-users

Re: [BackupPC-users] PST files -- accelerated "retiring"?

2009-07-09 02:10:00
Subject: Re: [BackupPC-users] PST files -- accelerated "retiring"?
From: Leen Besselink <leen AT consolejunky DOT net>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Thu, 09 Jul 2009 07:34:14 +0200
Jeffrey J. Kosowsky wrote:
> Adam Goryachev wrote at about 12:36:03 +1000 on Thursday, July 9, 2009:
> Jeffrey J. Kosowsky wrote:

Hi Folks,

>> Ken D'Ambrosio wrote at about 09:34:42 -0400 on Wednesday, July 8, 2009:
>>  > Hi, all.  Not having used BackupPC for about five years, I was wondering,
>>  > first and foremost, if there had come into existence a mechanism for
>>  > backing up open PST files.  Secondly, since PST files are
>>  > disproportionately large when it comes to incrementals, is there any way
>>  > to delete them at an accelerated rate?
>>  > 
> 
>> 1. Read the threads on using shadow mounts to backup open files like
>>    pst's. If you want, you can use the script that I posted to the
>>    group for automagically setting up and taking down shadow mounts
> 
>> 2. I also wrote a script, BackupPC_deleteFile for selectively deleting
>>    one or more files from one or more backups. One could presumably
>>    automate it to regularly delete past pst files using a simple cron
>>    job or equivalent. Again check out the archives. 
> 
> There is also a rsync binary that was posted which was able to
> automatically utilise shadow mounts to backup open files. I tried the
> above scripts, and got rather lost, rather than debug and work through
> it, I tried the rsync binary, which has worked perfectly since, and so I
> haven't touched it again....
> 

I guess you're talking about the patch I created based on the work that
Elias Penttilä did for rsync 2.6.x ?

The code is still a bit in development, but I think this is a good moment
to give an update of the state of that code.

I think I now have a version that works on 32-bit Windows XP all the way
up to the latest 32-bit Windows: Windows 7.

Which are probably the most different versions of 32-bit Windows that I
'need' to support if I want to support as many as possible.

That's also the only ones I've recently tested, although I've tested
Windows 2003 in the past as well, but I don't think I've broken anything
on that platform.

It's a patched version of the latest rsync 3.0.6 build on Cygwin.

Because Cygwin is pretty much 32-bit only and the 64-bit VSS API
only seems handles 64-bit calls (or whatever it's called) it only
works on Windows versions which are 32-bit.

It's still in development thus some of it is still hardcoded, that means
it works only in daemon mode and always uses shadow volumes, thus it's
readonly (writing to a shadow volume probably makes no sense).

On the other hand, it might be limited in what it supports, but that's
already pretty usefull to a lot of people I hope.

When creating a backup of Windows Vista or Windows 7 (possible Windows 2008
as well) Microsoft created an other hurdle called 'junction points' (sort of
like symlinks for directories), which rsync had a lot of problems with.

I now recoqnize they are their and skip them for now.I don't yet know if any
rsync protocol changes would be needed to support them properly. I've not
decided with to do with them, maybe I should talk to some people on the rsync
mailinglist.

It does log that it skipped them. Also usually no new ones are usually created,
so you could just create a one time list (or ones a day):

dir /aL /s > list.txt

On those recent Windows versions (Vista, 7 and possible 2008) if I wanted
to have a succesful transfer, I did have to skip:

- System Volume Information (it's a hidden/system directory, which holds lots
of things, including the files that store the Volume Shadow Copy information,
I've not figured out why it's complaining about that)

- hiberfil.sys (hibernation file, which you probably don't need on restore
anyway)

Those were the files Windows didn't want to transfer and still complaint
about even when using Shadow Volume Copy. Everything else worked.

Also you might need to start it with 'run as Administrator' and/or put
the user it's running as in the 'Backup Operators' group on Vista and
newer, but I've not done extensive testing with that.

If you don't like wasting space it's probably a good idea to skip
pagefile.sys as well, of which copying worked just fine, but possible isn't
all that useful.

I'd like to mention that technically Windows XP Shadow mounts work differently
I have to create a seperate binary for Windows XP (rsync-xp.exe).

So when deploying it, you'll need to use rsync-xp.exe (you can obviously
rename it to rsync.exe) on Windows XP and rsync.exe on the newer versions.

http://www.consolejunky.net/cwrsync-vss/versions/beta3/

( I do need to clean up the patches/code in that directory a bit, without build
instructions it might not be very clear how to use them, but the binaries are 
good )

I still need to update the readme with the current status:

http://www.consolejunky.net/cwrsync-vss/

I'll do that soon, I'm kind of busy this week.

You can find more discussion on the subject here:

http://www.itefix.no/i2/node/11984

I hope that gives you an idea of what is possible with it now and answers any
questions you may have.

The things I've mentioned that work, seem pretty stable right now.

But I can definitly use more testers. :-)

> Jeffrey, have you looked at the rsync binary which supports shadow mount
> backups? How does that compare to what your set of scripts do? I'd just
> be interested in whether your scripts offer some advantage I haven't
> considered...
> 
>> Not sure -- I haven't had a chance to look at it yet. I use cygwin
>> rsync so I wasn't too excited about installing a separate rsync
>> binary. Presumably the modified rsync is cleaner since it would be
>> able to avoid the kludge of having to separately start up vshadow.exe
>> (which requires additional kludges in my script since vshadow.exe is
>> somewhat broken in XP) and to assign drive letters to vshadow. Also,
>> assuming rsync is always running, you don't have the problem of having
>> to use something like 'at' to re-enter as an admin-level user (this is
>> due to a bug/feature in cygwin ssh where even ssh'ing as root doesn't
>> give you full admin power).
> 
>> My script has proved to be quite robust for me across multiple usage
>> cases so I haven't felt the need to try something new/better.
> 
> PS, recently I did a format and re-install of windows 2003 server up to
> the point of the first reboot (about 30-40 mins). Then I restored the
> entire c drive of a machine backed up using the rsync binary which
> supports backing up open files. Then booted the machine, which seemed to
> work perfectly... I'm not suggesting this is the ideal disaster recovery
> method, but it did work for me in this instance....
> 

That's pretty cool, thank you for testing Adam.

I guess that means I did something right. ;-)

> Regards,
> Adam
> 

Have a nice day,
        Leen Besselink.

------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time, 
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/