On Wed, Sep 18, 2013 at 10:35 AM, <afpteam AT sbcglobal DOT net> wrote:
>
>
> I may be putting more stock than I should in BackupPC adopting a "method" or
> grossly misunderstanding feasibility completely, in order to avoid file
> based recovery as much as possible, hence my reference to saving and
> restoring an "image" of the OS snapshot I'm hoping a VM should lend itself
> to.
Backuppc is just the wrong place to start if you want to back up only
large files that change every run (like VM images or database dumps).
>
>> BackupPC doesn't really do 'images', it backs up individual files, which
>> you
>> can assemble into a tarball when you want to restore.
>
> A snapshot file-set is still a "file based" functionality to BackupPC, I
> would hope. It may be an LVM tarballed as a definition of "image" perhaps.
> I may have abused the term "image", (not meaning to imply iso), but
> compressing an entire OS from two partitions into tar based, encrypted files
> transmitting off-machine is my meaning in this context, if that helps. I
> may also be giving up incremental backups in favor of an expedient "total
> restore" concept.
Backuppc just doesn't have any advantage in this process and adds a
certain amount of unnecessary overhead. What backuppc does very well
is back up large sets of files where there is a lot of duplication,
both between backup runs and on different target hosts. In this
scenario, all files with exactly the same content are pooled and
optionally compressed so you can keep a much larger backup history
online than you would expect. However, when file contents change
between runs, backuppc can use rsync to only transfer the differences
but the process of comparing against a compressed copy and
reconstructing the full new file by merging the old contents with the
differences is slow and inefficient and not very practical for files
the size of vm images. And, if that is all you have you'll end up
with unique files and no pooling.
>
>> Just curious, is there a reason you bought a virtual machine at a remote
>> location rather than experimenting on a local machine or virtual machine?
>
> Yes, proximity and bandwidth resources mostly with sufficient redundancy to
> attain reliability. I handle a number of global operations reliant on
> combinations of our own AND 3rd party VPS both, one of the reasons I need a
> restoration solution that works fairly full scope across many VM
> configurations. We tend to "nest in" solutions at each data center we deal
> with globally, which are fairly small client machines in every instance,
> Windows and Linux both.
>
>> BackupPC is a collection of perl scripts with a nice web interface front
>> end, which store data on the machine BackupPC is installed on. (you could
>> make it more complicated than that, but it's pretty advanced and there's
>> not
>> much call for it).
>
> I was hoping to springboard off of this script for VMs on ...
> http://www.redhat.com/archives/virt-tools-list/2009-October/msg00069.html
If you really want to save images, you probably want something more
like rdiff-backup that can store deltas instead of reconstructing
whole files. Or, do infrequent full-image saves that you would use
to bring up a base machine to a point where you could restore from a
file-based backuppc backup. This approach gives you easier access
to single file or directory restores (that in my experience are needed
a lot more often than full machine restores).
> I may need to also consider the backup server as the one "different"
> approach to the rest possibly, but having a "pair" of identical backup
> servers in separate locations lends itself to cross-distributed process for
> each backup server to backup the other, kind of thing.
Backing up a backuppc server presents its own problems. Normally, the
pool directory has millions of hardlinks which are hard for any
file-based backup scheme to reconstruct, and the archive filesystem is
usually too large to copy as an image - and both approaches require
the filesystem to be idle during the copy. Depending on the rate of
change and your backup windows, it may be practical to just run
independent servers in different locations hitting the same targets,
which will also eliminate any single point of failure.
> Restoring an "OS only" at a VPS vendor is quick and easy. Following this by
> over-writing the OS from snapshot, to restore integrated client specific
> application and data makes the most sense to me, if a VM disk storage can
> aptly represent a "stateful machine". Many times, discerning the OS file
> sets from Application and again from data in Windows for example is
> impossible, so again a "snapshot" represents a container based recovery
> concept, if I'm not all wet in assuming this is attainable. When addressing
> client user VM's you can never be sure what the client did to the OS or the
> data and in a disaster situation the VPS vendor is likely to be also facing
> severe issues at the time of a recovery.
Assuming you never work with physical hardware, you might be on the
right track, but I don't think backuppc is going to be the best tool.
On the other hand, if you can think in terms of making a 'base' image
to get to a point where you can drop the files back, backuppc would be
good for keeping a long history of files online with easy access to
any individual file or directory.
> Being a novice admin I am, doing file level restoration seems fraught with
> issues of time stamps, mnt folders and a whole myriad of "how to capture and
> restore Linux", not to mention the "running" state of a client VM.
Linux is pretty straightforward. Pretty much everything is a file
without a lot of magic. You might find the 'ReaR' package
interesting: it builds a bootable iso with a system's own tools that
comes up with a script to reconstruct the partitioning and filesystem
layout, then restore from a tar (or similar) backup. So it basically
distills all of the magic into a few shell scripts and it works on
either hardware or VMs.. I don't think anyone has built in a backuppc
restore yet but it should not be hard at all. You can also use
clonezilla to do image backups and it works on windows too, but you
have to shut down and reboot to use it.
> It
> looked to me like using a dynamic snapshot of the entire running VM, might
> permit a restorable "set" which could be sent back into the server, fsked
> and be right back up running. I may be dissolutioned, over stating
> simplicity here, but hopefully you can see my desire for a "package"
> restoration, versus file sets.
Except - in practice it seems much more common to accidentally delete
or corrupt a few files and not notice for a week than to have the host
melt completely.
> We deal with multiple flavors of several 3rd party VPS dependencies. So,
> the idea of having to restore 50 client VM's at one location remotely and
> patch each one differently back to a running state from file restores,
> LOOKED like I would be facing 3 hours or worse per VM in a disaster
> recovery. I need something where at worst, the basic VM is rebuilt from
> template, then blow the last working snapshot back in place and go.
> Hopefully the need to re-cook the template isn't even an issue since an
> image restore should basically reconstruct the entire VM, providing the boot
> load and container are still intact.
For a lot of things, that's the best approach - use a base VM image,
spin it up, then restore files on top of it.
> Again, please forgive if I'm over-simplifying an idealistic approach. I
> know many well accomplished Admins may look at this as folly, but I liken it
> to "Restoration As A Service" (RAAS) kind of thing, trying to do so outside
> the VPS hypervisor context. It may solve all of my woes at the price of
> larger backup copies less frequently, sacrificing incremental if necessary.
You are over-simplifying in the sense that different data and services
have different backup requirements. Sometimes you need 'live'
replication across locations. Sometimes you need long, long,
histories of changes. Sometimes you need a 'warm' spare that can spin
up quickly but might not be up to the minute with data. And
sometimes, especially in the non-VM world, you need to restore the
data onto different hardware or under a different OS than where the
backup was made.
> Is the approach even viable, perhaps if not with BackupPC, another tool set
> even?
Backuppc is great for the long-history, easy file access, sort of
thing, not so good for VM images, database dumps and other huge
fast-changing items.
--
Les Mikesell
lesmikesell AT gmail DOT com
------------------------------------------------------------------------------
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13.
http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
