BackupPC-users

Re: [BackupPC-users] Using rsync for blockdevice-level synchronisation of BackupPC pools

2009-09-10 21:26:41
Subject: Re: [BackupPC-users] Using rsync for blockdevice-level synchronisation of BackupPC pools
From: Timothy J Massey <tmassey AT obscorp DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Thu, 10 Sep 2009 21:17:12 -0400
Les Mikesell <lesmikesell AT gmail DOT com> wrote on 09/10/2009 05:23:20 PM:

> Timothy J Massey wrote:
> > 
> >> I thought there was a way to access the vmx image directly from 
linux, 
> >> but I don't know if it has to be mounted or if you can see a raw 
> >> partition.
> > 
> > Depends:  are we talking VMware Server, or ESX/ESXi?  And if we're 
talking 
> > ESXi, are we talking SAN, NAS or DAS (directly attached storage)?
> 
> Server.
> 
> > In the case of Server, the VMX images are right there on a Linux 
> > filesystem: do with them what you want.  Just snapshot properly.  I 
think 
> > I've commented enough on this in other e-mails.
> 
> I know there are there in the form of a directory of files which will be 

> nice for the rsync step.  What I want to know is if the physical host 
> can see the virtual disk/partition the same way a guest can. There is a 
> vmware-mount that I think lets you see the virtual filesystem, but I 
> want access to the raw virtual partition at a block device level.

No, you really don't!  :)

What is the advantage of this?  You can simply rsync the *file* just as 
easy as try to work with "raw virtual partitions".  I see no advantage, 
only problems.  Even better, rsync is already set up to handle files!  Why 
not just use it as-is?

> >> I was hoping I could make a virtual partition visible on 
> >> physical backuppc server so I could sync it as a raid1, then remove 
it 
> >> and have something that could be used in a virtual machine or copied 
via 
> > 
> >> rsync, then used by a remote virtual machine.
> > 
> > What is a "virtual partition"?  Do you mean the VMFS directly?  Or do 
you 
> > mean that you want to *mount* the filesystem *inside* of the VMware 
image?
> 
> I was hoping to be able to see a block device on the host that would be 
> the same thing a guest would see.  Then, without running a guest I could 

> either let the virtual partition sync as a raid member or dd an image 
> copy to it.  When that is complete, I'd like to be able to rsync the 
> directory of files that hold the vmdk virtual disk off to another 
> machine where a virtual machine could be started to access it the usual 
way.

You don't *need* all of that complexity.  Do *exactly* what you're 
describing, but use cp instead of dd to copy the files.  I think you need 
to really think long and hard about the hoops you're trying to jump 
through.  The logic that you'd use for a physical machine is obselete when 
you're dealing with VMware.  Just copy the native files as native files to 
another machine with VMware server, and start the host.  That simple.

There is *no* need for dealing with RAID, dd, etc.  cp'ing the files is 
logically *identical* to a DD or RAID1 rebuild of a physical machine.  And 
*way* simpler.

> > While it is theoretically possible to configure VMware guests (at 
least on 
> > Server) to use physical partitions directly (and therefore eliminate a 

> > layer of complexity), this is *not* a recommended configuration. 
Frankly, 
> > the idea of doing *anything* to a VMware guest filesystem at the block 

> > layer is pretty much a non-starter--especially when VMFS is involved.
> 
> I want it the other way around - the physical host accessing a virtual 
disk.

Again, you *really* *really* do not.  There is **NO** advantage to this. 
Simply treat the VMX files as if they *were* a physical copy made by 
dd--pretend that step is done for you!  :)

> > Basically, with VMware, you have few and limited choices for 
image-level 
> > backup without using the pay-for tools.
> > 
> > Server:  You can capture images at the filesystem layer (as standard 
UNIX 
> > files), assuming that you snapshot the guest properly.  No extra cost, 
and 
> > 100% reliable backups, but a pretty manual process and slight downtime 

> > twice.  Not useful for daily backups without a lot of work (you can't 
> > script snapshot removal in Server).  Mainly useful for quarterly or so 

> > disaster recovery images.
> 
> I suppose I could do dd over ssh to image copy to a running guest if the 

> physical host can't do it directly, then shut the guest down for the 
rsync.

Again, what is your logic here?  You don't move the contents of a block 
device from one guest to another.  YOU MOVE THE ENTIRE GUEST.  Period. 
This is so, very, very way better than what you're trying to do.

It's like waving a magic wand and cloning an entire pretend-physical (i.e. 
virtual) machine into two complete, identical machines.  Or better yet, 
imagine you've magically cloned the hard drives and slid them into an 
identical machine.  By simply cp'ing the contents of the guest directory 
you have accomplished exactly that.

No messing with LVM, RAID, etc.  Just a simple cp, and if you do the 
snapshotting properly, you can do it with only a reboot worth of downtime.

> > Anyway, the idea of trying to automatically work with VMware images at 
the 
> > block level without VMware's tools is not that great.   You can't 
automate
> > the use of snapshots in Server (it's intentionally not scriptable).
> 
> On the local side I don't really need a snapshot since I've got the 
> master copy.  On the remote side I'd want to keep two copies, switching 
> between them for updates or something similar.

Again, make copies via cp.

> > You 
> > *could* shut down the guest, make an LVM snapshot, restart the guest 
and 
> > use the LVM snapshot to perform a block-level rsync-style (or even 
> > cp-style) copy of the images.  You might be able to do something 
similar 
> > with ESXi and a NAS that allows LVM-style snapshots.  But outside of 
that, 
> > there be dragons...
> 
> If I have to change the parent OS, it would probably be to opensolaris 
> so I could work with zfs snapshots.

But you can't get VMware for OpenSolaris!  :)  I hate to tell you, ZFS 
buys you *NOTHING* in this situation.  You'll be forced to use VMware 
snapshotting anyway, in which case you no longer *need* filesystem 
snapshotting.

Tim Massey


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>