BackupPC-users

Re: [BackupPC-users] Using rsync for blockdevice-level synchronisation of BackupPC pools

2009-09-10 16:15:55
Subject: Re: [BackupPC-users] Using rsync for blockdevice-level synchronisation of BackupPC pools
From: Timothy J Massey <tmassey AT obscorp DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Thu, 10 Sep 2009 16:04:15 -0400
Les Mikesell <lesmikesell AT gmail DOT com> wrote on 09/10/2009 02:41:58 PM:

> Timothy J Massey wrote:
> > 
> > VMware Sever 2.0 supports disks in single files, or split in 
> > arbitrary-sized pieces (usually 2GB), both fully allocated or 
> > grow-on-demand.  ESX/ESXi only supports single-file images, and it is 
> > designed to use VMFS. 
> 
> I thought there was a way to access the vmx image directly from linux, 
> but I don't know if it has to be mounted or if you can see a raw 
> partition.

Depends:  are we talking VMware Server, or ESX/ESXi?  And if we're talking 
ESXi, are we talking SAN, NAS or DAS (directly attached storage)?

In the case of Server, the VMX images are right there on a Linux 
filesystem: do with them what you want.  Just snapshot properly.  I think 
I've commented enough on this in other e-mails.

In the case of ESX, you can use the Linux management console to deal with 
them, almost like a native Linux filesystem, but with the same limitations 
as above.  Of course, with ESX, you have the proper tools for block-based 
changes-only backups already, using VMware's tools.

In the case of ESXi (which has *zero* supported management on the server), 
you have to do it on the storage device.  (Or, pay for the VMware tools, 
which is by far the simplest, most reliable choice).  Therefore, it 
depends on how the storage is connected to the server.

        In the case of DAS, you're *very* stuck without doing weirdness 
like using unsupported tricks to start an SSH server on the ESXi 
machine--not recommended for production.  Or using the for-pay tools.  But 
ESXi with DAS makes very little sense.

        In the case of SAN (e.g. iSCSI or Fibre Channel), you're still 
stuck, because while the block device might be accessible remotely, it's 
still using VMFS, and you can't exactly mount that.  Again, you're going 
to have to do unsupported weirdness or use the pay tools.

        In the case of NAS (e.g. NFS), you can access the files just like 
you would from any other NFS share, or depending on what's providing the 
NFS share, from the NFS server itself.  And with all of the same types of 
issues that come from accessing the files directly as you would with the 
VMware Server product.  And losing the advantages of VMFS (which, without 
the for-pay tools, isn't much).

> I was hoping I could make a virtual partition visible on 
> physical backuppc server so I could sync it as a raid1, then remove it 
> and have something that could be used in a virtual machine or copied via 

> rsync, then used by a remote virtual machine.

What is a "virtual partition"?  Do you mean the VMFS directly?  Or do you 
mean that you want to *mount* the filesystem *inside* of the VMware image?

While it is theoretically possible to configure VMware guests (at least on 
Server) to use physical partitions directly (and therefore eliminate a 
layer of complexity), this is *not* a recommended configuration.  Frankly, 
the idea of doing *anything* to a VMware guest filesystem at the block 
layer is pretty much a non-starter--especially when VMFS is involved.

Basically, with VMware, you have few and limited choices for image-level 
backup without using the pay-for tools.

Server:  You can capture images at the filesystem layer (as standard UNIX 
files), assuming that you snapshot the guest properly.  No extra cost, and 
100% reliable backups, but a pretty manual process and slight downtime 
twice.  Not useful for daily backups without a lot of work (you can't 
script snapshot removal in Server).  Mainly useful for quarterly or so 
disaster recovery images.

Of course, daily backups via BackupPC works inside of the guest just 
fine...  :)

ESX:  You have to pay for ESX.  With ESX, you have the full suite of tools 
available to back up images, INCLUDING automatic block-based incremental 
backups of images:  Incremental backups that only back up changed blocks 
automatically!  *Really* straightforward.  By far the best production 
solution, and not *that* expensive:  something like $2000 for 3 servers?

ESXi:  You either have to pay for the tools (just like ESX), or you are 
limited to sleight of hand on the NAS, and you lose the advantages of 
VMFS.  If you use DAS or SAN storage, there's no layer in which you can 
get access to the image files without doing something to put your ESXi 
server in an unsupported configuration (and which might get removed in the 
future).

One small alternative to the "unsupported ESXi SSH server" situation: 
Veeam FastSCP.  http://www.veeam.com/vmware-esxi-fastscp.html  I know that 
VMware and Veeam are in a bit of an uneasy situation between them. 
Basically, Veeam makes tools that do what the VMware tools do.  I know 
that VMware has tried to prevent Veeam from allowing their tools to work 
with ESXi in the past.


Anyway, the idea of trying to automatically work with VMware images at the 
block level without VMware's tools is not that great.  You can't automate 
the use of snapshots in Server (it's intentionally not scriptable).  You 
*could* shut down the guest, make an LVM snapshot, restart the guest and 
use the LVM snapshot to perform a block-level rsync-style (or even 
cp-style) copy of the images.  You might be able to do something similar 
with ESXi and a NAS that allows LVM-style snapshots.  But outside of that, 
there be dragons...

Tim Masesy


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>