BackupPC-users

Re: [BackupPC-users] Using rsync for blockdevice-level synchronisation of BackupPC pools

2009-09-11 12:18:34
Subject: Re: [BackupPC-users] Using rsync for blockdevice-level synchronisation of BackupPC pools
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Fri, 11 Sep 2009 11:14:22 -0500
Timothy J Massey wrote:
> 
> So you're attempting to convert a physical BackupPC server into a virtual 
> image?  VMware has conversion tools that do this.  I've only used the 
> Windows version of VMware Converter, but it has worked perfectly for 
> converting a physical host into a virtual host.  There is a Linux version.

I'm hoping to accomplish a couple of different things in one step.  I 
don't want to convert my existing server to VMware.  I want to make a 
snapshot copy of the backuppc partition with as little downtime as 
possible - and sync'ing a RAID member will do that.  Then I want a copy 
of that offsite - and so far splitting into 2GB chunks looks like a good 
way to make rsync work.  Then, if the chunked remote copy just happened 
to be in a form that could connect up directly to a VMware guest that 
could be set up for disaster recover restores, so much the better.

> I would look into this, rather than trying to do a poor-man's version of 
> it.  It's a very simple process, and after a somewhat long wait (but 
> shorter than doing a RAID1 rebuild) you will have a brand new virtual 
> clone of your physical box!

I'd be very surprised if the converter can do it faster than a raid 
rebuild - and that's not what I want anyway.  I only want the single 
partition copied.  The physical host has other drives that aren't related.

> And if all you're doing is to try to capture a file-based version of your 
> block device (a physical partition) that you want to mount using some 
> other physical server (or even a virtual server, come to think of it), I 
> think you'd be *far* better off just dd'ing the partition into a file and 
> using a loopback mount to mount it someplace else.
> 
> In other words, the only time you should be dealing with VMDK files is if 
> you're trying to create a new virtual guest.  And if you are doing this, 
> the proper way of doing this is *not* by trying to use LVM/RAID weirdness, 
> but using the VMware Converter tools to do this for you properly.
> 
> If you're *not* trying to create a new virtual guest, then don't mess with 
> VMDK files.  They're an annoyance that should only be dealt with if you 
> actually have to.

I'd like to accomplish both at once - that is, image copy/raid sync to 
get a snapshot, and have the result usable by a separate VM.  However, I 
haven't been able to figure out how do do it with the vmware (server 
2.x) utilities.  I can create a chunked disk with vmware-diskmanager and 
I can connect it so the host sees the whole disk image in one piece with 
vmware-mount and the -f option, but I can't find a way to see a raw 
partition.  I could mount a single partition if it had a filesystem on 
it but I don't see how to access the partition in a way that mdadm will 
like.

> 
> VirtualBox compares fairly with the free VMware Server, but VMware server 
> is about 10% of what you can do with VMware--with the paid-for tools.
> 
> When it comes to commercial tools, VMware is in a class by itself, though 
> Citrix is trying hard with XenServer (still too cumbersome and unpolished 
> compared to VMware, and requires VM hardware for Windows).  When it comes 
> to free-as-in-beer, XenServer is the best.  It's still cumbersome, but 
> they give you several of the items for free that VMware charges for.
> 
> VirtualBox is neither the best tool overall, nor the best tool for free. 
> And unfortunately, the GPL'ed code is only a fraction of what you really 
> need for a usable virtualization environment.  If you want GPL tools, KVM 
> (especially in RHEL 5.4) is the best around.

I'm not convinced that any of that matters when the real issue is moving 
a physical disk head around.

> The only advantage that VirtualBox has is that it runs on OpenSolaris (or 
> OS/2...).  For me, that's a non-feature.  Obviously, YM *does* V...  :)

You left out Macs, which just happens to matter to me but not so much 
for this project.  There's a free virtualbox for intel based Macs and no 
free vmware product.  And with only a bit of tweaking you can make a 
guest image created under vmware boot and run under virtualbox.

> Interesting thoughts.  I've never been a fan of running BackupPC inside of 
> a virtualized guest.  Basically, my philosophy is to put as little between 
> my backups and the hardware as possible.  I don't even use compression on 
> my backups!  The idea of putting my backups inside of a virtual disk on 
> top of yet another filesystem is not overly appealing.  Now I've got two 
> ways for EXT3 to screw me!  :)
> 
> But the ability to rsync collections of VMDK files to a remote host *is* 
> appealing.  Interesting...

And having a VM image prepared to do restores is also appealing since it 
isolates the install from the hardware you might have available.  I can 
do that now from my laptop using a USB adapter to connect the disk with 
the mirror of the backuppc partition but it would be nicer to have 
remote copies that were completely virtual and automatically updated.

> How much performance do you lose using a loopback mount?  It's *gotta* be 
> less than the overhead of virtualization!  I like that idea even better. 

This is the effect I was hoping to get by vmware-mounting the vmdk into 
the physical host.

> But all it buys you is being able to use rsync directly on a file instead 
> of coming up with a way to copy a block device in an rsync-like manner... 
> And, to me, that's the best way of all.
> 
> Of course, now we've come full circle:  how do you copy a physical block 
> device in an rsync-like manner?  :)

Maybe the fuse/perl driver mentioned earlier would work with one end in 
the physical backuppc server and the other in the remote disaster 
recovery VMware guest.   But, there is a timing issue unless some sort 
of local snapshot capability is added and I'd prefer to avoid LVM.  I 
suppose I could sync my existing disk into the raid, break it, and mount 
it back separately for the rsync step to decouple the transfer time.

-- 
   Les Mikesell
     lesmikesell AT gmail DOT com


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>