BackupPC-users

Re: [BackupPC-users] Using rsync for blockdevice-level synchronisation of BackupPC pools

2009-09-09 12:00:45
Subject: Re: [BackupPC-users] Using rsync for blockdevice-level synchronisation of BackupPC pools
From: Christian Völker <chrischan AT knebb DOT de>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Wed, 09 Sep 2009 17:56:25 +0200
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

>> Third try was to use "dump" for this hoping it would transfer only
>> changed blocks after the initial dump. No way. After one day it
>> transferred 300GB (!). I thought, dump might not bee a good solution...
> I suspect dump sees the link count change in the pool file's inode as you 
> expire 
>   a backup and add a new one as an update.
Yeah, something like this. So dump cannot be used. I don't need to
understand, I'm just surprised.

>> Last attempt now was to move the pool device to a VMware virtual disk
>> and let rsync run over this file. Thus rsync backing up the block
>> device. Best attepmt, I thought. Result: rsync transfers after the first
>> run  ~300GB, too.
> I haven't tried that approach but always thought it should work. 
I thought this might be a good idea, indeed. But it issn't.

Here the results of the "update" run, taken from the LVM snapshot. I
can't access the file directly as it's locked by VMware ESX (BackupPC is
running). The update run was started approx 12h after the first one.

=====================================================
sent 583112982865 bytes  received 55 bytes  15555694.41 bytes/sec
total size is 583041810920  speedup is 1.00
=====================================================
So rsync transferred the whole file again. No speedup. :-(
I'll try to split it in 2GB chunks but I think it's not really worth
testing. Total run time approx 10 hours!

This is the result of the LVM snapshot taken before rsync and removed
after rsync done:
=====================================================
    Using logical volume(s) on command line
  --- Logical volume ---
  LV Name                /dev/vg1/snap_backuppc
  VG Name                vg1
  LV UUID                CcEkAd-W8dk-EEPz-9DU3-LX5t-FMc7-xSBpqu
  LV Write Access        read only
  LV snapshot status     active destination for /dev/vg1/backup_nfs
  LV Status              available
  # open                 0
  LV Size                553.75 GB
  Current LE             17720
  COW-table size         30.00 GB
  COW-table LE           960
  Allocated to snapshot  5.27%
  Snapshot chunk size    64.00 KB
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:3

Wed Sep  9 16:41:49 CEST 2009
=====================================================


> Also you might be able to use vmware's snapshots to see how much really 
> is changing.
Yeah, good idea. Just to realize how many stuff changes during a day.


Any other clue? Looks like this is really an issue- as some special
tricks like I tried don't work either...

I'll check with the VMware snapshots about the amount which changes- and
then I'll ahve a look at drbs (or so ;)) if it's suitable through a slow
remote line.


Any further ideas?

Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with CentOS - http://enigmail.mozdev.org/

iD8DBQFKp9Ap0XNIYlAXmzsRAmoTAJ4zFuB4z8IZWbdp7v1JtFnhQXiIPgCdHIxq
WUHPXA9nLwpqUfg6zbJeMK0=
=g0KV
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>