Hello all,
I was wondering if a new type of backup could be added to the list (right now
we have tar,rsync,archive, etc). I was thinking in a LVM snapshot, but the
backup wouldn't be done on the FS level but at block level.
I'm talking about this because I've a special need for backing up a large FS
(ext4) that has 16TB with small files (hundreds of millions of them) stored in
a HP EVA (this is the storage of an IMAP cluster).
I use backup for 12 other places and I like it a lot, not only on the
functionality but also on the reliability, right now there's a tape backup
(with omni backup) for the storage, but the performance is miserable and I need
to find another solution. I've done tests with BackupPC and the results are
really nice but these results are done only with a part of the data (20GB,
small files, and the same FS, using rsync type backup)
I'm not sure about doing the 16TB (performance, backup duration) so I thinking
in some kind of block device backup.
Idea:
1 - Create lvm snapshot of the block device
2 - Backup lvm snapshot (I could use DD, but then it would be a full backup
every time I do a backup), something like rsync where the only the changed
blocks of the block device.
Benefits:
1 - Performance, althoughtthe gains only show after 70% of full disk, 45%-50%
full disk for small files.
2 - Restore backup directly into volume.
3 - Possibility of mount on a loop device.
Conns:
The first backup should take ages, and initial FS should have zeros on it's
free space (so the initial backup can use the compression efficiently)
This approach is only possible on unix/linux FS.
The LVM methods for creating snapshots aren't standard and partitioning /
volume creation need to be addressed and thought before deployment (is this a
conn??)
The recover method should be able to restore the block device (in this case an
LVM volume).
I can see lots of difficulties with this approach but the benefits can be great
too.
What do you all think about this.
Cheers,
Pedro
On Thursday 03 March 2011 16:48:49 Jeffrey J. Kosowsky wrote:
> Just a plug that Craig Barratt -- the BackupPC creator -- has posted
> several detailed emails to the developers list
> (backuppc-devel AT lists.sourceforge DOT net) outlining the key design and
> feature changes that he is implementing in 4.x.
>
> >From all appearances, 4.x is a *substantial* rewrite and includes many
> of the features and goodies that we have all been clamoring for
> particularly regarding getting rid of hard links and the challenges
> they cause in archiving the backups. There also seem to be several
> important extensions (e.g., extended attributes) and efficiency
> improvements.
>
> If anyone is interested in knowing what is in the pipeline and more
> importantly if anyone has comments or suggestions, now is probably the
> time to chime in before things get too fixed in stone. So I would
> suggest that you look at the archives of the developer list and/or
> subscribe to it to see the ongoing discussion.
>
> Craig is incredibly open and welcoming of inputs and feedback.
> (Craig I hope you don't mind my plugging your postings here, but you
> know I'm your fan)
>
>
>
>
> ------------------------------------------------------------------------------
> Free Software Download: Index, Search & Analyze Logs and other IT data in
> Real-Time with Splunk. Collect, index and harness all the fast moving IT data
> generated by your applications, servers and devices whether physical, virtual
> or in the cloud. Deliver compliance at lower cost and gain new business
> insights. http://p.sf.net/sfu/splunk-dev2dev
> _______________________________________________
> BackupPC-users mailing list
> BackupPC-users AT lists.sourceforge DOT net
> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki: http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
--
----------------------------------------------------------------------------------------------------------
Pedro M. S. Oliveira
IT Consultant
Email: pmsoliveira AT gmail DOT com
URL: http://www.linux-geex.com
Cellular: +351 96 5867227
----------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------
What You Don't Know About Data Connectivity CAN Hurt You
This paper provides an overview of data connectivity, details
its effect on application quality, and explores various alternative
solutions. http://p.sf.net/sfu/progress-d2d
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
