BackupPC-users

Re: [BackupPC-users] Problems with hardlink-based backups...

2009-08-20 04:56:25
Subject: Re: [BackupPC-users] Problems with hardlink-based backups...
From: David <wizzardx AT gmail DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Thu, 20 Aug 2009 10:51:55 +0200
Thanks for the replies so far :-) They were very informative.

About BackupPC itself, I'm still evaluating whether or not to actually
use it, but I'm starting to decide against it. Here are my reasons:

1) We're not backing up a lot of machines with a huge amount of
duplicate data between machines. Just about every server and user's
data is different. The common stuff between servers (/usr/, etc),
isn't that large (the vast majority of storage is for unique data).
For user machines, we backup their user folders, not the entire C:
drive. Pooling common files from different machines isn't a priority.

2) A big problem, is user dbx files, 2 GB etc .. don't want to store
multiple copies of those. Actually a reverse diff-based approach works
a lot better imo.

My current backup system lets me define which kind of "history
storage" system to use for backups. rdiff-backup (for most places,
where it works), and hardlinks, for servers which cause problems with
rdiff-backup (although that led to my current problems with du &
locate, which I'm currently researching). I might add more "history
storage" systems, if I find something more appropriate later (eg
HashBackup or gibak), or write my own. I lose that kind of flexibility
if I change to most "fully integrated, prepackaged" backup systems
(like BackupPC and most others), as opposed to command-line tools
which you can script and mix & match to get a backup system that works
best for your setup.

Which is also why I asked earlier about the ability to mix and match
parts of BackupPC separately :-)

Yeah, there are downsides to home-brewed stuff, and I prefer to use
premade stuff most of the time. But when none of the existing stuff
matches my needs, I won't hesitate throw together scripts that do it
better (for my needs), by scripting command-line tools, or writing new
tools and then calling them. That's the unixy way :-). You can read
more here:

http://en.wikipedia.org/wiki/Unix_philosophy#Mike_Gancarz:_The_UNIX_Philosophy

3) The incremental/full system of backuppc is bothersome. I don't want
to copy over full servers later, after initial rsync (or if I do,
relatively infrequently, like once a month). I actually do want most
of the backups to be incremental (ie, how rsync does it in hardlink
snapshot-like schemes). But, a lot can change during incrementals, and
dealing with those multiple incremental levels seems kind of annoying,
although I'm sure there are good reasons for them.

4) Possibly lots of redundant config in the text files

This is very minor in the grand scheme of things, but it's a pet peeve
of mine, with backup systems I've seen in general. Every single server
or user backup etc has the complete backup details in the config
files, even if they are 99% similar.

This violates the "DRY" (Don't Repeat Yourself) programming principle
(yeah, I treat backup configuration as a programming exercise :-) ).

Like if you have 40 servers, then it looks like you need to define all
the details for all servers, rather than just defining the parts that
changed per-server.

In my own config, adding a new server to the backup config is as
simple as adding one line, like this to my server backups config file:

    bkp('192.168.0.2 complete backup') # Router

This basically adds the complete specification for a backup, to a list
of backups to be run (after all the backup configs are loaded into
memory, and filtered according to user command-line arguments and so
on to the main backup script, which itself is run daily from cron).

Earlier in the config file for server backups, there is a Python class
definition, where you define the details, kind of like a template. And
those classes can also inherit from other Python classes, to customize
a few details. Or take advantage of other Python programming
constructs.

Also, passwords are stored separately, in a secure text file, using a
~/.pgass-like format, that supports wildcards for individual fields
(for those of you familiar with PostgreSQL). eg:

    rsync:192.168.0.2::root:rrbackups:gib5Gryn

(gib5Gryn is a password I just generated with apg, for this example)

Although, these types of config files are more oriented at people who
prefer to edit text files directly (eg, programmers like myself :-) ),
and understand how classes, inheritance, and other programming-related
things work, rather than going through a web frontend. And adding a
templating-type system can introduce more complexity by itself.

Web frontends like BackupPC's are probably a lot more usable in
general though, especially for non-programmers :-)

David.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>