On Sun, Mar 10, 2013 at 04:12:31AM +0100, Holger Parplies wrote:
> Hi,
>
> Peter Carlsson wrote on 2013-03-09 11:10:59 +0100 [Re: [BackupPC-users]
> Archiving incremental backups?]:
> > [...]
> > I regulary do an archive to a USB HDD that I store offsite. This is a
> > manual step since I have to bring the HDD, do the archiving, and then
> > store the HDD offsite again.
> >
> > To know for sure that files that are modified in between these regular
> > archives, I want to make tar archives of the incremental backups and
> > move them on a daily basis offsite over the Internet.
> >
> > Why I only want to do this for the incremental backuped files is to
> > reduce the amount of data over the Internet.
>
> in theory, you could use rsync to transfer incremental changes over the
> internet. This could even be substantially less traffic than an incremental
> tar, presuming you have changing files that can efficiently be transferred
> with rsync (like growing log files or large files where only small portions
> change from day to day). In the worst case (only new files, no changed files)
> there should not be much difference, except that rsync requires more
> computational power, and that rsync (with the right options) will track
> deletions. rsync also has built-in capability to limit transfer bandwidth.
>
> The thing is, you would need to keep an image of your target file system on
> both ends, i.e. unpack the tar file on your USB HDD and have it accessible
> over
> the internet, and have a local copy on or near your BackupPC server. You could
> use the original file system (the one you back up in the first place) instead
> of a copy, presuming your concern is not to mirror backup state (your file
> system will probably have changed since the last backup), and the file system
> can handle the additional load of the rsync run.
>
> One thing I would like to note, though, is that you really want at least two
> independent offsite copies, so you will not be left without one if things
> break while you are replacing the old copy with a new one. This is especially
> true with tar archives. An rsync run will generally not destroy much (maybe
> one file) if it fails prematurely, though it will leave you with a state of
> your offsite copy that probably never existed on the original (which is easily
> fixed if you can just restart rsync). But you should note that requiring the
> offsite copy to be online (at least during the incremental update) makes it
> somewhat vulnerable, so having an additional *offline* offsite backup would
> be a good idea.
>
> > But what I want to achieve is, at least in my
> > opinion, better than to only have the full manual archives that at
> > best will be done one or two times a month.
>
> With rsync, in my opinion, you can almost get away without the full manual
> archives. The same note applies here as in the frequently asked question,
> "with rsync, do I need full backups at all?". Ideally, you would turn on the
> rsync option "--ignore-times" regularly (e.g. once a month) to catch any
> (rare) changes rsync might have missed (but that's just a detail).
>
> I don't know if what I described seems possible in your scenario. We should
> figure that out first before going into too much detail.
>
> > The most important thing is that it is simple and automatic, otherwise
> > it will never be done.
>
> It should be possible to make this automatic (except for exchanging or syncing
> the offsite drives once in a while, but that's much like your monthly manual
> archives now). It doesn't seem overly complex, but that depends somewhat on
> your setup. How were you planning to get the incrementals over the internet?
> Have you done size estimates to see if it is feasible?
>
>
> Jeffrey hinted at the possibility to tar up the pc/host/num directory for an
> incremental backup. While that is certainly possible, it leaves you with the
> problem that you would need to unmangle names and interpret attrib files.
> That's not too difficult, but it would require some coding. The thing is, how
> (and under what circumstances) would you ever *use* the offsite backup? In the
> event of a catastrophe? Bring in the disk, restore the full backup, restore
> all incrementals? I'd be in favour of having a working file system image (or
> better, two identical ones - one as a backup remaining offsite) which you can
> just plug in and use, if things need to go really fast, or copy over without
> needing to think much (i.e. without much that can go wrong).
>
> Another thing to keep in mind are backup levels. Normally, your incrementals
> will tend to be relative to the last full, meaning they will grow from day to
> day (because day 2 repeats the changes from day 1 and so on). You *can* fix
> that (at the cost of more complexity at backup time), but you probably don't
> want to bother with this approach anyway :).
>
>
> There's, of course, the third possibility of just setting up an offsite
> BackupPC server that makes independent backups of the target host. You'd
> want to have a VPN for that, but my guess is that incremental tars would be
> sent over one, too :-).
>
> Regards,
> Holger
Hi,
Thanks for your detailed reply!
I realize after yours and others explanations, that I have not thought
about all the shortcomings, but I was thinking this could be a good
compromise. This would allow me (at least with some effort) to restore
modified files even if a crash happened between two full archives.
I will go back to my drawing board and think more about what I want to
achieve, now that I have additional information.
Best regards,
Peter Carlsson
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
