Amanda-Users

Re: Amanda vs. rsync vs. ... (was: Re: using disk instead of tape)

2006-09-06 10:06:53
Subject: Re: Amanda vs. rsync vs. ... (was: Re: using disk instead of tape)
From: Gene Heskett <gene.heskett AT verizon DOT net>
To: amanda-users AT amanda DOT org
Date: Wed, 06 Sep 2006 10:00:15 -0400
On Wednesday 06 September 2006 04:23, Geert Uytterhoeven wrote:
>On Tue, 5 Sep 2006, Phil Howard wrote:
>> If you want all those benefits of restore, and don't mind having a disk
>> with a filesystem already on it, then why not use something like rsync
>> to make backups?  As long as you aren't working with over about a
>> million individual files, it works great.  It makes a replica of a
>> filesystem or multi-filesystem tree, and gives you direct access to
>> every individual file for restore purpose.  Use multiple disks to make
>> multiple backups. When backing up to a disk previously used, rsync
>> avoids the writing work for files not changed (according to matching
>> meta data, though this can be turned off).  And rsync works well over a
>> network via ssh.
>>
>> So I can't really understand your argument.  What you seem to
>> specifically want that dismisses raw disk might well be better served
>> with rsync instead of Amanda.  I might want Amanda, though, for huge
>> volume and speed.
>
>Now it starts to become interesting :-)
>
>This is actually what I've been in mind to post since a long time...
>First, let's say I use Amanda and vtapes to backup my home systems.
>
>I like Amanda, because it's simple to set up, robust, ease of recovery,
> ... However, storing backups offsite over the Internet (say, on a remote
> disk at a friend's place) is not an option, due to the monthly upload
> quota enforced by all ISPs here (in Belgium).
>
>I like rsync, since it only transfers what needs to be transfered. But it
>doesn't keep multiple days of backups and hard links can be tricky.

Writing a nearly identical rsync line for crontab, to be exec'd only on x 
day of the week, such that rsync uses a different directory on the raid 
for each (active) day of the week is one way to handle this problem.  
We've been doing that at the tv station for about 4 years now.

We've had to build a bigger raid of course, at least twice, starting at 
320GB but the last rebuild took it over the terrabyte marker by quite a 
bit.

Its been very handy.  We can lose a drive in a very important machine, 
replace it, re-install the os, then rsync its data from the raid, and have 
that machine back in service as if nothing ever happened in less than a 
day's elapsed time & with only an hour or 2 of actual, on the machine 
work.  And thats getting faster as gigabit cards and switches are being 
cycled into to replace the now aging 100base-T stuffs.

>I tried rdiff-backup, which keeps reverse-incrementals, but it can take
> lots of memory on the client side (i.e. not suitable to backup old
> machines) and doesn't work well with hard links.
>
>I also use duplicity, which keeps reverse-incrementals and supports
> encryption and authentication (nice for offsite backups of my digital
> pictures on a big scratch disk at work :-), but it can take lots of
> space on $TMPDIR on the client side, and it doesn't support hard links.
>
>So my ideal backup solution would be Amanda, with support for
> incrementally storing backups at a remote location :-)
>
>In theory, it should be possible to write a tool to take the tar archives
> as created by Amanda and calculate differentials, and reassemble the tar
> archives at the other end of the network pipe, right? Or are there
> better solutions?
>
One idea might be to have another drive located remotely, set it up 
similarly to the vtape lashup amanda is using, with a pair of crontab 
entries, one to re-cycle the 'data' link on the remote drive in a round 
robin fashion, and then rsync /path/to/data to the remotes /path/to/data 
sometime later in the morning after amanda has finished.  I've thought of 
doing that from here to my shops machine, but that mobo doesn't like 2 
drives on the same pata cable even if they are the same brand of drives.

Of course, looking at the bigger picture, if a fire took this house, but 
left the shop standing, I'd have a hell of a lot more important problems 
than recovering this machine...

>Gr{oetje,eeting}s,
>
>      Geert
>
>--
>Geert Uytterhoeven -- There's lots of Linux beyond ia32 --
> geert AT linux-m68k DOT org
>
>In personal conversations with technical people, I call myself a hacker.
> But when I'm talking to journalists I just say "programmer" or something
> like that. -- Linus Torvalds

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2006 by Maurice Eugene Heskett, all rights reserved.