BackupPC-users

Re: [BackupPC-users] How do I use an external USB drive as backup target?

2010-02-14 17:23:23
Subject: Re: [BackupPC-users] How do I use an external USB drive as backup target?
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Sun, 14 Feb 2010 16:22:02 -0600
John Hudak wrote:
>      External USB drives are a *BAD* idea for multiple reasons:
>     - Slow
>     - Unreliable
>     - Subject to being disconnected
>     etc.
> 
> Yes, I know 1, and 2 is 'it depends', and 3 is exactly the reason why I 
> want to use usb drives

Note that there are other ways to disconnect drives that don't share the USB 
disadvantages, like hot-swap SATA enclosures, or ESATA connections.

>      > >From a data reliability standpoint, is it better to run a backup
>     session to
>      > USB drive 1, and then repeat the backup to USB drive 2? OR
>      > run a backup session to USB drive 1, and then copy the backup
>     directories to
>      > USB drive 2???
>     Look at the archives and FAQ - this has been discussed *many* times so
>     no point in wasting peoples time in rehashing.
> 
> I did a quick search of the archives before asking - I did not find a 
> definitive answer...

I doubt if there is a definitive answer. It will depend on the scale and your 
circumstances.  I'm more comfortable with an always-mirrored internal pair plus 
offsite copies.

>      > My understanding is that files that are backed up (using either
>     rsync or
>      > smb) are 'encrypted' (for lack of a better word), and to view
>     them I need to
>      > use zcat.-True?
> 
>     There is a better word -- *compressed*
> 
> So that is the word that is not clearly used in the documentation.  

Filename mangling isn't optional, compression is, as
http://backuppc.sourceforge.net/faq/BackupPC.html#configuration_parameters
should make clear.  See the entry for $Conf{CompressLevel}.

> There are many ways that backups can be manipulated: stored in a 
> completely nonstandard/proprietary file system and protocol such as 
> z-san, they can be encrypted, and they can be compressed.  The backup PC 
> doc talks about using compression, but does not state if any compression 
> is used in the default configuration.

??? The defaults are shown, but with a packaged version you need to check to 
see 
if the packager made his own changes.

> Compression is often 
> configuration parameter.  It does not make sense to compress many audio 
> and video formats.  If the data to be backed up consists predominantly 
> of these types of files, then it makes no sense to waste CPU cycles 
> applying compression to get < 5% compression.

Note that backuppc doesn't compress/uncompress every time you touch a file so 
the tradeoffs may be different than you expect. Once the initial version of a 
file has been compressed in the pool, matching copies from the same or other 
backups won't have the operation repeated.

> 
>      > Also, can the backup profile be specified to perform complete
>     data copies
>      > periodically, as opposed to a baseline and then periodic
>     incrementals?
> 
>     Read the documentation and FAQ.
> 
> I have read the doc, (where IMHO) this should have been clearly stated.  
> Instead the doc frequently introduces a topic with 1-2 sentences, 

I'm not sure what you are reading here, but set the first 3 variables under the 
'What to backup and when to do it' section if you don't like the default 
once-a-week full:
$Conf{FullPeriod} = 6.97;
$Conf{IncrPeriod} = 0.97;
$Conf{FullKeepCnt} = 1;
The rest are for special cases that you can figure out if you need them.

 > then
> goes off on a tangent for 1-3 paragraphs about how things were done in a 
> previous verson (completely irrevelant),

Irrelevant to you, perhaps.  Not so for people who understood the importance of 
backups a few years ago and now are upgrading.

> or talks about what will be 
> comming (again, irrelevant), or points one to another section of the 
> doc, in the middle of some other thread that is related to the topic but 
> does not address the topic at hand, or,  introduces a topic, then talks 
> about 3-4 other ways to accomplish the same thing, without telling the 
> reader exactly how to do the initial topic to begin with.  So my 
> fault...I need to also read FAQs.

Just look at what the config options let you specify.

>     The probability is either 0 if no bugs in the software (or your
>     configuration of it) or 100% if bugs in the software and your dataset
>     triggers the bug. Your question is not very well-framed and pretty
>     meaningless. I suggest you learn a bit more about backup in general
>     and backuppc in particular. There is a lot of good documentation on
>     BackupPC in the Wikki and in the archives, I suggest you reference it...
> 
> 
> Well, in the extremely simplistic and ideal case, it is 0 or 1.  In the 
> real world, where algorithms are badly designed, or implemented, or 
> both, then problems arise due to things not originally considered in the 
> development which cause errors to arise in the backup procedure that is 
> not about the conversion of 0/1's in computer memory or the transferring 
> of them to tape, disk, etc.

It is still 0 or 1, and if you knew what was wrong with a process in a way that 
could meaningfully be used to calculate the risk of it happening, you'd 
probably 
just fix the problem instead of writing a research paper.  In the real world 
you 
probably won't ever see an error that gets by TCP's correction or rsync's 
algorithm in your lifetime, but you will several times lose the complete disk 
drive that they stored it on.

> The question to you may be meaningless, but it is not to me, and it 
> wasn't to an R&D group that I worked in to look at this precise issue.

It's pretty meaningless in the real world because it is so many orders of 
magnitude lower than mechanical risks that you have to cover anyway. You are 
better off worrying if you'll be hit by lightning as you walk out the door with 
your USB drive.  If you really care about it, you have to do end-to-end tests 
on 
restored content - and you still probably won't account for that possible 
lightning strike.

> Again, the documentation (e.g. manual) for backup PC, leaves a lot to be 
> desired.  While I was reading through the  issues in the forum, it 
> struck me the number of times I came across the phrase "I followed the 
> directions and it did not work/having trouble," etc.   I guess I have to 
> spend a lot of time searching archives to find this info, and then 
> figure out if it applies to this version, or a previous version.

You are over analyzing things.  Backuppc has one, and only one quirk which some 
people have trouble understanding, and that is the way it handles pooling with 
hard links.  Everything else just layers over normal applications that either 
already work or they don't. There's no magic involved and the packaged versions 
should take care of all the details (at the expense of pre-determining the 
archive location).  Actually the hardlinks are a pretty normal OS function as 
well but backuppc uses them more heavily than anything else.

-- 
   Les Mikesell
    lesmikesell AT gmail DOT com



------------------------------------------------------------------------------
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/