Amanda-Users

Backup image tape spanning, a look back and a look ahead...

2005-02-23 10:30:28
Subject: Backup image tape spanning, a look back and a look ahead...
From: Bruce.Skinner AT drdc-rddc.gc DOT ca (Bruce S. Skinner)
To: amanda-hackers AT amanda DOT org
Date: 23 Feb 2005 14:04:50 +0000
Hello,

I'm looking for some feedback from the Amanda team on the future of
tape spanning of backup images in Amanda.

It always pleases me when I see Free and Open Source software score a
success and it would be nice to see Amanda as one of those successes.
You may say "What's he talking about?  Amanda is already a success!"
and you would be right, but only up to a point.  So before you
discount these comments as those of a newbie who doesn't see the "big
picture," please read on a little and consider.

My experience with computer system backup goes back to the days when
we used punched paper tape and cards to do backups.  I've used 800 BPI
to 6250 BPI 9-tracks, exabyte's, DAT's, various IBM cartridge's, DLT's
of EVERY type, as well as some really exotic backup media like optical
tape.  Our lab has been involved in Free and Open Source development
since long before the term was coined.  We had the first ARPANet site
in Canada back when TCP/IP was initially being developed.  We have
developers on staff who made contributions to Stallman's original
TOPS-20 EMACS and we have spent lot's of money over the years buying
support for things like the GNU development tools knowing full well
that in the long run everyone would benefit from the efforts of those
programmers who got to work full time on the tools because our support
dollars went to pay their salaries.  As a result I feel that I have
some perspective to bring to this issue.

As I said, it would be nice to see Amanda as one of those successes.

In order for a software package to be a success, it has to reach a
critical mass or it winds up dying.  In my experience Free Software
that has reached critical mass has a much longer lifetime than any
proprietary stuff (I use GNU Emacs and the Gnu Tools for software
development and see no end in sight for them).  Amanda risks
maintaining it's critical mass every time some IT shop considers
choosing it as their backup solution and then walks away because
Amanda assumes that individual tape media will ALWAYS be bigger than
individual filesystems.

This assumption has not always been so in the past and just because it
has been so for a few years I see no reason why we should expect it to
continue to be so in the future.  There will always be a race between
technologies for the biggest or fastest and there is no telling who is
going to be out front next week, let alone next year.  It is not
reasonable to design a system that you know is likely to break
whenever disk capacity pulls ahead in the race.  If in that
eventuality you are going to have to adopt another backup solution,
the reasonable choice is to adopt that other backup solutiuon now.

I understand that the Amanda developer's concern is about reliability
and it should be.  However, our concept of what constitutes a reliable
system is subject to change every time we change our perspective.

For instance, if we focus on a specific instance of a system and note
that it is available to it's users 100% of the time 24 hours/day and 7
days/week and allows those users to perform the job that the system
was originally specified to perform, we could define that system as a
100% reliable system.

Let us now expand our view to encompass a larger portion of the parent
organization.  We still see the system we were just focused on is
humming away at it's originally intended job.  We also see a new
system constructed in the room next door at considerable effort and
expense.  This new system will perform not only the job of the first
system, but some expansion to the original requirements of that
system.  This is an expansion that the organization would have made
some time ago, but the original system, just wasn't flexible enough to
allow the expansion, so the organization put off spending the
resources for an entire new replacement system because of the large
one-time cost.  In the meantime, the lack of the expanded
functionality has cost them business, set back their research program
or whatever it is they do.  Was the first system 100% reliable?  Did
it fulfill the organization's requirements all of the time?  In my
view, it didn't

In my experience, a reliable computer system is one that can evolve
steadily and smoothly along with the needs of the parent organization
without causing major disruptions in how the organization operates.
When I look at candidates for a system component, one of the first
things I assess is it's long term viability.  If the component is an
expensive one (and compared to other computer system components
reliable tape drives historically are) then that component better have
a long useful lifetime and the lifetime of the software I use with
that expensive hardware had better match or exceed the lifetime of
that hardware.

Our lab has paid good money for backup solutions in the past and will
continue to do so in the future.  We are yet again looking for a
backup solution.  I'm providing recommendations for software and would
love to recommend Amanda but unless I see a commitment to include
image splitting in Amanda I can't recommend it and that would be a
pity.


regards :-)
BruceS


BTW.  One of my favourite backup solutions was VMS backup,
particularly in the days of 9-track tapes when tape and drive
reliability was a real issue.  To counter this, VMS backup wrote lot's
of redundenacy info on the tapes and on a number of occasions, when I
had managers raising questions about the reliability of various backup
solutions, I enjoyed demo'ing VMS backup's reliability by breaking a
tape in the middle and cutting a few inches out of the middle of the
tape and splicing it back together.  Upon restore VMS backup would
note "Tape Error - restoring from redundancy block" and continue a
flawless restore. :-)

 ::backup:

-- 

Bruce S. Skinner
Defence R&D Canada - Atlantic
9 Grove St.              <mailto:Bruce.Skinner AT drdc-rddc.gc DOT ca>
P.O. Box 1012            <http://www.drdc-rddc.dnd.ca>
Dartmouth  NS
CANADA                    tel: (902) 426-3100 x205
B2Y 3Z7                   fax: (902) 426-9654