Les Mikesell wrote at about 01:40:27 -0500 on Tuesday, September 1, 2009:
 > Jeffrey J. Kosowsky wrote:
 > >  > >  > > I guess I can't answer your question without knowing what use 
 > > cases
 > >  > >  > > you are worried about.
 > >  > >  > 
 > >  > >  > All of them.
 > > 
 > > Name the top 10 that worry you most so that I can get a feeling for
 > > how hard they are to solve. Again, I can't address a generic fear.
 > 
 > 1) you've run out of disk space, corrupting the database
Would cause similar problem with current implementation - such that
either attrib files not created or links not created between pc and
pool trees or only partial files transferred. Similar solution would
be to have a variable like current DfMaxUsagePct to make sure that
backup doesn't run (or stops gracefully) if filesystem close to
full. Also, all databases need to deal with such issues so it is
hardly a new problem without reasonable solution.
 > 2) you lose power/crash while expiring
This would just mean that expiry not completed meaning that some
expired files not deleted. Same problem occurs if power/crash during
BackupPC_Nightly. The actual deletion is atomic. Just run expiry again.
 > 3) you lose power/crash while adding files
Similar problem occurs if power/crash while running BackupPC_link.
Similar solution would be to run a consistency check program like I
have written for current solution.
 > 4) you sync the db and files to another machine - do they still
          match?
Certainly much less of a problem than currently trying to sync pc and
pool trees to another machine. Currently, when you do a "block copy"
you end up either having to unmount partition or make a shadow
copy.

- Best solution:
           Stop BackupPC
           Sync pool and database
           Restart BackupPC
- Alternative solution if you don't want to fully stop BackupPC.
           Make sure BackupPC_Nightly (i.e., expiry not running)
           Export database in consistent state
           Sync database
           Sync pool
           Run analog of BackupPC_Nightly on new copy to expire any
                   additional pool files added during sync
           Restart BackupPC_Nightly on source

- More generally, if you are worried about consistency then run
  analogous tools to those I have created for the current
  implementation. Though, if anything the problem is more severe in
  the current implementation since there remains great uncertainty
  about the reliability of any rsync of a large pool/pc tree. I think
  I remember Holger claiming that nobody has really tested that.

 > 5) concurrent hash collisions with different processes adding files
Easily avoidable since they all funnel through same database which can
use locking/queuing. No different from multiple
users/processes/transactions using any database.

 > There's probably plenty more...

I'm sure there are -- and solutions are likely to be just as simple
and often analogous or identical to problems with existing
implementation.

Again, nobody ever claimed this would be a "trivial" extension. Just
that fault cases and 'bugs' are going to be a problem to any major
change. Now for those happy with the current stable implementation,
that alone could be a very good reason not to change since any major
change introduces the problem of new bugs (and that would also go for
things like a major rewrite to include protocol 30). But that is not a
reason to invalidate the approach if other people think it brings
enough advantages.

Interestingly, though you don't list the one issue that could be more
of a fundamental problem which is that a major database corruption
could lose your entire backup database. That would be my biggest fear
and would require regular backup of the database itself (though not
necessarily the pool, particularly if you included full file md5sums
to resolve chain renumbering ambiguities). Also, you might want to
occasionally run consistency checks on the database itself. But in my
mind this issue dwarfs the others you mentioned and is the only one
that really has no good parallel in the current implementation.


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/