Les Mikesell wrote at about 10:06:40 -0500 on Tuesday, June 2, 2009:
 > Peter Walter wrote:
 > > 
 > >> Still, it would be awesome to combine the simplicity and pooling
 > >> structure of BackupPC with the flexibility of a database
 > >> architecture...
 > >>   
 > > I, for one, would be willing to contribute financially and with my very 
 > > limited skills if Craig, or others, were willing to undertake such an 
 > > effort. Perhaps Craig would care to comment.
 > 
 > The first thing needed would be to demonstrate that there would be an 
 > advantage to a database approach - like some benchmarks showing an 
 > improvement in throughput in the TB size range and measurements of the 
 > bandwidth needed for remote replication.

No one ever claimed that the primary advantages of a database
approach is throughput. The advantages are really more about
extensibility, flexibility, and transportability. If you don't value
any of the 7 or so advantages I listed before, then I guess a database
approach is not for you.

Also, while clearly, a database approach would in general have more
computational overhead (at least for backups), from my experience the
bottlenecks are network bandwidth and disk speed. In fact, some people
have implemented BackupPC to run native on a 500MHz ARM processor
without effective slowdown. (On the other hand, restore-like
operations would likely be faster since it would be simpler to walk
down the hierarchy of incremental backups) So, I don't think you would
find any significant slowdowns from a database approach. If anything a
database approach could allow significantly *faster* backups since the
file transfers could be split across multiple disks which is not
possible under BackupPC unless you use LVM.

 > Personally I think the way to make things better would be to have a 
 > filesystem that does block-level de-duplication internally. Then most of 
 > what backuppc does won't even be necessary.   There were some 
 > indications that this would be added to ZFS at some point, but I don't 
 > know how the Oracle acquisition will affect those plans.

Ideally, I don't think that the backup approach should depend on the
underlying filesystem architecture. Such a restriction limits the
transportability of the backup solution just as currently BackupPC
really only works on *nix systems with hard links. A database approach
allows one to get away from dependence on specific filesystem features.
That doesn't mean there isn't room for specialized filesystem
approaches but just that such a requirement limits the audience for
the backup solution since it will be a while before we all start
running ZFS-type filesystems and then we will have the issue of
requiring different optimizations and code for different filesystem
approaches.

------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises 
looking to deploy the next generation of Solaris that includes the latest 
innovations from Sun and the OpenSource community. Download a copy and 
enjoy capabilities such as Networking, Storage and Virtualization. 
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/