Jeffrey J. Kosowsky wrote:
> 
>  > >> Still, it would be awesome to combine the simplicity and pooling
>  > >> structure of BackupPC with the flexibility of a database
>  > >> architecture...
>  > >>   
>  > > I, for one, would be willing to contribute financially and with my very 
>  > > limited skills if Craig, or others, were willing to undertake such an 
>  > > effort. Perhaps Craig would care to comment.
>  > 
>  > The first thing needed would be to demonstrate that there would be an 
>  > advantage to a database approach - like some benchmarks showing an 
>  > improvement in throughput in the TB size range and measurements of the 
>  > bandwidth needed for remote replication.
> 
> No one ever claimed that the primary advantages of a database
> approach is throughput. The advantages are really more about
> extensibility, flexibility, and transportability. If you don't value
> any of the 7 or so advantages I listed before, then I guess a database
> approach is not for you.

I just consider a filesystem to be a reasonable place to store backups 
of files, where a database is a stretch, and I know how to deal with 
most of the problems with filesystems and what to expect from them where 
databases introduce a whole new set of issues.  What's the equivalent of 
fsck for a corrupted database and how long does it take to fix a TB of data?

> Also, while clearly, a database approach would in general have more
> computational overhead (at least for backups), from my experience the
> bottlenecks are network bandwidth and disk speed. In fact, some people
> have implemented BackupPC to run native on a 500MHz ARM processor
> without effective slowdown. (On the other hand, restore-like
> operations would likely be faster since it would be simpler to walk
> down the hierarchy of incremental backups) So, I don't think you would
> find any significant slowdowns from a database approach. If anything a
> database approach could allow significantly *faster* backups since the
> file transfers could be split across multiple disks which is not
> possible under BackupPC unless you use LVM.

Again, I know how to deal with filesystems and I'd use a raid0/10/6 if I 
wanted to split over multiple disks.  But I don't because I want to be 
able to sync the whole thing to one disk that I can remove and I want to 
be able to access data from any single disk just by plugging it in to 
some still-working computer.

>  > Personally I think the way to make things better would be to have a 
>  > filesystem that does block-level de-duplication internally. Then most of 
>  > what backuppc does won't even be necessary.   There were some 
>  > indications that this would be added to ZFS at some point, but I don't 
>  > know how the Oracle acquisition will affect those plans.
> 
> Ideally, I don't think that the backup approach should depend on the
> underlying filesystem architecture. 

It wouldn't depend on it, it would just mean that you might be able to 
store 10x or more data for the same price where there is a lot of 
redundancy.

> Such a restriction limits the
> transportability of the backup solution just as currently BackupPC
> really only works on *nix systems with hard links.

Transportability?  I can access my backuppc disk copy using a USB 
adapter cable from a vmware instance of linux on my laptap while it is 
also still running windows.  I can do the same from a Mac, probably with 
the free virtualbox if I didn't want to pay for Vmware fusion.  You can 
boot just about anything with a CD or USB drive into linux and mount it. 
  You can't get much more portable than that - the OS itself is both 
portable and transportable.  And opensolaris under vmware/virtualbox 
would work as well if that's what it takes for a quick remote restore.

> A database approach
> allows one to get away from dependence on specific filesystem features.

Some real world examples please?  Are you thinking of replicating from 
one OS to another?

> That doesn't mean there isn't room for specialized filesystem
> approaches but just that such a requirement limits the audience for
> the backup solution since it will be a while before we all start
> running ZFS-type filesystems and then we will have the issue of
> requiring different optimizations and code for different filesystem
> approaches.

We already do have the issue of different optimizations for different 
filesytems - and databases are even worse.

-- 
   Les Mikesell
    lesmikesell AT gmail DOT com



------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises 
looking to deploy the next generation of Solaris that includes the latest 
innovations from Sun and the OpenSource community. Download a copy and 
enjoy capabilities such as Networking, Storage and Virtualization. 
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/