BackupPC-users

Re: [BackupPC-users] Problems with hardlink-based backups...

2009-08-30 17:24:03
Subject: Re: [BackupPC-users] Problems with hardlink-based backups...
From: Les Mikesell <lesmikesell AT gmail DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Sun, 30 Aug 2009 15:54:25 -0500
dan wrote:
>     The hybrid system also has many other advantages including:
>     - Allows Backuppc to work on OS's/FS's that don't support Unix-type
>      hardlinks such as Windoze
> 
> Here is an important point.  Using SQL to store metadata opens up the 
> platforms backuppc can run on.  Considering that no filesystem is ideal 
> for backuppc because of the mix of I/O intensity, hardlinking needs, and 
> operating system support) it would be very nice to de-couple backuppc 
> from the filesystem.  If an effort was made to put metadata on an SQL 
> database, you could even pick and choose your database based on 
> preference and performance.  You could pick a filesystem that was best 
> suited to the data you planned to backup. 

You seem to have the illusion that sql can magically avoid the head motions 
that 
make backuppc slow while still getting the same things on the same disks.  
While 
it is possible to tune most sql servers to put different tables on different 
drives, there's a fair chance that in a default configuration it will perform 
worse than using links.


>     - Allows for more expandable, robust, and faster storage of
>      metadata. Continuing to expand attrib files to include ACL's and
>      other extended attributes will just make the hack messier and
>      slower.
> 
> how about small but incredibly fast I/O SSD drives for the database!  
> SSD are just too expensive to use with backuppc right now unless you can 
> use ZFS and use them for cache drives(which is extremely playform limited)

I'm not sure being platform-limited matters that much.  Why wouldn't I be able 
to install OpenSolaris/zfs on anything where I'd be likely to run a database?

>     - Allows for more granular security and access controls to backups
> 
> How about much much easier PHP work getting backup info and a faster 
> backuppc interface only having to hit the database for all its info and 
> not having to touch the filesystem.

I don't have a lot of use for backuppc info other than knowing that it 
completed.

> One of the biggest concerns with backuppc that is constantly discussed 
> on this list is syncing the backup data between two or more servers.  
> Simply reducing the file count by eliminating the hardlinks would allow 
> rsync to be used reliably and effectively.  SQL replication can keep 
> metadata updated constantly and a watchdog that monitors the SQL for 
> changes could keep the filesystems that store data synced easily as 
> well.

Maybe, but again it won't do what you want by default.  Most sql replication 
schemes work more or less in real time which probably isn't what you want at 
all.

 >  Once the metadata and config moves to a database, so many things
> become very easy.

No, they just become different. You now have to use database-specific tools to 
touch anything and if the file contents aren't included in the database you've 
made it impossible to do atomic operations.

> A single backuppc server could handle many more 
> concurrent backups because multple data storage devices can seperate IO 
> and relieve the pressure on the IO system of the OS.

That has yet to be proven.

-- 
   Les Mikesell
    lesmikesell AT gmail DOT com


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>