|
Re: [BackupPC-users] clustered file system and multiple servers
2008-08-06 00:16:42
I have investigated cluster filesystems for backuppc. You will have very poor I/O on all cluster filesystems. I/O is the most important factor for backuppc and the performance difference between a local filesystem(or even iscsi or aoe) and a cluster filesystem can be an order of magnitude slower I/O performance. I had hoped to use a cluster filesystem to maintain a redundant copy of my backuppc data but performance was incredibly bad. I find fuse filesystems such as mysqlfs or zfs-fuse to be much faster.
On Mon, Aug 4, 2008 at 10:56 PM, Holger Parplies <wbppc AT parplies DOT de> wrote:
Hi,
sabujp wrote on 2008-08-04 23:18:41 -0400 [[BackupPC-users] clustered file system and multiple servers]:
>
> Can a dev let me know if the files in the pool are FLOCK'd before writing,
use the force, read the source. "grep -r flock backuppc-3.1.0" suggests that
flock is used, but not on pool files. Not surprising, considering BackupPC_link
and BackupPC_nightly (or two instances of BackupPC_link) may not run
concurrently (BackupPC_link is responsible for entering new files into the
pool).
Come to think of it, the reason for this restriction is of a different nature:
BackupPC_nightly sometimes needs to rename pool files (with a common BackupPC
hash, when one or more files out of the chain are deleted), while BackupPC_link
may insert a new file with the same BackupPC hash. You can't prevent the
resulting race condition with flock() - at least you don't effectively change
anything in the single-threaded case (you'd need a rather global lock).
> i.e. is there a chance that two servers backing up into the same top level
> data directory could "mangle" a file in the pool in this manner?
I don't think you'd have mixed file contents (or, effectively, a corrupt
compressed file), but there seems to be a chance of linking a file in a backup
to the wrong pool file (wrong contents altogether).
You could probably use flock() to prevent two instances of
BackupPC_link/BackupPC_nightly running simultaneously on different servers,
but there are more things you would want to think about (running
BackupPC_nightly on more than one server does not make much sense, even if
they don't run concurrently; limiting simultaneous backups over all servers;
ensuring BackupPC_dump is not run twice simultaneously for the same host ...).
In short: sharing a pool between servers is currently not supported.
Regards,
Holger
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
|
|
