BackupPC-users

Re: [BackupPC-users] clustered file system and multiple servers

2008-08-05 00:57:17
Subject: Re: [BackupPC-users] clustered file system and multiple servers
From: Holger Parplies <wbppc AT parplies DOT de>
To: sabujp <sabujp AT gmail DOT com>
Date: Tue, 5 Aug 2008 06:56:02 +0200
Hi,

sabujp wrote on 2008-08-04 23:18:41 -0400 [[BackupPC-users]  clustered file 
system and multiple servers]:
> 
> Can a dev let me know if the files in the pool are FLOCK'd before writing,

use the force, read the source. "grep -r flock backuppc-3.1.0" suggests that
flock is used, but not on pool files. Not surprising, considering BackupPC_link
and BackupPC_nightly (or two instances of BackupPC_link) may not run
concurrently (BackupPC_link is responsible for entering new files into the
pool).

Come to think of it, the reason for this restriction is of a different nature:
BackupPC_nightly sometimes needs to rename pool files (with a common BackupPC
hash, when one or more files out of the chain are deleted), while BackupPC_link
may insert a new file with the same BackupPC hash. You can't prevent the
resulting race condition with flock() - at least you don't effectively change
anything in the single-threaded case (you'd need a rather global lock).

> i.e. is there a chance that two servers backing up into the same top level
> data directory could "mangle" a file in the pool in this manner?

I don't think you'd have mixed file contents (or, effectively, a corrupt
compressed file), but there seems to be a chance of linking a file in a backup
to the wrong pool file (wrong contents altogether).

You could probably use flock() to prevent two instances of
BackupPC_link/BackupPC_nightly running simultaneously on different servers,
but there are more things you would want to think about (running
BackupPC_nightly on more than one server does not make much sense, even if
they don't run concurrently; limiting simultaneous backups over all servers;
ensuring BackupPC_dump is not run twice simultaneously for the same host ...).

In short: sharing a pool between servers is currently not supported.

Regards,
Holger

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/