On Tue, May 18, 2010 at 02:04:46PM -0700, Robin Lee Powell wrote:
> A customer we're backing up has a directory with ~500 subdirs and
> hundreds of GiB of data. We're using BackupPC in rsync+ssh mode.
>
> As a first pass at breaking that up, I made a bunch of seperate host
> entries like /A/*0, /A/*1, ... (all the dirs have numeric names).
>
> That seems to select the right files, but it doesn't work because
> BackupPC ends up running a bunch of them at once, hammering that
> customer's machine.
You can set it up so only a few of those hosts will run at the same
time using
$Conf{UserCmdCheckStatus} = 1;
and a $Conf{DumpPreUserCmd}/$Conf{DumpPostUserCmd} that know the host
names and implement a counting semaphore to make sure only some number
of them are running at the same time. I posted a longer sketch of how
I limit the number of parallel backups to remote sites in the archives
some time ago.
> I could make those into share names, but I'm worried about running
> out of command-line argument space in that case; that is, that
> "rsync /A/*0" will at some point in the future expand to a hundred
> or more directories and break.
>
> What I want to do is have a bunch of shares like [ "/A", "/A", ...],
> and have something like:
>
> $Conf{BackupFilesOnly} = {
> "/A" => "/A/*0",
> "/A" => "/A/*1",
> "/A" => "/A/*2",
> }
>
> But obviously that's not going to work.
>
> Does anyone have any other way to handle this?
How about a set of symbolic links on the host:
/backup_slices/A0
/backup_slices/A1
/backup_slices/A2
/backup_slices/A3
/backup_slices/A4
/backup_slices/A5
...
which all point to /A. Then a spec:
$Conf{BackupFilesOnly} = {
"/backup_slices/A0/." => "/*0",
"/backup_slices/A1/." => "/*1",
"/backup_slices/A2/." => "/*2",
...
}
Not pretty, but I think it will work as the trailing . should cause
rsync to dereference the symbolic link, and the resulting rsync will
start in /A.
--
-- rouilj
John Rouillard System Administrator
Renesys Corporation 603-244-9084 (cell) 603-643-9300 x 111
------------------------------------------------------------------------------
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|