BackupPC-users

Re: [BackupPC-users] BackupFilesExclude strips trailing / on patterns for rsync

2008-10-09 00:15:27
Subject: Re: [BackupPC-users] BackupFilesExclude strips trailing / on patterns for rsync
From: John Rouillard <rouilj-backuppc AT renesys DOT com>
To: backuppc-users AT lists.sourceforge DOT net
Date: Thu, 9 Oct 2008 04:11:46 +0000
On Thu, Oct 09, 2008 at 04:07:48AM +0200, Holger Parplies wrote:
> John Rouillard wrote on 2008-10-08 20:06:21 +0000 [[BackupPC-users]
> BackupFilesExclude strips trailing / on patterns for rsync]: 
> > [...]
> > I have set up a rule like this to eliminate many gigbaytes of files
> > with the following config file entries:
> > 
> >   $Conf {BackupFilesExclude} =
> >    ...,
> >    '/home/yum' => [
> > [...]
> >        "- **/.svn/",
> > [...]
> >        "+ */",
> > [...]
> >      ],
> >   };
> > 
> > After starting and stopping a full backup after 5 minutes I saw
> > (annotated with @ signs):
> > 
> > [...]
> > @  Sent exclude: **/.svn
> > [...]
> > @  Sent include: *
> > 
> > Note the lines marked with @'s. It looks like the trailing /'s from
> > the exclude list for those items is being removed ( "= **/.svn/" is
> > sent as "**/.svn" and "+ */" is sent as "*"). The trailing / is
> > supposed to force these patterns to be interpreted only for
> > directories.
> > 
> > So is this doing the right thing, and the message:
> > 
> >   Sent include: *
> > 
> > should really read:
> > 
> >   Sent include directories only: *
> > 
> > Or am I going to get a huge backup of everything?
> 
> I must admit that I haven't tried it out. I haven't got time for more than a
> quick glance at the source, but in File::RsyncP 0.64, FileList/exclude.c
> function make_exclude I see
> 
>     if (pat_len > 1 && ret->pattern[pat_len-1] == '/') {
>         ret->pattern[pat_len-1] = 0;
>         mflags |= MATCHFLG_DIRECTORY;
>     }
> 
> Without fully understanding the code right now, this seems to mean that the
> trailing / is stored as a flag and deleted from the pattern (which is later
> output), so I would *guess* that it is, in fact, doing the right thing
> (actually, this is consistent with a grep over the last months' mailing list
> e-mails which mention not a single "Sent exclude" log line listed with a path
> with a trailing slash). You might want to try it out with a smaller amount of
> data (maybe backup /tmp with "+ */", "- *" and see if it contains any files)
> or simply abort the backup if you find that it transfers files it should not.

Thanks for looking into this. I agree from my read through of the
code.

Also after 6 hours the backup completed and seems to have worked as
desired.

So the logged output is just misleading. Probably need a bug request
filed. Does backupc have a bug/feature tracker at sourceforge.net? Not
finding a link to it at the moment.

> > Also I had to stop the backup to see any output at all on the
> > XferLOG.z. This is really annoying, is there some flag that can be
> > turned on to get it to flush after every output line?
> 
> I don't believe there is, probably due to the way compression is implemented.

Yeah, I sort of figured it was something like that. Probably waiting
to output the first block of the compressed file with the
dictionary. Maybe a feature request to output the XferLOG as a normal
file and then compress it when the backup is completed is a useful
feature.
 
-- 
                                -- rouilj

John Rouillard
System Administrator
Renesys Corporation
603-244-9084 (cell)
603-643-9300 x 111

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>