Amanda-Users

Re: Wildcards in disklist--revisited, only using exclusions:

2006-11-15 14:06:38
Subject: Re: Wildcards in disklist--revisited, only using exclusions:
From: Jean-Francois Malouin <Jean-Francois.Malouin AT bic.mni.mcgill DOT ca>
To: "Ian R. Justman" <ianj AT ian-justman DOT com>
Date: Wed, 15 Nov 2006 13:28:01 -0500
Hi there,

* Ian R. Justman <ianj AT ian-justman DOT com> [20061115 12:39]:
> Ian R. Justman wrote:
> 
> >Hello.
> >
> >Tried using includes again, and still I'm having the same problem.  It's 
> >backing up nothing.  In fact, one item was still using the exclude 
> >method and it still backed up everything.
> >
> >And for what it's worth, the version of GNU tar that's running on that 
> >machine shows the following when I do a gtar --version:
> >
> >tar (GNU tar) 1.15.1
> >
> >Anything I need to be aware of with this version of GNU tar?
> >
> >Thanks.
> 
> Perhaps I should clarify this message:
> 
> I tried to use wildcards both in the "exclude" and "include" contexts in 
>  disklist stanzas, both with and without trailing slashes.  I seem to 
> be running into the same result each time.  If I use the "include" 
> statement, nothing gets backed up with each stanza I use those wildcards 
> in.  If I use excludes, everything gets backed up as many times as there 
> are stanzas.
> 
> It doesn't seem to matter whether I have the trailing slash or not. 
> What I'm curious about now is whether it's a GNU tar issue, in which 
> case I should change which version of GNU tar I'm running, or if it's 
> something I'm doing wrong.  It all seems to have something to do with 
> what's in files referenced by the --files-from/--exclude-from gtar flags.
> 
> The players in this case are as follows:
> 
> Machine in question being backed up:
> 
> Sun Ultra 5 running Solaris 2.6 (SunOS 5.6)
> AMANDA 2.5.0p2 which I built myself
> GNU tar 1.15.1 from Sunfreeware.com
> 
> Machine which is holding the backups:
> 
> Dual Pentium II system running FreeBSD 4.8
> AMANDA 2.5.0p2 using the amanda-server-2.5.0p2 port

eek. I would upgrade to at least 2.5.1p1 but that's just me.

> 
> The disklist on the FreeBSD machine has the following in the disklist 
> pertinent to the affected host:
> 
> trillian /var/spool/PO/mailbox/0-99 /var/spool/PO/mailbox {
>   user-tar
>   include "./[0-9]/" "./[1-9][0-9]"
> } 1

bad. see below.

> 
> [...]
> 
> trillian /var/spool/PO/mailbox/100-199 /var/spool/PO/mailbox {
>   user-tar
>   include "./1[0-9][0-9]"
> } 1

looks ok

> 
> [...]
> 
> trillian /var/spool/PO/mailbox/200-299 /var/spool/PO/mailbox {
>   user-tar
>   include "/.2[0-9][0-9]"
> } 1

bad

> 
> [...]
> 
> trillian /var/spool/PO/mailbox/300-399 /var/spool/PO/mailbox {
>   user-tar
>    include "./3[0-9][0-9]"
> } 1

looks ok

> 
> [...]
> 
> trillian /var/spool/PO/mailbox/400-499 /var/spool/PO/mailbox {
>   user-tar
>   include "./4[0-9][0-9]"
> } 1

looks ok

I think some of those include have bad syntax. From the amanda manpage.
Read aloud the last two lines:

      include [ list|file ][[optional][ append ][  string ]+]
              Default: file ".". There are two include lists, in-
              clude file and include list. With  include  file  ,
              the  string is a glob expression. With include list
              , the string is a file name on the client  contain-
              ing glob expressions.

              All  include  expressions  are  expanded by AMANDA,
              concatenated in one file and passed to GNU-tar as a
              --files-from  argument.  They  must start with "./"
              and contain no other "/".

One example from a DLE of mine. amanda-2.5.1p1 (soon 2.5.1p2)
and tar (GNU tar) 1.13.25 

yorick assembly_1 /data/assembly { 
    tar-bsdtcpauth-100
    include "./1*"
}

works ok for me. The sendback.assembly_1.<data>.include
and sendsize.assembly_1.<data>.include match the include
in the dle.

You might want to dig what tar is doing in the sendbackup debug
files and test it yourself manually. From one DLE for instance I
had the following:

yorick /data/griffin {
    tar-calc-100
    exclude append "./Experimental_data/*"
} 1

gtar --create --file - --directory /data/griffin
--one-file-system --listed-incremental
/opt/amanda/var/amanda/gnutar-lists/yorick_data_griffin_0.new
--sparse --ignore-failed-read --totals --exclude-from
/tmp/amanda/sendbackup._data_griffin.20061115061105.exclude .

cat /tmp/amanda/sendbackup._data_griffin.20061115061105.exclude
./Experimental_data/*

HTH
jf

> 
> [...]
> 
> trillian /var/spool/PO {
>   user-tar
>   exclude "./mailbox"
> } 1
> trillian /etc user-tar
> 
> I removed the trailing slashes in the wildcarded lines per Jean-Louis' 
> suggestion and it's still not working.  The only stanzas for this 
> machine which actually work are the last two.  The first one is because 
> it does not specify any wildcards in the "exclude" statement.  The 
> second one is because I outright back up that entire directory hierarchy.
> 
> At this point, am I missing anything that might be of any help?
> 
> Is there anything I'm doing wrong at all or should I be using a 
> different version of gtar since it seems to be the root of the problem, 
> and if so, which version?
> 
> Thanks for any help on this matter.
> 
> --Ian.

-- 
<° ><