Amanda-Users

Re: Disklist - partial directory backup - advice please

2002-08-19 21:06:56
Subject: Re: Disklist - partial directory backup - advice please
From: John Ouellette <ouellet AT amnh DOT org>
To: Gene Heskett <gene_heskett AT iolinc DOT net>
Date: Mon, 19 Aug 2002 20:47:52 -0400 (EDT)
Ok, now that's interesting...  I'll agree that the fact that you're using
2.4.3b3 and aren't having problems indicates that you're right.  
However...  The comments in the amanda.conf file say:

"Note that the `full pathname' of a file within its
filesystem starts with `./', because of the way amanda runs
gnutar: `tar -C $mountpoint -cf - --lots-of-options .' (note
the final dot!)  Thus, if you're backing up `/usr' with a
diskfile entry like ``host /usr gnutar-root', but you don't
want to backup /usr/tmp, your exclude list should contain
the pattern `./tmp', as this is relative to the `/usr' above.
Please refer to the man-page of gnutar for more information.
If a relative pathname is specified as the exclude list,
it is searched from within the directory that is
going to be backed up."

The man page says:

" For exclude list, If the file name is relative, the
disk name being backed up is prepended.  So if this
is entered:

           exclude list ".amanda.excludes"

the actual file use would be  /var/.amanda.excludes
for  a  backup of /var, /usr/local/.amanda.excludes
for a backup of /usr/local, and so on."

These suggest that the exclude pattern, whether as a exclude pattern or a 
file containing a list of patterns, is supposed to have *relative* 
pathnames, not absolute pathnames.  Yes, your list is relative, but your 
path for the exclude list file itself is absolute 
(/usr/local/etc/amanda/exclude.gtar) which is not quite what the docs 
recommend....

I'll try some absolute paths later: if they work, it suggests a 
documentation bug, rather than a coding one... 

J.

On Mon, 19 Aug 2002, Gene Heskett wrote:

> On Monday 19 August 2002 20:09, John Ouellette wrote:
> >We're going a bit far afield from what Kevin had originally posted
> > but oh well.
> >
> >Are you using version 2.4.2, Gene?
> 
> No, 2.4.3b3-20020805 right now.  And I'm not having any known 
> problems, none.  And other than some broken scripting in the 
> amstatus command, I've had zilch troubles with amanda in the last 4 
> or more months, thru a goodly number of 2.4.3b* versions.
> 
> >  Kevin and I are using v2.4.3b3
> > and have found the same problem with exclude lists.  What I have
> > found (again, with v2.4.3b3) was that the exclude keyword in a
> > dumptype definition *causes the size estimates from tar to fail*.
> >  The debug info from Amanda shows that tar is never even called. 
> > From one of my tests (sendsize.*.debug):
> >
> >++++++++++++++++++++++++++++++++++++++++++
> >sendsize: debug 1 pid 27066 ruid 499 euid 499 start time Mon Aug
> > 19 14:31:57 2002
> >/home/amanda/libexec/sendsize: version 2.4.3b3
> >sendsize: calculating for amname '/', dirname '/'
> >sendsize: getting size via gnutar for / level 0
> >++++++++++++++++++++++++++++++++++++++++++
> 
> Any one of my copious numbers of sendsize.*.debug files is exactly 
> like the rest, containing only a copy of the exclude file as 
> follows:
> -----
> ./usr/dumps/*
> ./var/pacct
> -----
> So I again submit that the exclude logic isn't broken, but that the 
> file itself simply isn't being found due to a lack of a full 
> pathlist in the dumptype specification, as is shown that I am using 
> below.
> 
> >(Note that there were three other partitions called with the same
> >dumptype: the size estimates weren't even attempted by amanda --
> > the above is the entire contents of the debug file.)  This with a
> > dumptype definition of:
> >
> 
> This is yours
> 
> >define dumptype TEST {
> >    global
> >    comment "test dumptype"
> >    program "GNUTAR"
> >    compress none
> >    index
> >    exclude "./amanda" <---whats the current pwd when this 
> executes?
> >    record no
> >}
> >
> >(The idea of the exclude was the same as Kevin's: to exclude the
> > holding disk areas from those disks.)  The result is the same if
> > I use:
> >
> >exclude "./amanda*"
> >exclude "*amanda*
> >exclude list ".amanda.exclude"
> >exclude list "./.amanda.exclude"
> >
> >etc., in the dumptype definition.  The exclude keyword seemed to
> > work in v2.4.2p2, but I decided to upgrade to 2.4.3b3 to get
> > other features working (I'm not sure if it was worth it now...).
> 
> Thats odd. I also note that you still aren't giving it a full path 
> to the exclude file.  Its possible that the fact that it can't find 
> it might be enough of an error to cause the tar call to fail.
> 
> >I've taken a brief look into the code for how exclude lists are
> > handled, but there was such a big change from v2.4.2 to v2.4.3,
> > I'm not sure if I'll be able to figure out what broke the exclude
> > keyword...
> >
> >Note that in the amandad debug file, the exclude list seems to be
> > parsed OK, so it really must be in the call to sendsize that the
> > error is occurring...
> >
> >J.
> >
> >On Mon, 19 Aug 2002, Gene Heskett wrote:
> >> On Monday 19 August 2002 16:12, John Ouellette wrote:
> >> >I've played around with the exclude lists for a while (as
> >> > defined in the dumptypes) and have had no luck in getting any
> >> > configs with exclude lists to work, whether I define the paths
> >> > with the leading ./ or no....
> >> >
> >> >Kevin was using an exclude list defined in the disklist, but I
> >> > assume it is the same code that parses this and passes it on
> >> > to 'tar', so the same bug might exist.
> >> >
> >> >And, according to the docs and examples, you do need the
> >> > quotes... (or get a parse error if you exclude them).
> >>
> >> I'm not here, but I'm using an exclude file too, here it is:
> >> ---------------------
> >> ./usr/dumps/*
> >> ./var/pacct
> >> ---------------------
> >>
> >> Also, I recall the filename didn't give the full path in your
> >> earler example, which should look like this in your amanda.conf:
> >> ------------------------------------------
> >> define dumptype root-tar {
> >>     global
> >>     program "GNUTAR"
> >>     comment "root partitions dumped with tar"
> >>     compress none
> >>     index yes
> >>     exclude list "/usr/local/etc/amanda/exclude.gtar"
>       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >>     priority low
> >> }
> >>
> >> define dumptype user-tar {
> >>     root-tar
> >>     comment "user partitions dumped with tar"
> >>     exclude list "/usr/local/etc/amanda/exclude.gtar"
> Again^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >>     compress server best
> >>     priority medium
> >> }
> >> -----------------etc, etc taking note that each "dumptype" can
> >> have its own unique name-it-whatever-file.
> >>
> >> >I've started to trudge through the code to see if the bug is
> >> > obvious, but haven't managed to get far.
> >> >
> >> >J.
> >> >
> >> >On Mon, 19 Aug 2002, Gene Heskett wrote:
> >> >> On Monday 19 August 2002 12:00, Kevin Passey wrote:
> >> >> >Thanks John
> >> >> >
> >> >> >That is what I am using.
> >> >> >
> >> >> >I only what to omit the /dumps folders.
> >> >>
> >> >> Did you specify them as "./dumps" (without the quotes of
> >> >> course) The missing leading dot in your line above says its
> >> >> your exclude file thats broken.   Tar's exludes are relative
> >> >> to the currently being processed directory, hence the leading
> >> >> ./
> >> >>
> >> >> >I think I will have to create them in a separate partition.
> >> >> >
> >> >> >Thanks for that.
> >> >> >
> >> >> >Regards
> >> >> >
> >> >> >Kevin
> >> >> >
> >> >> >
> >> >> >
> >> >> >-----Original Message-----
> >> >>
> >> >> From: John Ouellette [mailto:ouellet AT amnh DOT org]
> >>
> >> [...]
> 
> 

-- 
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
John Ouellette                     | Ph: 212-313-7919 
Department of Astrophysics         | Fax: 212-769-5007 
American Museum of Natural History | e-mail: ouellet AT amnh DOT org
Central Park West at 79th St.      | http://research.amnh.org/astrophysics
New York, NY  10024-5192           |
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++