Amanda-Users

Re: GNUTAR exclude lists not working in Windows or Linux

2005-06-02 04:59:31
Subject: Re: GNUTAR exclude lists not working in Windows or Linux
From: Jon LaBadie <jon AT jgcomp DOT com>
To: amanda-users AT amanda DOT org
Date: Thu, 2 Jun 2005 02:22:27 -0400
On Wed, Jun 01, 2005 at 10:11:07PM -0700, Joe Rhett wrote:
> On Wed, Jun 01, 2005 at 11:27:11PM -0400, Jon LaBadie wrote:
> > On Wed, Jun 01, 2005 at 07:25:44PM -0700, Joe Rhett wrote:
> > > Okay, last year I had observed that perfectly valid gnutar exclude lists
> > > were being ignored by amanda on Windows machines.  The best answer anyone
> > > could give me was to build my own tar program that does the excludes, and
> > > replace "runtar".
> > 
> > I forget the discussion, but there must have been more than that,
> 
> A bit more, mostly telling me to use SMB instead, which is nonsense.
> 
> > backing up of windows boxes does not use gnutar but smbclient.
> > Different cmd line syntax for excludes.
>  
> Every time I talk about Windows, you keep talking about smbclient.  Nobody,
> flat nobody that I know of is stupid enough to run SMB on a public
> webserver.  The windows machines in question are using amanda under cygwin.

Gee Joe, I apologize.  Somehow I lost track of the point that
you use cygwin.  It should have been obvious to me since in the
past year there was only one posting about cygwin besides yours.
I can't imagine why I might have assumed that like 95% of the
other posters you were using samba to backup a windows box.


> > > And as I noted before, and someone tried to explain away, it appears that
> > > the command line invocation for the exclude list is wrong (missing an 
> > > equals)
> > > runtar.20050601020202.debug:
> > >   running: /bin/tar: gtar --create --file - --directory / 
> > > --one-file-system --listed-incremental 
> > > /var/lib/amanda/gnutar-lists/client-host3__0.new --sparse 
> > > --ignore-failed-read --totals --exclude-from /etc/exclude.gtar . 
> > > 
> > 
> > I don't know what equals sign you feel is missing.
> > The man page I have for gnutar does not show an equal sign
> > needed with the --exclude-from option:
>  
> Some versions of tar out there apparently don't work properly without the
> equals sign.  Search for it, or trust me.  In any case, I agree that it
> isn't affecting these linux boxes -- either syntax appears to work.
> 
> > > /etc/exclude.gtar
> > >   $ cat /etc/exclude.gtar 
> > >   ./*
> > >   ,/
> > >   *
> > 
> > If I recall the syntax correctly, each entry must begin "./"
> > The first is valid and would exclude everything with the possible
> > exception of "dot" files (eg  .profile).
> > Second and third are invalid.
>  
> That's fine, I was trying everything possible.  Right now with those regexs
> I'm backing up 60gb a night from that system.  Wouldn't that suggest
> something is wrong?  I started with just the first regex and added others
> when it didn't work, so it isn't due to the bad regexes...

Actually I think it somehow is.  I ran a few tests with my gtar.
The command line was a pipe from one gtar creating an archive to
another generating a listing.  I wanted to see the effect of
various exclude patterns in the file.

My commandline was like this:

    amgtar --create --exclude-from /var/tmp/exclude \
           --directory /tmp --file - . | 
    amgtar --list --file -

I ran it first without the --exclude-from option, then with the
option and an empty file.  Same large list of files.

When I ran it with the option, but with the exclude file missing
I got an error and no archive was created.

Next I put a simple pair of patterns in the file:

  ./s2
  ./s4

and the file s2 was excluded, there was no s4 to begin with.

The next test was to put in the exclude file a single entry,

   ./*

This resulted in a single line of output, ./
i.e. no files were archived but the directory was noted.

Lastly I put in your three patterns,

   ./*
   ,/*
   *

The result I got was that nothing was archived, not even "."

I looked at the last two a little further, collected the
result of the gtar creation into a file instead of a pipe.

Each command created a file exactly 10K. The one from a
single exclude entry (./*) had a little data in the first
few hundred bytes, then nulls.  The file command recognized
this as a tar file.

The corresponding 10K file created with your three patterns
was simply a file of null bytes.  No data and of course it
was not recognized by the file command as a tar file.

Perhaps running the command by hand like this could help
sort out the exclude patterns for you.

-- 
Jon H. LaBadie                  jon AT jgcomp DOT com
 JG Computing
 4455 Province Line Road        (609) 252-0159
 Princeton, NJ  08540-4322      (609) 683-7220 (fax)