ADSM-L

Re: [ADSM-L] include/exclude pattern matching *.* vs *

2013-04-18 16:53:23
Subject: Re: [ADSM-L] include/exclude pattern matching *.* vs *
From: Andrew Raibeck <storman AT US.IBM DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Thu, 18 Apr 2013 16:49:13 -0400
> Does the OBJECTS of the schedule override client side INCLUDEs?

Based on what you've provided, the question has no real meaning, as there
is nothing to override. It doesn't matter. The INCLUDE is moot.

Now if you were asking whether OBJECTS overrides an EXCLUDE statement, the
answer to that is easy: no, it does not.

Consider this simple dsm.opt file:

   tcpserveraddress my.tsmserver.ibm.com
   passwordaccess generate
   include c:\users\*.*

If I define a schedule with ACTION=INCREMENTAL and OBJECTS=C: then what I
get is an incremental backup of the C: drive.

As TSM scans the C: drive, it will check each file against the
include/exclude list with these results:

1. If the file matches the pattern "c:\users\*.*" then it will be backed
up. For example, c:\users\list.txt will be backed up.

2. If the file does not match the pattern "c:\users\*.*" then it will be
backed up. For example, c:\data\bowling_tourney.xls will be backed up. This
is because any file is implicitly included if there is no EXCLUDE statment
to exclude it.

Now if you are trying to bind files to a certain management class, then the
INCLUDE statement makes a difference. For example, if my above INCLUDE
statement were written like this:

   include c:\users\*.* usermc

Then you have results like this:

* c:\users\list.txt will be bound to usermc beause it matches the above
pattern.

* c:\users\list will be bound to the default manaegment class (because it
does not match *.* -- this is where the *.* versus * comes into play)

* c:\users\andy\report.txt will be bound to the default management class
(there is where c:\users\*.* versus c:\users\...\*.* comes into play)

* c:\data\bowling_tourney.xls will be bound to the default management class

But in no way does the idea of "OBJECTS overriding INCLUDE" come into play.

The other case where INCLUDE matters is if your include/exclude list has
EXCLUDE * at the very top, e.g.:

   tcpserveraddress my.tsmserver.ibm.com
   passwordaccess generate
   exclude *
   include c:\users\*.*

In this case, files in directory c:\users (but not subdirectories of
c:\users) will be backed up. Everything else is excluded.

In your case, \\rose.adm.adp.vcu.edu\sbhealthadmin appears as a (network)
file system, so it is analogous to c: in my preceding examples. But the
same principles apply. If you are just using INCLUDE statements, with no
management class binding, and no EXCLUDE statements, then the INCLUDE
statements are moot (and only serve, apparently, to cause confusion. :-)

When you say nothing is backed up, can you provide more detail? What does
the dsmsched.log file show? What are the ending statistics? Does it scan a
number of objects commensurate with what you'd expect for the file system?
Or is it scanning only a small handful of objects? Are you getting any
error messages?

- Andy

____________________________________________________________________________

Andrew Raibeck | Tivoli Storage Manager Level 3 Technical Lead |
storman AT us.ibm DOT com

IBM Tivoli Storage Manager links:
Product support:
http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Storage_Manager

Online documentation:
https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/Tivoli
Documentation Central/page/Tivoli Storage Manager
Product Wiki:
https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/Tivoli
+Storage+Manager/page/Home

"ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu> wrote on 2013-04-18
15:19:04:

> From: Zoltan Forray <zforray AT VCU DOT EDU>
> To: ADSM-L AT vm.marist DOT edu,
> Date: 2013-04-18 15:20
> Subject: Re: include/exclude pattern matching *.* vs *
> Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu>
>
> To clarify:
>
> This "head" server is used to backup >40 CIFS shares.  Each share has its
> own node and thus their own schedule (gives each department control over
> their backups and restores).
>
> The schedule contains the share name via OBJECT, e.g. "OBJECTS=\\
> rose.adm.adp.vcu.edu\sbhealthadmin" - this has always worked...
>
> At the same time, each shares/nodes dsm.opt (not my decision - I told
them
> it was redundant), has an INCLUDE \\rose.adm.adp.vcu.edu\sbhealthadmin
>
> My question is, if I have the path correct in the schedule/OBJECTS
(without
> the *.*) but they have the INCLUDE statement wrong (e.g. INCLUDE \\
> rose.adm.adp.vcu.edu\sbhealthadmin\*.*)..........Who Is On First?
>
> Does the OBJECTS of the schedule override client side INCLUDEs?
>
> On Thu, Apr 18, 2013 at 3:05 PM, Andrew Raibeck <storman AT us.ibm DOT com>
wrote:
>
> > Hi Zoltan,
> >
> > I don't quite understand the question. If the file is INCLUDEd, how
would
> > OBJECTS change this?
> >
> > In looking over your initial post again (I read it too quickly the
first
> > time), something doesn't seem quite right:
> >
> > > We are troubleshooting a backup problem (nothing is being backed up)
and
> > > the OBJECTS option on the schedule says INCLUDE \\server\folder\*.*
> >
> > Are you saying that the schedule definition has an OBJECTS parameter
coded
> > like this?
> >
> >    objects="INCLUDE \\server\folder\*.*"
> >
> > If so, then that is not right, as the TSM client will interpret this as
> > trying to back up (a) a file called INCLUDE and (b) all files in
directory
> > \\server\folder whose names do not contain a dot '.' character.
> >
> > The OBJECTS parameter should specify only the file specification(s) you
> > want to back up, not TSM keywords. So this is probably more like what
you
> > want:
> >
> >    objects=\\server\folder\*
> >
> > and if you want to traverse subdirectories of \\server\folder, add this
to
> > the schedule definition as well:
> >
> >    options=-subdir=yes
> >
> > Think of the OBJECTS and OPTIONS parameters as containing the same
items
> > that you would use if you were running a command line backup. In this
> > example:
> >
> >    dsmc incremental \\server\folder\* -subdir=yes
> >
> > you could create a schedule with OBJECTS and OPTIONS values like I show
> > above.
> >
> > Another example:
> >
> >    dsmc incremental C:\Users\Zoltan\* C:\Users\Andy\* -subdir=yes
> > -compression=yes -compressalways=no
> >
> > Then you would create a schedule with this:
> >
> >    objects="C:\Users\Zoltan\* C:\Users\Andy\*"
> >    options="-subdir=yes -compression=yes -compressalways=no"
> >
> > Where the INCLUDE option comes in to play is if:
> >
> > (a) You want these files bound to a specific management class. In such
a
> > case, you would put an INCLUDE statement in the dsm.opt file like this:
> > include \\server\folder\...\* mymgmtclass
> >
> > (b) You have an "EXCLUDE *" statement at the very top of your
> > include/exclude list, which would cause all files not already included
to
> > be excluded. For example:
> >
> >    * This EXCLUDE at the top of the list causes files not included to
be
> > excluded
> >     exclude *
> >    * The follwing INCLUDEd files will be backed up
> >    include \\server\folder\...\*
> >    include c:\users\...\*
> >    include c:\data\...\*
> >    * and so on and so forth
> >
> > Note that INCLUDE and EXCLUDE, without any of the dot-extensions (e.g.,
> > EXCLUDE.DIR), operate only on files. They have no effect on
directories. So
> > if your include-list consisted only of this:
> >
> >    exclude *
> >
> > and you ran an incremental backup on a file system, no file objects
would
> > be backed up, but the directory structure would still be backed up.
> >
> > If this information is doesn't help answer the question, then it would
help
> > to have more detail about what you want to do, and what your current
client
> > options already look like.
> >
> > Note that the backup-archive client manuals have information about
using
> > INCLUDE and EXCLUDE (see "Online documentation" in my sig below).
> >
> > - Andy
> >
> >
> >
____________________________________________________________________________

> >
> > Andrew Raibeck | Tivoli Storage Manager Level 3 Technical Lead |
> > storman AT us.ibm DOT com
> >
> > IBM Tivoli Storage Manager links:
> > Product support:
> >
> > http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/
> Tivoli_Storage_Manager
> >
> > Online documentation:
> >
https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/Tivoli
> > Documentation Central/page/Tivoli Storage Manager
> > Product Wiki:
> >
https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/Tivoli
> > +Storage+Manager/page/Home
> >
> > "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu> wrote on 2013-04-18
> > 13:53:40:
> >
> > > From: Zoltan Forray <zforray AT VCU DOT EDU>
> > > To: ADSM-L AT vm.marist DOT edu,
> > > Date: 2013-04-18 14:03
> > > Subject: Re: include/exclude pattern matching *.* vs *
> > > Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu>
> > >
> > > Does the OBJECTS value on the schedule override an INCLUDE in the
> > dsm.opt?
> > >
> > > -------------
> > > Zoltan Forray
> > > TSM Software & Hardware Administrator
> > > Virginia Commonwealth University
> > > UCC/Office of Technology Services
> > > zforray AT vcu DOT edu - 804-828-4807
> > > Don't be a phishing victim - VCU and other reputable organizations
will
> > > never use email to request that you reply with your password, social
> > > security number or confidential personal information. For more
details
> > > visit http://infosecurity.vcu.edu/phishing.html
> > > On Apr 18, 2013 10:46 AM, "Andrew Raibeck" <storman AT us.ibm DOT com>
wrote:
> > >
> > > > Hi Zoltan,
> > > >
> > > > Yes, there is a difference. Don't confuse old-style DOS wildcard
> > matching
> > > > with TSM wildcard matching.
> > > >
> > > > '*' matches zero more more characters. Thus:
> > > >
> > > >    *.*
> > > >
> > > > will match file names that have a '.' in them, e.g.:
> > > >
> > > >    charts.xls
> > > >    myfile.
> > > >    .profile
> > > >
> > > > It will NOT match files that do not contain a dot, e..g:
> > > >
> > > >    myfile
> > > >    profile
> > > >
> > > > Best regards,
> > > >
> > > > - Andy
> > > >
> > > >
> > > >
> >
> >
____________________________________________________________________________

> >
> > > >
> > > > Andrew Raibeck | Tivoli Storage Manager Level 3 Technical Lead |
> > > > storman AT us.ibm DOT com
> > > >
> > > > IBM Tivoli Storage Manager links:
> > > > Product support:
> > > >
> > > > http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/
> > > Tivoli_Storage_Manager
> > > >
> > > > Online documentation:
> > > >
> >
https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/Tivoli
> > > > Documentation Central/page/Tivoli Storage Manager
> > > > Product Wiki:
> > > >
> >
https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/Tivoli
> > > > +Storage+Manager/page/Home
> > > >
> > > > "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu> wrote on
2013-04-18
> > > > 10:34:30:
> > > >
> > > > > From: Zoltan Forray <zforray AT VCU DOT EDU>
> > > > > To: ADSM-L AT vm.marist DOT edu,
> > > > > Date: 2013-04-18 10:39
> > > > > Subject: include/exclude pattern matching *.* vs *
> > > > > Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu>
> > > > >
> > > > > Is there a real difference in an INCLUDE statement using \*.* vs
\* ?
> > > > >
> > > > > We are troubleshooting a backup problem (nothing is being backed
up)
> > and
> > > > > the OBJECTS option on the schedule says INCLUDE \\server\folder
\*.*
> > > > >
> > > > > Is there a really good definitive document that picks through all
> > these
> > > > > kinds of wildcard patters to show what would be
includes/excluded?
> > All
> > > > of
> > > > > the examples I could root out show using 1-asterisk/*  but none
> > showed 2
> > > > *.*
> > > > >
> > > > > --
> > > > > *Zoltan Forray*
> > > > > TSM Software & Hardware Administrator
> > > > > Virginia Commonwealth University
> > > > > UCC/Office of Technology Services
> > > > > zforray AT vcu DOT edu - 804-828-4807
> > > > > Don't be a phishing victim - VCU and other reputable
organizations
> > will
> > > > > never use email to request that you reply with your password,
social
> > > > > security number or confidential personal information. For more
> > details
> > > > > visit http://infosecurity.vcu.edu/phishing.html
> > > > >
> > > >
> > >
> >
>
>
>
> --
> *Zoltan Forray*
> TSM Software & Hardware Administrator
> Virginia Commonwealth University
> UCC/Office of Technology Services
> zforray AT vcu DOT edu - 804-828-4807
> Don't be a phishing victim - VCU and other reputable organizations will
> never use email to request that you reply with your password, social
> security number or confidential personal information. For more details
> visit http://infosecurity.vcu.edu/phishing.html
>