Basic include/exclude for Linux?

ldmwndletsm

ADSM.ORG Senior Member
Joined
Oct 30, 2019
Messages
232
Reaction score
5
Points
0
PREDATAR Control23

Hi, I want to exclude everything on a Linux client with the exception of one or two file systems and then begin adding more file systems one at a time (maybe several per night depending), but there are too many to run first-time fulls in a single night, and I also want to test that things are getting backed up properly by running only a few at a time, with recover testing.

I've been reading about the include/exclude, but not sure if I have this right. We will be using a server schedule (via dsmcad on the client) with the Action parameter set to INCREMENTAL, and the Object parameter will be left blank so only the dsm.sys file on the client will be used to determine what needs to be backed up. I will probably put the include/exclude statements in their own separate file and reference that in dsm.sys.

1. If I wanted to start off backing up, say, only the file system /archive/1, and /archive/2 would I use something like this:

exclude /.../*
include /archive/2
include /archive/1

And once that's done, if I then wanted to add /archive/3 and maybe even /, /home and /var then it would change to:

exclude /.../*
include /var
include /home
include /
include /archive/3
include /archive/2
include /archive/1

Something like that but always with the 'exclude /.../*' line at the top and my include lines below that since it processes the excludes first and then the includes from bottom to top?

2. What about the 'DOMAIN' statement? Do we need one of those?

3. What about 'include.fs' and or 'exclude.fs'? Do I need to use those or just 'include'? I was a bit confused about these options.

4. Also, if you're not specifying any objects in the Objects parameter for the schedule then are subdirectories always included for anything listed in the include/exclude statements without having to have '-subdir=yes'?
 
PREDATAR Control23

Include is for files. Use domain for filesystems.
dsmc help domain for more info.
 
PREDATAR Control23

Okay, thanks. I read through that, but it looks like you can't use negation for anything that begins with 'all' so, as an example, how would I tell it to ignore everything (all-local) other than the /archive/1 file system?

I cant just use:

-domain all-local
domain /archive/1

According to the documentation, that won't work.
 
PREDATAR Control23

Or maybe it's easier than I thought? By default, if nothing is listed then it backs up everything, but If I only have entries for the file sytems that I want, e.g.:

domain /archive/1
domain /archive/2

then it would only back up these two? Then just add more as necessary? Otherwise, if you wanted everything but these two then it would instead be:

-domain /archive/1
-domain /archive/2

That sound right?
 
PREDATAR Control23

You are close:

-domain /archive/1
-domain /archive/2
This is one is wrong. You need a couple of things. As you said earlier, if you omit the domain statement, that means "all-local". As soon as you specify a domain statement, that's what is used. Also, the - is in the wrong spot.

So if your domain statement(s) exclude 2 filesystems, and include nothing, you get nothing. So you would need:

Code:
domain all-local
domain -/archive/1
domain -/archive/2

An alternate method of specifying the domain:
Code:
domain all-local -/archive/1 -/archive/2

Both are valid, I prefer the 1st method, easier to read.
 
PREDATAR Control23

Thanks much. Got it, I think. So as a test, I have only the following to just back up /tmp:

domain /tmp

and after saving that, and running `dsmc help domain`, it complains thus:

ANS1036S The option 'DOMAIN' or the value supplied for it is not valid. It was found in options file '/opt/tivoli/tsm/client/ba/bin/inclexcl_dsm.local'
at line number: 1
The complete entry: ' /tmp'

Huh?
 
PREDATAR Control23

And then as soon as I change it to instead:

exclude /.../*
include /tmp

it's happy, and I can run dsmc with no errors. Not sure what's going on here?
 
PREDATAR Control23

Is /tmp a filesystem or a directory? The domain option expects a filesystem, not a directory.


exclude /.../*
include /tmp
And this will backup the /tmp directory only. If you want to backup the files in it, you will need:

Code:
exclude /.../*
include /tmp/.../*
 
PREDATAR Control23

Let me add details:

If /tmp is a filesystem. You only need this:
Code:
domain /tmp

If /tmp is a directory, you need:
Code:
domain /
exclude /.../*
include /tmp/.../*
 
PREDATAR Control23

It is a file system. In fact, I've tried a number of file systems, including /, home, /var ... you name it. Every one of them generates the same error. These are ext4 and/or xfs file systems. I even tried what you suggested, pretending /tmp is a directory, and it doesn't like the first 'domain' line. I then removed that, and it was happy.

I then noticed that I had the following line in my dsm.sys file:

DOMAIN ALL-LOCAL

I removed that line, but same problem. To be clear, I have my include/exclude statements in their own separate file, referenced from dsm.sys as:

INCLExcl /opt/tivoli/tsm/client/ba/bin/inclexcl_dsm.sys

I then added the following line to my dsm.sys file:

domain /tmp

and remarked out the INCLExcl line from the dsm.sys file, and it's happy. I then unremarked the INCLExcl line, but made sure there were no domain lines in the include/exclude file, only in dsm.sys, and it seems content.

Is it in fact the case that all domain lines must be in dsm.sys, and these are not permitted in the include/exclude file itself?

If so, then if the number of file systems that I want to back up is less than the total number on the system then I would need to enumerate each of them with its own domain line (clarity) in dsm.sys, and the include/exclude file becomes moot, unless I want to further refine what under a given file system is backed up?
 
PREDATAR Control23

Is it in fact the case that all domain lines must be in dsm.sys, and these are not permitted in the include/exclude file itself?
Oh, you are using en inclexcl file. You can only put include/excludes in there. Your domain option belong in the option file, that's why.
 
PREDATAR Control23

Okay, thanks for clarifying that.

As an example, if I wanted to have /tmp but exclude the data2 subdirectory (and all its constituent files) under /tmp then I would 1. have 'domain /tmp' in dsm.sys and 2. also have 'exclude /tmp/data2/.../*' in my include/exclude file? Whereas, if I instead left off the '/.../*' then it would only skip the data2 directory name itself but still capture its contents?

The '/.../*' seems goofy? Why don't they just use '/*' ?
 
PREDATAR Control23

As an example, if I wanted to have /tmp but exclude the data2 subdirectory (and all its constituent files) under /tmp then I would 1. have 'domain /tmp' in dsm.sys and 2. also have 'exclude /tmp/data2/.../*' in my include/exclude file?
Correct.
Whereas, if I instead left off the '/.../*' then it would only skip the data2 directory name itself but still capture its contents?
It would skip the files in data2, but not in the subdirectories of data2.

The '/.../*' seems goofy? Why don't they just use '/*' ?
It probably has to do with the history of the product. It started out on the mainframe, so it's possible the naming conventions were different then. 32 years later, it's hard to change naming conventions without affecting all the current installations.

A few additional pieces of information. Exclude by itself only excludes files. So exclude /tmp/data2/.../*Includes all files in /tmp/data2 and its subdirectories. However, that directory structure is still scanned and the directories are backed up, not the files. If however you use exclude.dir /tmp/data2, the directory /tmp/data2 will be excluded as well as all files and subdirectories. That directory structure will not be scanned and the directory tree not backed up. One caveat, exclude.dir is absolute, so you can't later include something deeper in that directory structure.
 
PREDATAR Control23

Thank you. That was very helpful. I'm still trying to wrap my head around the logic. I'm from the EMC NetWorker world where the equivalent of includes/excludes is much simpler (not saying it's more powerful, though).

So if 'exclude /tmp/data2' skips the files in data2 but not the subdirectories (and their contents), then how would you perform the converse wherein you exclude the subdirectories of data2 (and their member files) but not the files immediately under data2?
 
PREDATAR Control23

So if 'exclude /tmp/data2' skips the files in data2 but not the subdirectories (and their contents), then how would you perform the converse wherein you exclude the subdirectories of data2 (and their member files) but not the files immediately under data2?
That exclude would not work, you would need exclude /tmp/data2/* to exclude all files in /tmp/data2/, but still backup the files in the subdirectories of /tmp/data2/.

There are several examples here that will probably help you understand the various combinations:
https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.7/client/c_cfg_wildcardxmps.html
https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.7/client/c_cfg_inclexclgroups.html
https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.7/client/c_cfg_excdir.html

Also keep in mind that it's cumulative when you deal with include/exclude. The client processes the list in reverse order. So it looks at the last include/exclude and sees if the file inspected matches the pattern, if it does, it includes or excludes it depending on the statement. If it doesn't , it moves up the list until it finds a match.
 
PREDATAR Control23

I saw some of that documentation earlier, but I think I missed the second one you listed.

So it works from the bottom up. But let me see if I understand this correctly. It looks like if it encounters an include, and then later something that would normally exclude that (higher up in the list), then the include remains in effect. But if instead it hits an exclude first, and then higher up something that would include that, then the exclude is not overridden and remains in effect. That sound right?

Sheesh. I think it's going to be a late night, I'm afraid.
 
PREDATAR Control23

BEWARE the options file is read from bottom up. I use preview backup command it generates a list of what that includeexclude statements will back up and which rule commands that.
 
Top