• Please help support our sponsors by considering their products and services.
    Our sponsors enable us to serve you with this high-speed Internet connection and fast webservers you are currently using at ADSM.ORG.
    They support this free flow of information and knowledge exchange service at no cost to you.

    Please welcome our latest sponsor Tectrade . We can show our appreciation by learning more about Tectrade Solutions
  • Community Tip: Please Give Thanks to Those Sharing Their Knowledge.

    If you receive helpful answer on this forum, please show thanks to the poster by clicking "LIKE" link for the answer that you found helpful.

  • Community Tip: Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING)

    Click the link above to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This notice will disappear after you have made at least 3 posts.

Basic include/exclude for Linux?

ldmwndletsm

ADSM.ORG Member
#1
Hi, I want to exclude everything on a Linux client with the exception of one or two file systems and then begin adding more file systems one at a time (maybe several per night depending), but there are too many to run first-time fulls in a single night, and I also want to test that things are getting backed up properly by running only a few at a time, with recover testing.

I've been reading about the include/exclude, but not sure if I have this right. We will be using a server schedule (via dsmcad on the client) with the Action parameter set to INCREMENTAL, and the Object parameter will be left blank so only the dsm.sys file on the client will be used to determine what needs to be backed up. I will probably put the include/exclude statements in their own separate file and reference that in dsm.sys.

1. If I wanted to start off backing up, say, only the file system /archive/1, and /archive/2 would I use something like this:

exclude /.../*
include /archive/2
include /archive/1

And once that's done, if I then wanted to add /archive/3 and maybe even /, /home and /var then it would change to:

exclude /.../*
include /var
include /home
include /
include /archive/3
include /archive/2
include /archive/1

Something like that but always with the 'exclude /.../*' line at the top and my include lines below that since it processes the excludes first and then the includes from bottom to top?

2. What about the 'DOMAIN' statement? Do we need one of those?

3. What about 'include.fs' and or 'exclude.fs'? Do I need to use those or just 'include'? I was a bit confused about these options.

4. Also, if you're not specifying any objects in the Objects parameter for the schedule then are subdirectories always included for anything listed in the include/exclude statements without having to have '-subdir=yes'?
 

ldmwndletsm

ADSM.ORG Member
#3
Okay, thanks. I read through that, but it looks like you can't use negation for anything that begins with 'all' so, as an example, how would I tell it to ignore everything (all-local) other than the /archive/1 file system?

I cant just use:

-domain all-local
domain /archive/1

According to the documentation, that won't work.
 

ldmwndletsm

ADSM.ORG Member
#4
Or maybe it's easier than I thought? By default, if nothing is listed then it backs up everything, but If I only have entries for the file sytems that I want, e.g.:

domain /archive/1
domain /archive/2

then it would only back up these two? Then just add more as necessary? Otherwise, if you wanted everything but these two then it would instead be:

-domain /archive/1
-domain /archive/2

That sound right?
 

marclant

ADSM.ORG Moderator
#5
You are close:

-domain /archive/1
-domain /archive/2
This is one is wrong. You need a couple of things. As you said earlier, if you omit the domain statement, that means "all-local". As soon as you specify a domain statement, that's what is used. Also, the - is in the wrong spot.

So if your domain statement(s) exclude 2 filesystems, and include nothing, you get nothing. So you would need:

Code:
domain all-local
domain -/archive/1
domain -/archive/2
An alternate method of specifying the domain:
Code:
domain all-local -/archive/1 -/archive/2
Both are valid, I prefer the 1st method, easier to read.
 

ldmwndletsm

ADSM.ORG Member
#6
Thanks much. Got it, I think. So as a test, I have only the following to just back up /tmp:

domain /tmp

and after saving that, and running `dsmc help domain`, it complains thus:

ANS1036S The option 'DOMAIN' or the value supplied for it is not valid. It was found in options file '/opt/tivoli/tsm/client/ba/bin/inclexcl_dsm.local'
at line number: 1
The complete entry: ' /tmp'

Huh?
 

ldmwndletsm

ADSM.ORG Member
#7
And then as soon as I change it to instead:

exclude /.../*
include /tmp

it's happy, and I can run dsmc with no errors. Not sure what's going on here?
 

marclant

ADSM.ORG Moderator
#8
Is /tmp a filesystem or a directory? The domain option expects a filesystem, not a directory.


exclude /.../*
include /tmp
And this will backup the /tmp directory only. If you want to backup the files in it, you will need:

Code:
exclude /.../*
include /tmp/.../*
 

marclant

ADSM.ORG Moderator
#9
Let me add details:

If /tmp is a filesystem. You only need this:
Code:
domain /tmp
If /tmp is a directory, you need:
Code:
domain /
exclude /.../*
include /tmp/.../*
 
#10
It is a file system. In fact, I've tried a number of file systems, including /, home, /var ... you name it. Every one of them generates the same error. These are ext4 and/or xfs file systems. I even tried what you suggested, pretending /tmp is a directory, and it doesn't like the first 'domain' line. I then removed that, and it was happy.

I then noticed that I had the following line in my dsm.sys file:

DOMAIN ALL-LOCAL

I removed that line, but same problem. To be clear, I have my include/exclude statements in their own separate file, referenced from dsm.sys as:

INCLExcl /opt/tivoli/tsm/client/ba/bin/inclexcl_dsm.sys

I then added the following line to my dsm.sys file:

domain /tmp

and remarked out the INCLExcl line from the dsm.sys file, and it's happy. I then unremarked the INCLExcl line, but made sure there were no domain lines in the include/exclude file, only in dsm.sys, and it seems content.

Is it in fact the case that all domain lines must be in dsm.sys, and these are not permitted in the include/exclude file itself?

If so, then if the number of file systems that I want to back up is less than the total number on the system then I would need to enumerate each of them with its own domain line (clarity) in dsm.sys, and the include/exclude file becomes moot, unless I want to further refine what under a given file system is backed up?
 

marclant

ADSM.ORG Moderator
#11
Is it in fact the case that all domain lines must be in dsm.sys, and these are not permitted in the include/exclude file itself?
Oh, you are using en inclexcl file. You can only put include/excludes in there. Your domain option belong in the option file, that's why.
 
#12
Okay, thanks for clarifying that.

As an example, if I wanted to have /tmp but exclude the data2 subdirectory (and all its constituent files) under /tmp then I would 1. have 'domain /tmp' in dsm.sys and 2. also have 'exclude /tmp/data2/.../*' in my include/exclude file? Whereas, if I instead left off the '/.../*' then it would only skip the data2 directory name itself but still capture its contents?

The '/.../*' seems goofy? Why don't they just use '/*' ?
 

marclant

ADSM.ORG Moderator
#13
As an example, if I wanted to have /tmp but exclude the data2 subdirectory (and all its constituent files) under /tmp then I would 1. have 'domain /tmp' in dsm.sys and 2. also have 'exclude /tmp/data2/.../*' in my include/exclude file?
Correct.
Whereas, if I instead left off the '/.../*' then it would only skip the data2 directory name itself but still capture its contents?
It would skip the files in data2, but not in the subdirectories of data2.

The '/.../*' seems goofy? Why don't they just use '/*' ?
It probably has to do with the history of the product. It started out on the mainframe, so it's possible the naming conventions were different then. 32 years later, it's hard to change naming conventions without affecting all the current installations.

A few additional pieces of information. Exclude by itself only excludes files. So exclude /tmp/data2/.../*Includes all files in /tmp/data2 and its subdirectories. However, that directory structure is still scanned and the directories are backed up, not the files. If however you use exclude.dir /tmp/data2, the directory /tmp/data2 will be excluded as well as all files and subdirectories. That directory structure will not be scanned and the directory tree not backed up. One caveat, exclude.dir is absolute, so you can't later include something deeper in that directory structure.
 
#14
Thank you. That was very helpful. I'm still trying to wrap my head around the logic. I'm from the EMC NetWorker world where the equivalent of includes/excludes is much simpler (not saying it's more powerful, though).

So if 'exclude /tmp/data2' skips the files in data2 but not the subdirectories (and their contents), then how would you perform the converse wherein you exclude the subdirectories of data2 (and their member files) but not the files immediately under data2?
 

marclant

ADSM.ORG Moderator
#15
So if 'exclude /tmp/data2' skips the files in data2 but not the subdirectories (and their contents), then how would you perform the converse wherein you exclude the subdirectories of data2 (and their member files) but not the files immediately under data2?
That exclude would not work, you would need exclude /tmp/data2/* to exclude all files in /tmp/data2/, but still backup the files in the subdirectories of /tmp/data2/.

There are several examples here that will probably help you understand the various combinations:
https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.7/client/c_cfg_wildcardxmps.html
https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.7/client/c_cfg_inclexclgroups.html
https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.7/client/c_cfg_excdir.html

Also keep in mind that it's cumulative when you deal with include/exclude. The client processes the list in reverse order. So it looks at the last include/exclude and sees if the file inspected matches the pattern, if it does, it includes or excludes it depending on the statement. If it doesn't , it moves up the list until it finds a match.
 
#16
I saw some of that documentation earlier, but I think I missed the second one you listed.

So it works from the bottom up. But let me see if I understand this correctly. It looks like if it encounters an include, and then later something that would normally exclude that (higher up in the list), then the include remains in effect. But if instead it hits an exclude first, and then higher up something that would include that, then the exclude is not overridden and remains in effect. That sound right?

Sheesh. I think it's going to be a late night, I'm afraid.
 

dicowins

ADSM.ORG Member
#17
BEWARE the options file is read from bottom up. I use preview backup command it generates a list of what that includeexclude statements will back up and which rule commands that.
 

Advertise at ADSM.ORG

If you are reading this, so are your potential customer. Advertise at ADSM.ORG right now.

UpCloud high performance VPS at $5/month

Get started with $25 in credits on Cloud Servers. You must use link below to receive the credit. Use the promo to get upto 5 month of FREE Linux VPS.

The Spectrum Protect TLA (Three-Letter Acronym): ISP or something else?

  • Every product needs a TLA, Let's call it ISP (IBM Spectrum Protect).

    Votes: 18 19.6%
  • Keep using TSM for Spectrum Protect.

    Votes: 57 62.0%
  • Let's be formal and just say Spectrum Protect

    Votes: 10 10.9%
  • Other (please comement)

    Votes: 7 7.6%

Forum statistics

Threads
31,576
Messages
134,620
Members
21,645
Latest member
PSCAI
Top