exclude.fs will not exclude files from incremental

[Myaso]

Hi

I have some difficulty understanding how "exclude.fs" option works.
TSM server is 5.4.4, tsm client is 5.4.1 on AIX 5.3.
Simple example from real life:

> lsfs /nexus
Name Nodename Mount Pt VFS Size Options Auto
/dev/fslv04 -- /nexus jfs2 4194304 rw yes

dsm.sys has line
exclude.fs /nexus

However, the following command in fact backs up all the files from that filesystem:
> dsmc i /nexus/
IBM Tivoli Storage Manager
Command Line Backup/Archive Client Interface
........

Incremental backup of volume '/nexus/'
Successful incremental backup of '/nexus/*'


Now my question is - what does "exclude.fs" statement do? (it definitely does NOT prevent the file-system to be incrementally backed up).


I will post any other details if necessary.


thank's in advance!
Myaso

 

[BBB]

Touch a file in /nexus and run the incremental again - does it say it backed up > 0 files?

Run "q files <nodename>" in the client or admin client and look at how much data is held for the nexus filesystem.

You may find it is actually excluding it. The successful message is just saying the backup had no errors and worked. Everything being excluded is not really an error.

 

[Myaso]

OK, BBB. Thank's for your answer.

Let me say what I have and what I do in detail.

1. I have a freshly created node on TSM server - there is NO data stored for it at all.

2. ON a client side, I have defined the following statement
exclude.fs /nexus
in dsm.sys

3. On a client side, I start the following commands:
> df -g /nexus
Filesystem GB blocks Free %Used Iused %Iused Mounted on
/dev/fslv04 2.00 1.64 18% 8653 3% /nexus
> find /nexus -type f | wc -l
6075

So, we see - /nexus IS an actual filesystem and IS NOT empty (it has some files in it)

4. Start the backup:
> dsmc i -verbose /nexus/ > /tmp/out1 2>&1 &

5. The process ends within few minutes. Then:
> more /tmp/out1
IBM Tivoli Storage Manager
Command Line Backup/Archive Client Interface
.....

Incremental backup of volume '/nexus/'
Directory--> 256 /nexus/lost+found [Sent]
Directory--> 256 /nexus/nexus-webapp-1.1.0 [Sent]
Directory--> 256 /nexus/pawels.install [Sent]
Directory--> 256 /nexus/sonatype-work [Sent]
Directory--> 256 /nexus/tmp [Sent]
Normal File--> 254 /nexus/.profile [Sent]
Normal File--> 1,042 /nexus/.sh_history [Sent]
.......
Successful incremental backup of '/nexus/*'


Total number of objects inspected: 8,645
Total number of objects backed up: 8,637
Total number of objects updated: 0
Total number of objects rebound: 0
Total number of objects deleted: 0
Total number of objects expired: 0
Total number of objects failed: 0
Total number of bytes transferred: 340.86 MB
Data transfer time: 2.48 sec
Network data transfer rate: 140,695.85 KB/sec
Aggregate data transfer rate: 6,059.85 KB/sec
Objects compressed by: 0%
Elapsed processing time: 00:00:57

6. Confirm on a TSM server side (tried to make that font smaller to fit into one line):
tsm: TSM>q occ <node>
Node Name Type Filespace FSID Storage Number of Physical Logical
Name Pool Name Files Space Space
Occupied Occupied
(MB) (MB)
---------- ---- ---------- ----- ---------- --------- --------- ---------
<node> Bkup /nexus 13 <stgpool> 6,055 343.11 343.11

7. > dsmc q inclexcl
IBM Tivoli Storage Manager
Command Line Backup/Archive Client Interface
Client Version 5, Release 4, Level 1.0
Client date/time: 01/28/09 15:33:02
(c) Copyright by IBM Corporation and other(s) 1990, 2007. All Rights Reserved.

Node Name: <node>
Session established with server TSM: AIX-RS/6000
Server Version 5, Release 4, Level 4.0
Server date/time: 01/28/09 15:33:02 Last access: 01/28/09 15:32:56

*** FILE INCLUDE/EXCLUDE ***
Mode Function Pattern (match from top down) Source File
---- --------- ------------------------------ -----------------
Excl Filespace /nexus /usr/tivoli/tsm/client/ba/bin/dsm.
sys
Excl Directory /ora??/oradata Server
Excl Directory /ora_tmp Server
Excl Directory /unix Server
Excl Directory /var/spool Server
Excl Directory /.../.TsmCacheDir TSM
Exclude All /.../tmp/.../* Server
Exclude All /.../core Server
No DFS include/exclude statements defined.

> dsmc q fil
IBM Tivoli Storage Manager
Command Line Backup/Archive Client Interface
Client Version 5, Release 4, Level 1.0
Client date/time: 01/28/09 15:38:30
(c) Copyright by IBM Corporation and other(s) 1990, 2007. All Rights Reserved.

Node Name: <node>
Session established with server TSM: AIX-RS/6000
Server Version 5, Release 4, Level 4.0
Server date/time: 01/28/09 15:38:30 Last access: 01/28/09 15:38:19

# Last Incr Date Type File Space Name
--- -------------- ---- ---------------
1 00/00/00 00:00:00 JFS2 /nexus



8. ??? Didn't exclude those files from the backup at all.

Thank you for any suggestions!!


Myaso

 

[BBB]

Ok it makes sense now.

Exclude.fs will stop the filespace from being backed up if you do "dsmc i" which is what should normally be doing. It will not stop it from being backed up if you go "dsmc i /nexus/", because you've just told it to do it, so it will ignore the exclude.fs. So basically, stop running dsmc i /nexus/ !

Alternatively, remove the filespace from the domain using the domain option. eg.
Domain -/nexus will remove it from the domain and it should then be totally excluded from backups.

 

[Myaso]

I have a feeling I understand this better Thank you.

I ask myself another question now: what is the reason they have an option "exclude.fs" when we can use one of the "domain -<name>" or "exclude.dir <name>" ?

Feels like I can override "exclude.fs NAME/" with dsmc I or SEL and the specific file name (which is locaed somewhere under NAME/) and have it atually backed up, while I can NOT override "exclude.fs NAME/" with the same dsmc commands.

I just ran a simple example.
Changed "exclude.fs" to
exclude.dir /nexus
in dsm.sys
Then, ran the commands:
> dsmc i -sub=yes -verbose /nexus/NEWDIR/
...
Incremental backup of volume '/nexus/NEWDIR/'
ANS1115W File '/nexus/NEWDIR/*' excluded by Include/Exclude list
Successful incremental backup of '/nexus/NEWDIR/*'



> dsmc sel -sub=yes -verbose /nexus/NEWDIR/
....
Selective Backup function invoked.

ANS1115W File '/nexus/NEWDIR/*' excluded by Include/Exclude list
Selective Backup processing of '/nexus/NEWDIR/*' finished without failure.


Looks like these examples confirm my supposition. Correct me if I'm wrong please.

Thank you again.

 

[PJ]

The main difference lies in how the client expires objects. exclude.dir leads to the client expiring the dir's content from the server while exclude.fs does not (apart from the filespace/filesystem vs. directory aspect)

From the client documentation:

exclude.fs Excludes file spaces matching the pattern. The client does not consider the specified file space for processing and the usual deleted-file expiration process cannot occur. If you exclude a file space that was previously included, existing backup versions remain on the server subject to retention rules specified in the associated management class definition. See Exclude options for more information.

exclude.dir Excludes a directory, its files, and all its subdirectories and their files from backup processing. For example, the statement exclude.dir /test/dan/data1 excludes the /test/dan/data1 directory, its files, and all its subdirectories and their files. Using the exclude.dir option is preferable over the standard exclude option to exclude large directories containing many files that you do not want to back up. You cannot use include options to override an exclude.dir statement. Only use exclude.dir when excluding an entire directory branch.

PJ

 

[Bartdo]

The way I would approach it is this:
Do you want to exclude a dir or a fs ?
DIR : Then use exclude.dir
FS: Then use Domain or exclude.fs
The advantage of exclude.fs is that it can match patterns.
The advantage of Domain is that you can specify ALL-LOCAL

Bart

 

[Asha]

Hi..
I am facing the same problem in one of my Solaris servers:

Dsm.sys file has the exclude statement as:

exclude.backup /var/audit/tsm
exclude.backup /tmp
exclude.backup /devices
exclude.backup /proc
exclude.fs /dev/vx/dmp
exclude.fs /dev/vx/rdmp
exclude.fs /sybswap/tmpfs1
exclude.fs /sybswap/tmpfs2

But during incremental backup it says:

07/03/09 13:57:06 Incremental backup of volume '/dev/vx/dmp'
07/03/09 13:57:06 Incremental backup of volume '/dev/vx/rdmp'

07/03/09 14:05:19 ANS1228E Sending of object '/dev/vx/dmp' failed
07/03/09 14:05:19 ANS1063E The specified path is not a valid file system or logical volume name.
07/03/09 14:05:19 ANS1228E Sending of object '/dev/vx/rdmp' failed
07/03/09 14:05:19 ANS1063E The specified path is not a valid file system or logical volume name.

Anything that can be added to get rid of this problem because of this we are getting a Failed 12 reason code for this server backup...
Thanks in advance...

 

[BBB]

Isn't /dev/vx/dmp etc a file, rather than a fs? Exclude it as a file rather than a fs.

 

[pburwell]

exclude.fs /nexus should be exclude.dir /nexus​