Incremental Backup ignores modified files

J

jeinhorn

ADSM.ORG Member
Joined
Apr 4, 2014
Messages
34
Reaction score
1
Points
0
Greetings, TSM Gurus.

Platform of our TSM Server is TSM 6.3.4.0 on Redhat 6.9
Platform of our TSM client is TSM 6.4.1.0 on Solaris 10

Here is the dsm.sys file from the client:
Code: 
SErvername  tsmserv.corp.kns.com
   COMMMethod           TCPip
   TCPPort              1500
   TCPServeraddress     10.8.52.21
   TCPBuffsize          32
   TCPWindowsize        64
   TCPNodelay           Yes
   TCPCLIENTADDRESS     10.24.50.15
   NODENAME             portaldev
   HTTPPort             1581
   LargeCommBuffers     Yes
   ManagedServices      Webclient Schedule
   PASSWORDAccess       generate
   ResourceUtilization  10
   SCHEDMode            Prompted
   TXNBytelimit         25600
   ERRORLOGNAME         /tmp/dsmerror.log
   SCHEDLOGNAME         /tmp/dsmsched.log
   SCHEDLOGRETENTION    10
   ERRORLOGRETENTION    10
   inclexcl             /opt/tivoli/tsm/client/ba/bin/inclexcl.list

Here is the dsm.opt file from the client:
Code: 
servername  tsmserv.corp.kns.com

domain  /export/home
subdir  yes
archsymlinkasfile  no

/export/home is a separate filesystem with about 77 files and folders:

Code: 
# df -h /export/home
Filesystem  size  used  avail capacity  Mounted on
rpool/export/home  5.0G  21M  5.0G  1%  /export/home

# find /export/home | wc -l
  77

Let's focus on these two files on the /export/home filesystem:

Code: 
# df /export/home/jeinhorn/testfile
/export/home  (rpool/export/home ):10442811 blocks 10442811 files

# df /export/home/jeinhorn/testfile2
/export/home  (rpool/export/home ):10442811 blocks 10442811 files

TSM has one backup copy of these two files:
Code: 
# dsmc query backup -inactive "/export/home/jeinhorn/testfile*"
. . . <snip> . . .
  Size  Backup Date  Mgmt Class  A/I File
  ----  -----------  ----------  --- ----
  12  B  07/17/17  10:30:37  DEFAULT  A  /export/home/jeinhorn/testfile
  25  B  07/17/17  11:22:42  DEFAULT  A  /export/home/jeinhorn/testfile2

Now I modify the two files. The "date" command shows that the modified date changed, as expected:
Code: 
# date
Mon Jul 17 11:52:14 EDT 2017
# vi /export/home/jeinhorn/testfile /export/home/jeinhorn/testfile2
. . . save and exit . . .
# ls -l /export/home/jeinhorn/testfile /export/home/jeinhorn/testfile2
-rw-r--r--  1 jeinhorn sysadmin  134 Jul 17 11:52 /export/home/jeinhorn/testfile
-rw-r--r--  1 root  root  100 Jul 17 11:52 /export/home/jeinhorn/testfile2

Incremental backup does not pick up the files!
Code: 
# dsmc incremental "/export/home/*" -subdir=yes
. . . <snip> . . .
Incremental backup of volume '/export/home/*'
Successful incremental backup of '/export/home/*'

Total number of objects inspected:  76
Total number of objects backed up:  0
Total number of objects updated:  0
Total number of objects rebound:  0
Total number of objects deleted:  0
Total number of objects expired:  0
Total number of objects failed:  0
Total number of bytes inspected:  20.31 MB
Total number of bytes transferred:  0  B
Data transfer time:  0.00 sec
Network data transfer rate:  0.00 KB/sec
Aggregate data transfer rate:  0.00 KB/sec
Objects compressed by:  0%
Total data reduction ratio:  100.00%
Elapsed processing time:  00:00:01

If I create a new file in the same directory, TSM incremental backup picks up the new file
Code: 
# vi /export/home/jeinhorn/testfile3
. . . save and exit . . .

# dsmc incremental "/export/home/*" -subdir=yes
. . . <snip> . . .
Incremental backup of volume '/export/home/*'
Normal File-->  16 /export/home/jeinhorn/testfile3 [Sent]
Successful incremental backup of '/export/home/*'

Total number of objects inspected:  77
Total number of objects backed up:  1
Total number of objects updated:  0
Total number of objects rebound:  0
Total number of objects deleted:  0
Total number of objects expired:  0
Total number of objects failed:  0
Total number of bytes inspected:  20.31 MB
Total number of bytes transferred:  227  B
Data transfer time:  0.00 sec
Network data transfer rate:  73,893.22 KB/sec
Aggregate data transfer rate:  0.20 KB/sec
Objects compressed by:  0%
Total data reduction ratio:  100.00%
Elapsed processing time:  00:00:01

Can anyone provide a clue about why TSM incremental backup is not picking up modified files?

Many Thanks,
Janet
 
What`s the content of:

inclexcl /opt/tivoli/tsm/client/ba/bin/inclexcl.list
 
Here it is:
Code: 
# cat /opt/tivoli/tsm/client/ba/bin/inclexcl.list
******* INCLUDE / EXCLUDE  SECTION ***********
exclude         /proc/*
exclude         /cdrom/*
exclude.fs      /tmp
exclude         /var/tmp/*
exclude         /var/run/*
exclude         /devices/*
exclude         /system/contract/*
exclude         /etc/mnttab/*
exclude         /etc/svc/volatile/*
exclude         /system/object/*
exclude         /dev/fd/*
 
I updated the client to 7.1.6.5, which I believe to be the latest client available for Solaris. It did not change the incremental backup issue.
 
Try:
Code: 
dsmc incremental "/export/home/*" -subdir=yes -auditlogging=full
Then look at dsmaudit.log in the client directory. It will tell you if the file is changed or not, and if it's excluded or not.
 
You may also have excludes from the server, check:
Code: 
dsmc query inclexcl
 
Marclant, Thanks for your suggestions.

I tried using the "-auditlogging=full" option before and after updating a file. The audit logfile consistently shows "Unchanged". (I tried it twice)

Code: 
# vi /export/home/jeinhorn/testfile3
. . . save and exit
# dsmc incremental "/export/home/*" -subdir=yes -auditlogging=full
. . . <snip> . . .
Total number of objects inspected:  77
Total number of objects backed up:  0
. . . <snip>
# vi /export/home/jeinhorn/testfile3
. . . save and exit
# dsmc incremental "/export/home/*" -subdir=yes -auditlogging=full
. . . <snip> . . .
Total number of objects inspected:  77
Total number of objects backed up:  0
. . . <snip>
# grep testfile3 dsmaudit.log
07/17/17  15:54:33 ANS1661I Unchanged: /export/home/jeinhorn/testfile3
07/17/17  15:56:00 ANS1661I Unchanged: /export/home/jeinhorn/testfile3

I don't see any related exclusions, and if there were any, then I think "dsmc incremental" would not have picked up /export/home/jeinhorn/testfile3 when I first created it.
Code: 
# dsmc query inclexcl
IBM Tivoli Storage Manager
Command Line Backup-Archive Client Interface
  Client Version 7, Release 1, Level 6.5
  Client date/time: 07/17/17  16:01:01
(c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights Reserved.

Node Name: PORTALDEV
Session established with server TSMSERV: Linux/x86_64
  Server Version 6, Release 3, Level 4.0
  Server date/time: 07/17/17  16:01:01  Last access: 07/17/17  15:56:00

*** FILE INCLUDE/EXCLUDE ***
Mode Function  Pattern (match from top down)  Source File
---- --------- ------------------------------ -----------------
Excl Filespace /tmp  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
Excl Directory /.../.TsmCacheDir  TSM
Exclude All  /dev/fd/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
Exclude All  /system/object/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
Exclude All  /etc/svc/volatile/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
Exclude All  /etc/mnttab/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
Exclude All  /system/contract/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
Exclude All  /devices/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
Exclude All  /var/run/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
Exclude All  /var/tmp/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
Exclude All  /cdrom/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
Exclude All  /proc/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
No DFS include/exclude statements defined.

I also checked the date/time on both server and client. They match, and they use the same timezone
Code: 
[root@portaldev /]# date
Mon Jul 17 16:02:24 EDT 2017

Code: 
[jeinhorn@tsmserv/]# date
Mon Jul 17 16:02:31 EDT 2017

Regards,
Janet
 
Hi,

maybe this behaviour is normal. There is a hint in the server admin reference guide under "define copygroup ", that MODE is only used in full incremental backups, not in partial, which means, that "last date modified", "file size", "file owner", "file permission" will be ignored in partial incremental backups like "dsmc incremental "/export/home/*" -subdir=yes".

MODE
Specifies whether Tivoli Storage Manager backs up a file only if the file has
changed since the last backup, or whenever a client requests a backup. This
parameter is optional. The default value is MODIFIED. Possible values are:

MODified
Specifies that Tivoli Storage Manager backs up the file only if it has
changed since the last backup. Tivoli Storage Manager considers a file
changed if any of the following is true:
  • The date last modified is different
  • The file size is different
  • The file owner is different
  • The file permissions are different
ABSolute
Specifies that Tivoli Storage Manager backs up the file regardless of
whether it has been modified.

The MODE value is used only for full incremental backup. This value is
ignored during partial incremental backup or selective backup

Please check, if the following works.
  • dsmc i
 
Hi Patthecat - You are correct! Well done. Command "dsmc incremental" picked up the modified files
Code: 
# dsmc incremental
. . . <snip> . . .
Incremental backup of volume '/export/home'
Normal File-->               134 /export/home/jeinhorn/testfile [Sent]
Normal File-->               100 /export/home/jeinhorn/testfile2 [Sent]
Normal File-->                34 /export/home/jeinhorn/testfile3 [Sent]
Normal File-->           156,155 /export/home/orcladm/.vnc/portaldev:8.log [Sent]
Normal File-->         4,146,045 /export/home/smarimut/phy_mem_status [Sent]
Successful incremental backup of '/export/home'
. . . <snip>

This behavior seems illogical to me. Suppose I have a filesystem mounted on /myfilesys and the only files that I care to backup are located under /myfilesys/importantfolder. What would be the method for doing incremental backup of files and subfolders under /myfilesys/importantfolder ?

I thank you very much for solving the mystery.

Regards,
Janet
 
You would exclude everything, then include just that:
Code: 
exclude /myfilesys/.../*
include /myfilesys/importantfolder/.../*
 
Thanks again to you both, Marclant and Patthecat.
Regards,
Janet
 
For anyone who comes across this thread, I have discovered another possible reason why "dsmc incremental" might not pick up modified files. The default value for copygroup "copy frequency" is 0, which means no limit or restriction. I discovered that a TSM Administrator who worked at my company years ago had set this limit to 1.
From spectrum protect documentation:
FREQuency
Specifies how frequently IBM Spectrum Protect can back up a file. This parameter is optional. IBM Spectrum Protect backs up a file only when the specified number of days has elapsed since the last backup. The FREQUENCY value is used only during a full incremental backup operation. This value is ignored during selective backup or partial incremental backup. You can specify an integer from 0 to 9999. The default value is 0, meaning that IBM Spectrum Protect can back up a file regardless of when the file was last backed up.
Click to expand...

From our environment:
Code: 
Protect: TSMSERV> query policyset linux active
PolDom Name   Pol Set Name  Def Mgmt Class    Description
LINUX         ACTIVE        MC_LINUX_WG       Policy set for Linux

Protect: TSMSERV> query copygroup LINUX PS_LINUX_WG mc_linux_wg type=backup format=detailed
                 Policy Domain Name: LINUX
                    Policy Set Name: PS_LINUX_WG
                    Mgmt Class Name: MC_LINUX_WG
                    Copy Group Name: STANDARD
                    Copy Group Type: Backup
               Versions Data Exists: 4
              Versions Data Deleted: 1
              Retain Extra Versions: 90
                Retain Only Version: 60
                          Copy Mode: Modified
                 Copy Serialization: Dynamic
                     Copy Frequency: 1            <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
                   Copy Destination: DP-MISC
Table of Contents (TOC) Destination:
     Last Update by (administrator): ADMIN
              Last Update Date/Time: 03/31/2010 00:28:23
                   Managing profile:
                    Changes Pending: No

When I encountered this problem recently, "dsmc selective" enabled me to force a backup of modified files despite the Copy Frequency setting shown above.

Regards,
Janet
 
You must log in or register to reply here.

Data Privacy Impact Assessment

Compliance Assurance in Every Byte: Discover the Impact of Our DPIA Services.

Sponsor ADSM.ORG

If you are reading this, so are your potential customer. Advertise at ADSM.ORG right now.

Navigation Menu

Forums
   IBM TSM
   EMC Backup and Recovery
   Symantec NetBackup  
   SAN  
   NAS
   Platforms / OS
Mailing List Archives
   ADSM-L
User Groups

NordVPN 3 Months FREE

Help Support ADSM.ORG and Get NordVPN with 64% Off and 3 Months FREE.

NordVPN with 64% off + 3 months FREE

Forum statistics

Threads
31,969
Messages
135,437
Members
21,949
Latest member
tsheli
Back
Top