1. Community Tip: Please Give Thanks to Those Sharing Their Knowledge.
    If you receive helpful answer on this forum, please show thanks to the poster by clicking "LIKE" link for the answer that you found helpful.
  2. Community Tip: Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING)
    Click the link above to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This notice will disappear after you have made at least 3 posts.

Incremental Backup ignores modified files

Discussion in 'Backup / Archive Discussion' started by jeinhorn, Jul 17, 2017.

  1. jeinhorn

    jeinhorn ADSM.ORG Member

    Joined:
    Apr 4, 2014
    Messages:
    32
    Likes Received:
    0
    Greetings, TSM Gurus.

    Platform of our TSM Server is TSM 6.3.4.0 on Redhat 6.9
    Platform of our TSM client is TSM 6.4.1.0 on Solaris 10

    Here is the dsm.sys file from the client:
    Code:
    SErvername  tsmserv.corp.kns.com
       COMMMethod           TCPip
       TCPPort              1500
       TCPServeraddress     10.8.52.21
       TCPBuffsize          32
       TCPWindowsize        64
       TCPNodelay           Yes
       TCPCLIENTADDRESS     10.24.50.15
       NODENAME             portaldev
       HTTPPort             1581
       LargeCommBuffers     Yes
       ManagedServices      Webclient Schedule
       PASSWORDAccess       generate
       ResourceUtilization  10
       SCHEDMode            Prompted
       TXNBytelimit         25600
       ERRORLOGNAME         /tmp/dsmerror.log
       SCHEDLOGNAME         /tmp/dsmsched.log
       SCHEDLOGRETENTION    10
       ERRORLOGRETENTION    10
       inclexcl             /opt/tivoli/tsm/client/ba/bin/inclexcl.list
    
    Here is the dsm.opt file from the client:
    Code:
    servername  tsmserv.corp.kns.com
    
    domain  /export/home
    subdir  yes
    archsymlinkasfile  no
    
    /export/home is a separate filesystem with about 77 files and folders:

    Code:
    # df -h /export/home
    Filesystem  size  used  avail capacity  Mounted on
    rpool/export/home  5.0G  21M  5.0G  1%  /export/home
    
    # find /export/home | wc -l
      77
    
    Let's focus on these two files on the /export/home filesystem:

    Code:
    # df /export/home/jeinhorn/testfile
    /export/home  (rpool/export/home ):10442811 blocks 10442811 files
    
    # df /export/home/jeinhorn/testfile2
    /export/home  (rpool/export/home ):10442811 blocks 10442811 files
    
    TSM has one backup copy of these two files:
    Code:
    # dsmc query backup -inactive "/export/home/jeinhorn/testfile*"
    . . . <snip> . . .
      Size  Backup Date  Mgmt Class  A/I File
      ----  -----------  ----------  --- ----
      12  B  07/17/17  10:30:37  DEFAULT  A  /export/home/jeinhorn/testfile
      25  B  07/17/17  11:22:42  DEFAULT  A  /export/home/jeinhorn/testfile2
    
    Now I modify the two files. The "date" command shows that the modified date changed, as expected:
    Code:
    # date
    Mon Jul 17 11:52:14 EDT 2017
    # vi /export/home/jeinhorn/testfile /export/home/jeinhorn/testfile2
    . . . save and exit . . .
    # ls -l /export/home/jeinhorn/testfile /export/home/jeinhorn/testfile2
    -rw-r--r--  1 jeinhorn sysadmin  134 Jul 17 11:52 /export/home/jeinhorn/testfile
    -rw-r--r--  1 root  root  100 Jul 17 11:52 /export/home/jeinhorn/testfile2
    
    Incremental backup does not pick up the files!
    Code:
    # dsmc incremental "/export/home/*" -subdir=yes
    . . . <snip> . . .
    Incremental backup of volume '/export/home/*'
    Successful incremental backup of '/export/home/*'
    
    Total number of objects inspected:  76
    Total number of objects backed up:  0
    Total number of objects updated:  0
    Total number of objects rebound:  0
    Total number of objects deleted:  0
    Total number of objects expired:  0
    Total number of objects failed:  0
    Total number of bytes inspected:  20.31 MB
    Total number of bytes transferred:  0  B
    Data transfer time:  0.00 sec
    Network data transfer rate:  0.00 KB/sec
    Aggregate data transfer rate:  0.00 KB/sec
    Objects compressed by:  0%
    Total data reduction ratio:  100.00%
    Elapsed processing time:  00:00:01
    
    If I create a new file in the same directory, TSM incremental backup picks up the new file
    Code:
    # vi /export/home/jeinhorn/testfile3
    . . . save and exit . . .
    
    # dsmc incremental "/export/home/*" -subdir=yes
    . . . <snip> . . .
    Incremental backup of volume '/export/home/*'
    Normal File-->  16 /export/home/jeinhorn/testfile3 [Sent]
    Successful incremental backup of '/export/home/*'
    
    Total number of objects inspected:  77
    Total number of objects backed up:  1
    Total number of objects updated:  0
    Total number of objects rebound:  0
    Total number of objects deleted:  0
    Total number of objects expired:  0
    Total number of objects failed:  0
    Total number of bytes inspected:  20.31 MB
    Total number of bytes transferred:  227  B
    Data transfer time:  0.00 sec
    Network data transfer rate:  73,893.22 KB/sec
    Aggregate data transfer rate:  0.20 KB/sec
    Objects compressed by:  0%
    Total data reduction ratio:  100.00%
    Elapsed processing time:  00:00:01
    
    Can anyone provide a clue about why TSM incremental backup is not picking up modified files?

    Many Thanks,
    Janet
     
  2.  
  3. marclant

    marclant ADSM.ORG Moderator

    Joined:
    Jun 16, 2006
    Messages:
    2,486
    Likes Received:
    348
    Occupation:
    Accelerated Value Specialist for Spectrum Protect
    Location:
    Canada
    What`s the content of:

    inclexcl /opt/tivoli/tsm/client/ba/bin/inclexcl.list
     
  4. jeinhorn

    jeinhorn ADSM.ORG Member

    Joined:
    Apr 4, 2014
    Messages:
    32
    Likes Received:
    0
    Here it is:
    Code:
    # cat /opt/tivoli/tsm/client/ba/bin/inclexcl.list
    ******* INCLUDE / EXCLUDE  SECTION ***********
    exclude         /proc/*
    exclude         /cdrom/*
    exclude.fs      /tmp
    exclude         /var/tmp/*
    exclude         /var/run/*
    exclude         /devices/*
    exclude         /system/contract/*
    exclude         /etc/mnttab/*
    exclude         /etc/svc/volatile/*
    exclude         /system/object/*
    exclude         /dev/fd/*
    
     
  5. jeinhorn

    jeinhorn ADSM.ORG Member

    Joined:
    Apr 4, 2014
    Messages:
    32
    Likes Received:
    0
    I updated the client to 7.1.6.5, which I believe to be the latest client available for Solaris. It did not change the incremental backup issue.
     
  6. marclant

    marclant ADSM.ORG Moderator

    Joined:
    Jun 16, 2006
    Messages:
    2,486
    Likes Received:
    348
    Occupation:
    Accelerated Value Specialist for Spectrum Protect
    Location:
    Canada
    Try:
    Code:
    dsmc incremental "/export/home/*" -subdir=yes -auditlogging=full
    Then look at dsmaudit.log in the client directory. It will tell you if the file is changed or not, and if it's excluded or not.
     
  7. marclant

    marclant ADSM.ORG Moderator

    Joined:
    Jun 16, 2006
    Messages:
    2,486
    Likes Received:
    348
    Occupation:
    Accelerated Value Specialist for Spectrum Protect
    Location:
    Canada
    You may also have excludes from the server, check:
    Code:
    dsmc query inclexcl
     
  8. jeinhorn

    jeinhorn ADSM.ORG Member

    Joined:
    Apr 4, 2014
    Messages:
    32
    Likes Received:
    0
    Marclant, Thanks for your suggestions.

    I tried using the "-auditlogging=full" option before and after updating a file. The audit logfile consistently shows "Unchanged". (I tried it twice)

    Code:
    # vi /export/home/jeinhorn/testfile3
    . . . save and exit
    # dsmc incremental "/export/home/*" -subdir=yes -auditlogging=full
    . . . <snip> . . .
    Total number of objects inspected:  77
    Total number of objects backed up:  0
    . . . <snip>
    # vi /export/home/jeinhorn/testfile3
    . . . save and exit
    # dsmc incremental "/export/home/*" -subdir=yes -auditlogging=full
    . . . <snip> . . .
    Total number of objects inspected:  77
    Total number of objects backed up:  0
    . . . <snip>
    # grep testfile3 dsmaudit.log
    07/17/17  15:54:33 ANS1661I Unchanged: /export/home/jeinhorn/testfile3
    07/17/17  15:56:00 ANS1661I Unchanged: /export/home/jeinhorn/testfile3
    
    I don't see any related exclusions, and if there were any, then I think "dsmc incremental" would not have picked up /export/home/jeinhorn/testfile3 when I first created it.
    Code:
    # dsmc query inclexcl
    IBM Tivoli Storage Manager
    Command Line Backup-Archive Client Interface
      Client Version 7, Release 1, Level 6.5
      Client date/time: 07/17/17  16:01:01
    (c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights Reserved.
    
    Node Name: PORTALDEV
    Session established with server TSMSERV: Linux/x86_64
      Server Version 6, Release 3, Level 4.0
      Server date/time: 07/17/17  16:01:01  Last access: 07/17/17  15:56:00
    
    *** FILE INCLUDE/EXCLUDE ***
    Mode Function  Pattern (match from top down)  Source File
    ---- --------- ------------------------------ -----------------
    Excl Filespace /tmp  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
    Excl Directory /.../.TsmCacheDir  TSM
    Exclude All  /dev/fd/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
    Exclude All  /system/object/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
    Exclude All  /etc/svc/volatile/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
    Exclude All  /etc/mnttab/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
    Exclude All  /system/contract/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
    Exclude All  /devices/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
    Exclude All  /var/run/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
    Exclude All  /var/tmp/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
    Exclude All  /cdrom/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
    Exclude All  /proc/*  "/opt/tivoli/tsm/client/ba/bin/inclexcl.list"
    No DFS include/exclude statements defined.
    
    I also checked the date/time on both server and client. They match, and they use the same timezone
    Code:
    [root@portaldev /]# date
    Mon Jul 17 16:02:24 EDT 2017
    
    Code:
    [jeinhorn@tsmserv/]# date
    Mon Jul 17 16:02:31 EDT 2017
    
    Regards,
    Janet
     
  9. Patthecat

    Patthecat Newcomer

    Joined:
    Jul 11, 2017
    Messages:
    3
    Likes Received:
    2
    Hi,

    maybe this behaviour is normal. There is a hint in the server admin reference guide under "define copygroup ", that MODE is only used in full incremental backups, not in partial, which means, that "last date modified", "file size", "file owner", "file permission" will be ignored in partial incremental backups like "dsmc incremental "/export/home/*" -subdir=yes".

    MODE
    Specifies whether Tivoli Storage Manager backs up a file only if the file has
    changed since the last backup, or whenever a client requests a backup. This
    parameter is optional. The default value is MODIFIED. Possible values are:

    MODified
    Specifies that Tivoli Storage Manager backs up the file only if it has
    changed since the last backup. Tivoli Storage Manager considers a file
    changed if any of the following is true:
    • The date last modified is different
    • The file size is different
    • The file owner is different
    • The file permissions are different
    ABSolute
    Specifies that Tivoli Storage Manager backs up the file regardless of
    whether it has been modified.

    The MODE value is used only for full incremental backup. This value is
    ignored during partial incremental backup or selective backup


    Please check, if the following works.
    • dsmc i
     
    jeinhorn likes this.
  10. jeinhorn

    jeinhorn ADSM.ORG Member

    Joined:
    Apr 4, 2014
    Messages:
    32
    Likes Received:
    0
    Hi Patthecat - You are correct! Well done. Command "dsmc incremental" picked up the modified files
    Code:
    # dsmc incremental
    . . . <snip> . . .
    Incremental backup of volume '/export/home'
    Normal File-->               134 /export/home/jeinhorn/testfile [Sent]
    Normal File-->               100 /export/home/jeinhorn/testfile2 [Sent]
    Normal File-->                34 /export/home/jeinhorn/testfile3 [Sent]
    Normal File-->           156,155 /export/home/orcladm/.vnc/portaldev:8.log [Sent]
    Normal File-->         4,146,045 /export/home/smarimut/phy_mem_status [Sent]
    Successful incremental backup of '/export/home'
    . . . <snip>
    
    This behavior seems illogical to me. Suppose I have a filesystem mounted on /myfilesys and the only files that I care to backup are located under /myfilesys/importantfolder. What would be the method for doing incremental backup of files and subfolders under /myfilesys/importantfolder ?

    I thank you very much for solving the mystery.

    Regards,
    Janet
     
  11. marclant

    marclant ADSM.ORG Moderator

    Joined:
    Jun 16, 2006
    Messages:
    2,486
    Likes Received:
    348
    Occupation:
    Accelerated Value Specialist for Spectrum Protect
    Location:
    Canada
    You would exclude everything, then include just that:
    Code:
    exclude /myfilesys/.../*
    include /myfilesys/importantfolder/.../*
     
    jeinhorn likes this.
  12. jeinhorn

    jeinhorn ADSM.ORG Member

    Joined:
    Apr 4, 2014
    Messages:
    32
    Likes Received:
    0
    Thanks again to you both, Marclant and Patthecat.
    Regards,
    Janet
     

Share This Page