ADSM-L

[ADSM-L] Attempt to Restore Root ('/') Filesystem from TSM Image Backup With Incremental Changes Produces Error ANS4004E

2014-09-11 10:26:21
Subject: [ADSM-L] Attempt to Restore Root ('/') Filesystem from TSM Image Backup With Incremental Changes Produces Error ANS4004E
From: "J. Adam Craig" <jacraig AT VCU DOT EDU>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Thu, 11 Sep 2014 10:23:23 -0400
Hello!

I am currently in the process of developing / testing a strategy to utilize
TSM's image backup functionality for bare metal system restores.  On my
test box, I have six EXT4 filesystems with image backups sent to TSM as
follows:

# dsmc backup image / -snapshotproviderimage=LINUX_LVM
# dsmc backup image /boot
# dsmc backup image /home -snapshotproviderimage=LINUX_LVM
# dsmc backup image /opt -snapshotproviderimage=LINUX_LVM
# dsmc backup image /tmp -snapshotproviderimage=LINUX_LVM
# dsmc backup image /var -snapshotproviderimage=LINUX_LVM


The test system is also on a regular incremental backup schedule, and so,
after submitting the image backups for all filesystems, I add / modify /
delete a few files on each filesystem and then run a successful incremental
backup as follows:

# dsmc incr


I then "hose" the box by booting into a live environment and re-formatting
each of the six filesystems afresh to EXT4.  Within this same live
environment, I have the TSM 7.1.0.3 client installed (the same version as I
used to send the image and incremental backups to TSM from the system I now
wish to restore).

With the TSM client installed and configured in the live environment, I
confirm that I can see that the image backups are available to restore:

    Image Size Stored Size FSType     Backup Date     Mgmt Class A/I Image
Name
   ---------- ----------- ------ ------------------- ---------- ---
----------
  1  16.00 GB     16.00 GB  EXT4  09/08/2014 09:08:19 DEFAULT     A  /
  2 500.00 MB    500.00 MB  EXT4  09/08/2014 09:00:07 DEFAULT     A  /boot
  3   8.00 GB      8.00 GB  EXT4  09/08/2014 09:11:40 DEFAULT     A  /home
  4   8.00 GB      8.00 GB  EXT4  09/08/2014 09:21:55 DEFAULT     A  /opt
  5   8.00 GB      8.00 GB  EXT4  09/08/2014 09:24:18 DEFAULT     A  /tmp
  6 160.00 GB    160.00 GB  EXT4  09/08/2014 09:27:01 DEFAULT     A  /var


Satisfied that all is well, I now mount the now freshly-formatted root
('/') filesystem to the '/rescue' directory in my live environment, and
attempt to restore it from the image.  Since I have incremental backups
that include various additions, changes, and deletions to the filesystem,
I've elected to restore the filesystem as follows:

[root@livecd ~]# dsmc restore image / /rescue -incremental -deletefiles
IBM Tivoli Storage Manager
Command Line Backup-Archive Client Interface
  Client Version 7, Release 1, Level 0.3
  Client date/time: 09/10/2014 18:53:18
(c) Copyright by IBM Corporation and other(s) 1990, 2014. All Rights
Reserved.

Node Name: *******.***.***.***
Session established with server ****: Linux/x86_64
  Server Version 6, Release 3, Level 4.200
  Server date/time: 09/10/2014 14:53:46  Last access: 09/10/2014 14:53:37


Restore Image Function Invoked.

ANS8048W Warning!  Performing image restore of the Linux file system '/' to
an alternate destination '/rescue' is not recommended as this may result in
duplicate UUIDs leading to failed mounts after a successful restore.

Continue (Yes (Y)/No (N)) y
***************************** WARNING ********************************
Restoring a file system or raw logical volume will replace any data that
currently resides there and all file system parameters. Are you sure you
want to replace
    File System/Volume: '/rescue'?  (Yes (Y)/No (N)) y
Restoring  17,179,869,184  [Done]

Restore processing finished.
Restoring           4,096 / --> /rescue/ [Done]

Total number of objects restored:             2
Total number of objects failed:               0
Total number of bytes transferred:        16.00 GB
Data transfer time:                      195.00 sec
Network data transfer rate:           86,036.95 KB/sec
Aggregate data transfer rate:         79,144.66 KB/sec
Elapsed processing time:               00:03:31
ANS4004E Error processing '/': destination file or directory is write locked


As can be seen, the image restore completes successfully, but when TSM
attempts to reconcile the subsequent changes reflected by the later
incremental backup, error ANS4004E is issued.  I have tested to confirm
whether or not the mounted '/rescue' directory is writeable, and it is.

Is it possible that the TSM client application is exercising some sort of
protection that prevents the restore feature from recovering a root ('/')
filesystem from backup?  If so, that certainly would be understandable, but
is there an override for scenarios, such as the one above, when that really
is what I want to do?  What am I missing?

Also, for the record, it is worth mentioning that if I don't pass the
'-incremental -deletefiles' options, the restore completes successfully and
I can then mount the other filesystems within the '/rescue' directory and
recover them from their respective image backups.  Upon exiting the live
environment and attempting to boot the system, I am greeted by a successful
boot to the system in the state it was in when the image backups were made,
and from what was a completely hosed box, which is precisely what I'm
after.  However, I'd love to be able to include changes up to the latest
incremental backup as part of my bare metal restore operation.

Any assistance is greatly appreciated.

Thanks!
-- Adam
______________________
*J. Adam Craig*
UNIX & Windows Operating Systems Engineer
VCU Computer Center
804.828.4886

"Don't be a phishing victim -- VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information.  For more details,
visit http://infosecurity.vcu.edu/phishing.html";