TSM fails to start after mksysb restore of TSM server - db2 error

Cyclone

ADSM.ORG Member
Joined
Dec 1, 2009
Messages
55
Reaction score
1
Points
0
Location
Maryland
PREDATAR Control23

Hi everyone.

I am running TSM v7.1.1.100 on a AIX 6.1 TL9 server. For years, I have backed up my TSM server with a mksysb. Everything except storage pools, tsmdb, logs, and vmctl are in rootvg. So this one mksysb tape and the tsm db backup tapes have been all that I have needed to restore my TSM server.

This worked with TSM v6.2 & v6.3.4 which I ran for several years. I have a DR server and I'd restore from the mksysb, and then restore the TSM db from tape and I'd have my DR tsm server up and running in a few hours.

I upgraded to v7.1.1.100 a few months ago and I found a problem when I restore my mksysb to the DR server. The restore goes ok, but when I try to start up TSM, it fails. It looks like it can't start up the database.

$ id
uid=1003(rpsinst1) gid=1001(tsmsrvrs) groups=1(staff)

$ /opt/tivoli/tsm/server/bin/dsmserv -u rpsinst1 -i /home/rpsinst1
ANR7800I DSMSERV generated at 07:38:46 on Nov 23 2014.
Tivoli Storage Manager for AIX
Version 7, Release 1, Level 1.100
Licensed Materials - Property of IBM
(C) Copyright IBM Corporation 1990, 2014.
All rights reserved.
U.S. Government Users Restricted Rights - Use, duplication or disclosure
restricted by GSA ADP Schedule Contract with IBM Corporation.
ANR7801I Subsystem process ID is 6094872.
ANR0900I Processing options file /home/rpsinst1/dsmserv.opt.
ANR7811I Using instance directory /home/rpsinst1.
ANR4726I The ICC support module has been loaded.
ANR0990I Server restart-recovery in progress.
ANR0237E Fail to start the database manager. An unexpected system error occurred.
ANR0171I dbiconn.c(1844): Error detected on 0:1, database in evaluation mode.
ANR0162W Supplemental database diagnostic information: -1:08001:-1032 ([IBM][CLI Driver] SQL1032N No start
database manager command was issued. SQLSTATE=57019
).
ANR0237E Fail to start the database manager. An unexpected system error occurred.
$


I opened up a PMR with IBM and we found that db2 would not start. Whenever we issued a db2 command, it would respond with "Illegal Instruction".

$ db2dart TSMDB1 /DB
Illegal instruction

$ db2start
Illegal instruction

$ db2stop
Illegal instruction


IBM recommended that I uninstall and reinstall the TSM software on the server. I did this and sure enough, after that, the TSM server would start and I could restore my database. Has anyone else seen this and did you find out what is causing this problem with restoring from the mksysb? Again, I used to make mksysb tapes with TSM v6.x up and running and it would not be a problem, I could restore just fine.

In trying to trouble shoot this problem, I have made multiple mksysb attempts with the TSM instances shut down and even with the db2fmcd and any other db2 process stopped. But each time, the same results, Illegal Instruction.

I have pushed IBM tech support a little bit more on this. They came back and said that db2 does not support mksysb restores and that it was the db2 team who suggested that db2 (ie TSM) be re-installed. TSM v7.1.x has been out for a year now and I must not be the only one who has tried to restore their TSM server via mksysb. Just looking for some feed back from those who are successfully restoring from mksysb backups, or those who have had to resort to another method if they had this problem.

Thanks,

Don
 
PREDATAR Control23

iocp0 can be on defined after mksysb restore

lsattr -El iocp0

if Defined status, execute: mkdev -l iocp0
 
PREDATAR Control23

Have you dropped whole TSM+DB2 after restoring mksysb and before restoring TSM server?
 
PREDATAR Control23

iocp0 can be on defined after mksysb restore

lsattr -El iocp0

if Defined status, execute: mkdev -l iocp0

I was hoping that this was the problem as I had this as defined. I did use smitty and enabled iocp0, and rebooted. But still the problem persisted. But thank you for your reply.

$ lsattr -El iocp0
autoconfig available STATE to be configured at system restart True
$
$ lsdev -Cc iocp
iocp0 Available I/O Completion Ports

$ db2dart TSMDB1 /DB
Illegal instruction
$
 
PREDATAR Control23

Have you dropped whole TSM+DB2 after restoring mksysb and before restoring TSM server?
No, after restoring the mksysb, I'm usually (v6.3.x) able to just start TSM (library instance with its db on rootvg filesystesms) or restore the db on the primary instance without dropping db2/tsmdb. The storage for the primary instances is virgin diskspace on the dr server.
 
PREDATAR Control23

Try running db2iupdt command, from root db2iupdt <instanceid>

It works for my ...
 
PREDATAR Control23

I came across the same issue this week. Since I didn't readily have the TSM software handy, I resolved it by dropping the database and then uncataloging the database.

db2 drop db tsmdb1

Then I came across this issue when I tried to recreate the instance: http://www-01.ibm.com/support/docview.wss?uid=swg1IT02135

Had to uncatalog the db.

$ db2 list database directory

System Database Directory

Number of entries in the directory = 1

Database 1 entry:

Database alias = TSMAL001
Database name = TSMDB1
Node name = TSMNODE1
Database release level = 10.00
Comment = TSM SERVER DATABASE
VIA TCPIP
Directory entry type = Remote
Catalog database partition number = -1
Alternate server hostname =
Alternate server port number =

There should be only one entry reported and the database alias
name should be "TSMAL001," otherwise do not continue with
these steps.

After confirming that there is a single "TSMAL001" database
alias remaining, issue the "db2 uncatalog database TSMAL001"
command to remove the remaining DB alias. For example:

$ db2 uncatalog database tsmal001
DB20000I The UNCATALOG DATABASE command completed successfully.
DB21056W Directory changes may not be effective until the
directory cache is refreshed.


After this, I was able to recreate the TSM instance, and then issue DSMSERV REMOVEDB TSMDB1. I was then able to restore the TSM server.
 
  • Like
Reactions: rgg
PREDATAR Control23

For the sake of posterity, the problem that I had was that I was restoring my Power7 server mksysb to a Power5 server. No this was not a problem when TSM was running versions 5.5.x and 6.3.x, but once I had upgraded to version 7.1.x, the problem appeared. I opened a PMR with IBM. The results were that if I restore it on a Power5 server, that I would have to re-install the TSM binaries over the restore software. Then it would start up. The real solution is for me to restore another power7 server.
 
Top