tsm problem with recovery log

K

komzik

ADSM.ORG Member
Joined
Nov 12, 2009
Messages
17
Reaction score
0
Points
0
Hi.

I have TSM server 5.3.5.2 on Windows 2003. Suddenly tsm service was stoped. When I start tsm service it stops after a few seconds. I checked Event Viewer but I found nothing. Only in conslole.log I found this:

ANR4726I The ICC support module has been loaded.
ANR0990I Server restart-recovery in progress.
ANR0200I Recovery log assigned capacity is 10000 megabytes.
ANR0201I Database assigned capacity is 25000 megabytes.
ANR0306I Recovery log volume mount in progress.
ANR0285I Database page shadowing started using file dbpgshdw.bdt.
ANR0353I Recovery log analysis pass in progress.
ANR0354I Recovery log redo pass in progress.
ANR0355I Recovery log undo pass in progress.
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D pkthread.c(853): ThreadId <0> Run-time assertion "CmpLsn( *pageLsnP,
hdrP->updtLsn ) != LESSTHAN" at dblog.c(1130) failed.
ANR9999D ThreadId <0> issued message 9999 from:
ANR7837S Internal error DBLOG666 detected.

ANR9999D ThreadId <0> issued message 7837 from:


I think I have problem with recovery log.
I found in Internet:

http://www.adsm.org/lists/html/ADSM-L/2008-02/msg00251.html

"A) Restore the DB (Point in Time)
B) Do a DUMPDB, FORMATLOAD, LOADDB, AUDITDB

A) is painful regarding data-loss, B) is painful regarding time-loss.
We've decided to go for B), the first 3 steps used 1/2 day, let's see
how AUDITDB performs. "

but I don`t have access to CLI (tsm service is stoped, I can`t run it).
Any ideas ?
 
Find where the dsmserv.exe file is. Go to that directory from within a DOS prompt. You can now use the command "dsmserv" to start the server in the fore-ground or perform such things as the AuditDB.

-Aaron
 
I started command "dsmserv auditdb"
This is output:

ANR0900I Processing options file c:\program files\tivoli\tsm\Server1\dsmserv.-
opt.
ANR7800I DSMSERV generated at 01:40:46 on Jul 13 2007.

Tivoli Storage Manager for Windows
Version 5, Release 3, Level 5.2

Licensed Materials - Property of IBM

(C) Copyright IBM Corporation 1990, 2006.
All rights reserved.
U.S. Government Users Restricted Rights - Use, duplication or disclosure
restricted by GSA ADP Schedule Contract with IBM Corporation.

ANR4726I The ICC support module has been loaded.
ANR0990I Server restart-recovery in progress.
ANR0200I Recovery log assigned capacity is 10000 megabytes.
ANR0201I Database assigned capacity is 25000 megabytes.
ANR0306I Recovery log volume mount in progress.
ANR0285I Database page shadowing started using file dbpgshdw.bdt.
ANR0353I Recovery log analysis pass in progress.
ANR0354I Recovery log redo pass in progress.
ANR0355I Recovery log undo pass in progress.
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D ThreadId <0> 1059F07D outDiagf()+16d
ANR9999D ThreadId <0> 1020CC38 TbUndoExternal()+e8
ANR9999D ThreadId <0> 101FA821 dbInRecovery()+d21
ANR9999D ThreadId <0> 101FAAFB DbRestartRecovery()+3b
ANR9999D ThreadId <0> 101D43A1 dbInit()+471
ANR9999D ThreadId <0> 10017B80 admStartServer()+330
ANR9999D ThreadId <0> 10019B8B admAuditServer()+6b
ANR9999D ThreadId <0> 1000E7F3 pkReleaseMutex()+43
ANR9999D ThreadId <0> 10004FEA adsmMain()+85a
ANR9999D ThreadId <0> 1000105F enterAdsmMain()+f
ANR9999D ThreadId <0> 004010AF main()+5f
ANR9999D ThreadId <0> 0040267F mainCRTStartup()+143
ANR9999D ThreadId <0> 7C82F23B ProcessIdToSessionId()+209
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D ThreadId <0> 1059F07D outDiagf()+16d
ANR9999D ThreadId <0> 1020CC38 TbUndoExternal()+e8
ANR9999D ThreadId <0> 101FA821 dbInRecovery()+d21
ANR9999D ThreadId <0> 101FAAFB DbRestartRecovery()+3b
ANR9999D ThreadId <0> 101D43A1 dbInit()+471
ANR9999D ThreadId <0> 10017B80 admStartServer()+330
ANR9999D ThreadId <0> 10019B8B admAuditServer()+6b
ANR9999D ThreadId <0> 1000E7F3 pkReleaseMutex()+43
ANR9999D ThreadId <0> 10004FEA adsmMain()+85a
ANR9999D ThreadId <0> 1000105F enterAdsmMain()+f
ANR9999D ThreadId <0> 004010AF main()+5f
ANR9999D ThreadId <0> 0040267F mainCRTStartup()+143
ANR9999D ThreadId <0> 7C82F23B ProcessIdToSessionId()+209
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D ThreadId <0> 1059F07D outDiagf()+16d
ANR9999D ThreadId <0> 1020CC38 TbUndoExternal()+e8
ANR9999D ThreadId <0> 101FA821 dbInRecovery()+d21
ANR9999D ThreadId <0> 101FAAFB DbRestartRecovery()+3b
ANR9999D ThreadId <0> 101D43A1 dbInit()+471
ANR9999D ThreadId <0> 10017B80 admStartServer()+330
ANR9999D ThreadId <0> 10019B8B admAuditServer()+6b
ANR9999D ThreadId <0> 1000E7F3 pkReleaseMutex()+43
ANR9999D ThreadId <0> 10004FEA adsmMain()+85a
ANR9999D ThreadId <0> 1000105F enterAdsmMain()+f
ANR9999D ThreadId <0> 004010AF main()+5f
ANR9999D ThreadId <0> 0040267F mainCRTStartup()+143
ANR9999D ThreadId <0> 7C82F23B ProcessIdToSessionId()+209
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D ThreadId <0> 1059F07D outDiagf()+16d
ANR9999D ThreadId <0> 1020CC38 TbUndoExternal()+e8
ANR9999D ThreadId <0> 101FA821 dbInRecovery()+d21
ANR9999D ThreadId <0> 101FAAFB DbRestartRecovery()+3b
ANR9999D ThreadId <0> 101D43A1 dbInit()+471
ANR9999D ThreadId <0> 10017B80 admStartServer()+330
ANR9999D ThreadId <0> 10019B8B admAuditServer()+6b
ANR9999D ThreadId <0> 1000E7F3 pkReleaseMutex()+43
ANR9999D ThreadId <0> 10004FEA adsmMain()+85a
ANR9999D ThreadId <0> 1000105F enterAdsmMain()+f
ANR9999D ThreadId <0> 004010AF main()+5f
ANR9999D ThreadId <0> 0040267F mainCRTStartup()+143
ANR9999D ThreadId <0> 7C82F23B ProcessIdToSessionId()+209
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D ThreadId <0> 1059F07D outDiagf()+16d
ANR9999D ThreadId <0> 1020CC38 TbUndoExternal()+e8
ANR9999D ThreadId <0> 101FA821 dbInRecovery()+d21
ANR9999D ThreadId <0> 101FAAFB DbRestartRecovery()+3b
ANR9999D ThreadId <0> 101D43A1 dbInit()+471
ANR9999D ThreadId <0> 10017B80 admStartServer()+330
ANR9999D ThreadId <0> 10019B8B admAuditServer()+6b
ANR9999D ThreadId <0> 1000E7F3 pkReleaseMutex()+43
ANR9999D ThreadId <0> 10004FEA adsmMain()+85a
ANR9999D ThreadId <0> 1000105F enterAdsmMain()+f
ANR9999D ThreadId <0> 004010AF main()+5f
ANR9999D ThreadId <0> 0040267F mainCRTStartup()+143
ANR9999D ThreadId <0> 7C82F23B ProcessIdToSessionId()+209
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D ThreadId <0> 1059F07D outDiagf()+16d
ANR9999D ThreadId <0> 1020CC38 TbUndoExternal()+e8
ANR9999D ThreadId <0> 101FA821 dbInRecovery()+d21
ANR9999D ThreadId <0> 101FAAFB DbRestartRecovery()+3b
ANR9999D ThreadId <0> 101D43A1 dbInit()+471
ANR9999D ThreadId <0> 10017B80 admStartServer()+330
ANR9999D ThreadId <0> 10019B8B admAuditServer()+6b
ANR9999D ThreadId <0> 1000E7F3 pkReleaseMutex()+43
ANR9999D ThreadId <0> 10004FEA adsmMain()+85a
ANR9999D ThreadId <0> 1000105F enterAdsmMain()+f
ANR9999D ThreadId <0> 004010AF main()+5f
ANR9999D ThreadId <0> 0040267F mainCRTStartup()+143
ANR9999D ThreadId <0> 7C82F23B ProcessIdToSessionId()+209
ANR9999D tbundo.c(210): ThreadId <0> Error 2 on delete from table DS.Overflow
for undo.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D ThreadId <0> 1059F07D outDiagf()+16d
ANR9999D ThreadId <0> 1020CC38 TbUndoExternal()+e8
ANR9999D ThreadId <0> 101FA821 dbInRecovery()+d21
ANR9999D ThreadId <0> 101FAAFB DbRestartRecovery()+3b
ANR9999D ThreadId <0> 101D43A1 dbInit()+471
ANR9999D ThreadId <0> 10017B80 admStartServer()+330
ANR9999D ThreadId <0> 10019B8B admAuditServer()+6b
ANR9999D ThreadId <0> 1000E7F3 pkReleaseMutex()+43
ANR9999D ThreadId <0> 10004FEA adsmMain()+85a
ANR9999D ThreadId <0> 1000105F enterAdsmMain()+f
ANR9999D ThreadId <0> 004010AF main()+5f
ANR9999D ThreadId <0> 0040267F mainCRTStartup()+143
ANR9999D ThreadId <0> 7C82F23B ProcessIdToSessionId()+209
ANR9999D pkthread.c(853): ThreadId <0> Run-time assertion "CmpLsn( *pageLsnP,
hdrP->updtLsn ) != LESSTHAN" at dblog.c(1130) failed.
ANR9999D ThreadId <0> issued message 9999 from:
ANR9999D ThreadId <0> 1059F07D outDiagf()+16d
ANR9999D ThreadId <0> 10007403 pkLogicAbort()+23
ANR9999D ThreadId <0> 101C7D62 DbWriteLogRecordV()+2f2
ANR9999D ThreadId <0> 101C7EDD dbLogWriteV()+ad
ANR9999D ThreadId <0> 101C7F3B dbLogWrite()+1b
ANR9999D ThreadId <0> 1020BCB6 bvUndoRange()+f6
ANR9999D ThreadId <0> 101FA914 dbInRecovery()+e14
ANR9999D ThreadId <0> 101FAAFB DbRestartRecovery()+3b
ANR9999D ThreadId <0> 101D43A1 dbInit()+471
ANR9999D ThreadId <0> 10017B80 admStartServer()+330
ANR9999D ThreadId <0> 10019B8B admAuditServer()+6b
ANR9999D ThreadId <0> 1000E7F3 pkReleaseMutex()+43
ANR9999D ThreadId <0> 10004FEA adsmMain()+85a
ANR9999D ThreadId <0> 1000105F enterAdsmMain()+f
ANR9999D ThreadId <0> 004010AF main()+5f
ANR9999D ThreadId <0> 0040267F mainCRTStartup()+143
ANR9999D ThreadId <0> 7C82F23B ProcessIdToSessionId()+209
ANR7837S Internal error DBLOG666 detected.

ANR9999D ThreadId <0> issued message 7837 from:
ANR9999D ThreadId <0> 1059EB90 outMsgf()+20
ANR9999D ThreadId <0> 10007419 pkLogicAbort()+39
ANR9999D ThreadId <0> 101C7D62 DbWriteLogRecordV()+2f2
ANR9999D ThreadId <0> 101C7EDD dbLogWriteV()+ad
ANR9999D ThreadId <0> 101C7F3B dbLogWrite()+1b
ANR9999D ThreadId <0> 1020BCB6 bvUndoRange()+f6
ANR9999D ThreadId <0> 101FA914 dbInRecovery()+e14
ANR9999D ThreadId <0> 101FAAFB DbRestartRecovery()+3b
ANR9999D ThreadId <0> 101D43A1 dbInit()+471
ANR9999D ThreadId <0> 10017B80 admStartServer()+330
ANR9999D ThreadId <0> 10019B8B admAuditServer()+6b
ANR9999D ThreadId <0> 1000E7F3 pkReleaseMutex()+43
ANR9999D ThreadId <0> 10004FEA adsmMain()+85a
ANR9999D ThreadId <0> 1000105F enterAdsmMain()+f
ANR9999D ThreadId <0> 004010AF main()+5f
ANR9999D ThreadId <0> 0040267F mainCRTStartup()+143
ANR9999D ThreadId <0> 7C82F23B ProcessIdToSessionId()+209
Entering exception handler.
Leaving exception handler.
 
That does not look good. I would open a support call and give IBM the output. It looks like the recovery log is corrupted and not repairable. I think your only option is to restore the DB.

-Aaron
 
1) DSMSERV DUMPDB
2) DSMSERV LOADFORMAT
3) DSMSERV LOADDB
4) DSMSERV AUDITDB

Is this correct order ?
Or ony DSMSERV RESTORE DB ? - I have last full DB backup and volhist.out file.
 
That is the correct order to attempt to bypass the corrupted log. If after doing that, you're still getting the errors then you'll have to restore the database and then you'll use the restore command.

-Aaron
 
I exec. command:

dsmserv loadformat 1 f:\tsmdata\server1\log_vol.dsm 1 h:\tsmdata\server1\db_vol.dsm 9216

but I got error:

Invalid database volume count for format mode.

???

[+] I forgot about log size.
 
Last edited:
Another problem.

I make dsmserv loaddb command

This is output:

ANR0900I Processing options file C:\Program Files\Tivoli\TSM\server\dsmserv.o-
pt.
ANR7800I DSMSERV generated at 01:40:46 on Jul 13 2007.

Tivoli Storage Manager for Windows
Version 5, Release 3, Level 5.2

Licensed Materials - Property of IBM

(C) Copyright IBM Corporation 1990, 2006.
All rights reserved.
U.S. Government Users Restricted Rights - Use, duplication or disclosure
restricted by GSA ADP Schedule Contract with IBM Corporation.

ANR4726I The ICC support module has been loaded.
ANR0990I Server restart-recovery in progress.
ANR0200I Recovery log assigned capacity is 9216 megabytes.
ANR0201I Database assigned capacity is 9216 megabytes.
ANR0306I Recovery log volume mount in progress.
ANR0353I Recovery log analysis pass in progress.
ANR0354I Recovery log redo pass in progress.
ANR0355I Recovery log undo pass in progress.
ANR0352I Transaction recovery complete.
ANR4003I LOADDB: Database load process started.
ANR1437E No device configuration files could be used.
ANR4005E LOADDB: Database load process terminated due to error (-10).
ANR8200I TCP/IP driver ready for connection with clients on port 1500.

Still I can`t restore db :(

[+] in folder C:\Program Files\Tivoli\TSM\server I don`t have devcfg file. devcfg file is in C:\Program Files\Tivoli\TSM\server1. Maybe is it a problem ? Should I copy this file from C:\Program Files\Tivoli\TSM\server1 to C:\Program Files\Tivoli\TSM\server ?
 
Last edited:
You can edit your dsmserv.opt file to have an entry that locates the devconfig file. I would not copy it.

-Aaron
 
I`m afraid that my new DB file is too small. May I delete old DB file and recovery log ?
I have fresh database dump.
 
Last edited:
As you have a database dump to load from AND the last TSM DB Backup, you should be OK with deleting the old database files to make room to restore them. Normally I do not use new files but rather over-write the existing ones.

-Aaron
 
I started tsm service. Thanks for help :)
But I have one more question.

This is output Q DB

db.png


Should I reduce DB size ?
 
No.
You can if you want, but it is just information for you, if there is some circumstances that you need to claim some space on storage where yor database volumes resides, it is not a recommendation.
 
You must log in or register to reply here.

Data Privacy Impact Assessment

Compliance Assurance in Every Byte: Discover the Impact of Our DPIA Services.

Sponsor ADSM.ORG

If you are reading this, so are your potential customer. Advertise at ADSM.ORG right now.

Navigation Menu

Forums
   IBM TSM
   EMC Backup and Recovery
   Symantec NetBackup  
   SAN  
   NAS
   Platforms / OS
Mailing List Archives
   ADSM-L
User Groups

NordVPN 3 Months FREE

Help Support ADSM.ORG and Get NordVPN with 64% Off and 3 Months FREE.

NordVPN with 64% off + 3 months FREE

Forum statistics

Threads
31,982
Messages
135,471
Members
21,953
Latest member
smeegama
Back
Top