Restore failed

[gvieira]

Hi all!



I was trying to restore a directory that had not been changed for a long time. It was backed up during a previous release of TSM. Now i'm using 5.3.



During restore I get



ANR0548W Retrieve or restore failed for session 25434 for node SERVER_X (WinNT) processing file space \\server_x\d$ 2 for file \USERDATA\IT\USER\SETUPS\ DOTNETFX.EXE stored as Backup - data integrity error detected. (SESSION: 25434)



I have this for several files. Logs show that TSM tries then to access copypool to retrive the files but, when provide the offsite tapes I still cannot restore my files.



I have no errors on my tapes. What can I do to make sure this does not happen since there many files I can restore without any problems?





Thanks for all comments on this



Gabriel Vieira

 

[crazymad]

Hi, have you tried AUDIT VOLUME? I'd try to audit it (first with fix=no, than depending on situation with fix=yes) and try the resotre again.



Hope it helps.

 

[gvieira]

Hi



Thanks for your interrest. I checked in the tape from the copy pool, updated to reado... and started the audit. I could see some more files



ANR2317W Audit Volume found damaged file on volume 000567: Node SERVER_X, Type Backup (Active), File space \\server_x\

d$, fsId 2, File name \USERDATA\IT\DIR\ FILE.XLS is number 1 of 1 versions.



and suddenly the server crashes down!



On event viewer I only get

<UL>

Event Type: Error

Event Source: ADSMServer

Event Category: None

Event ID: 27

Date: 24-01-2006

Time: 13:46:01

User: N/A

Computer: SV001199

Description:

TSM Server Diagnostic: ANR9999D: ADSM Exception Information: file = pkthread.c, line = 2256,Code = c0000005, Address = 105124CB

Attempt to read data at address 4~

[/list]



This behaviour happned again whe started the server on console mode.



Any ideas??



Thanks



Gabriel

 

[may]

It s look like that you TSM DB is corupted...

you can make an "Audit DB" but this can take some long time to complete...

 

[gvieira]

«audit db» ?? I think my TSM does not know this audit. Could be a litle more specific on your suggestion?



Thanks



Gabriel Vieira

 

[crazymad]

It does



you must audit your db from the command line:



./dsmserv auditdb fix=yes



or maybe fix=no firstly - to see what's gonna be rapired.



Hope it helps

 

[gvieira]

Yes it does. :grin:



I did the audit with fix=yes but it didn't solve my restore problem.

<UL>

ANR1165E Error detected for file in storage pool COPYPOOL: Node SERVER_X, Type

Backup, File space \server_xd$, fsId 2, File name

USERDATAIT... MSOINTL.DLL.

ANR0836W No query restore processing session 1 for node SV001188 and

\sv001188d$ failed to retrieve file USERDATA...MSOINTL.DLL - file being skipped.

[/list]



Running the audit on this volume still crashes the server. What else? :cry: ~Is the data lost?

What can be done to prevent such things from happening in the future? These audits should occur during reclamations!!



Thanks.

 

[heada]

Why is TSM attempting to restore from the copypool volume? what happened to the primary pool volume?



Have you tried the copypool volume in a different (cleaner?) drive?



Has the drive changed since the backup was made (new microcode, etc)?



Just a few things that rattled around in my head.



-Aaron

 

[gvieira]

Thanks,

Nothing changed regarding drives or microcodes.

TSM asks for the copy pool because the primary pool fails to restore.



By the way I'm running 5.3.0.0 and noticed that we have now 5.3.2.2. Does any one recommend the upgrade?

 

[heada]

Why does the primary fail to restore? Is it available? in an error state?



-Aaron

 

[gvieira]

Hi again



This all started because I couldn't restore one file. During restore says file was damaged and asked for copypool. I checked in the volume from copypool and TSM again tells me the file is damaged.

I start an audit on the volume and find out that more files on this are damaged. During this operation TSM server crashes.

I then follow May's advice and audit my database and find errors on it. I start the audit with FIX=yes but this doesn't work.

Now I did the UNLOADDB, LOADFORMAT and LOADDB. Running a new audit on my database returns NO problems.

Still running the audit fix=yes on the volume shows damaged files and marks them to delete:



<UL>

ANR2308W Audit Volume marking damaged file as damaged on volume CAFS15: Node SV001188, Type Backup (Inactive), File

space \sv001188d$, fsId 2, File Name USERDATA... FILE.PPT is number 1 of 1

versions.

ANR4132I Audit volume process ended for volume CAFS15; 46 files inspected, 0 damaged files deleted, 20 damaged files

marked as damaged, 0 objects updated.

ANR0987I Process 3 for AUDIT VOLUME (REPAIR) running in the BACKGROUND processed 46 items with a completion state of

SUCCESS at 09:10:27.

[/list]



You may see that there are files damaged but TSM does not actually delete them. Also doing a new incremental backup does not correct the situation.



Moreover reclamation fails on this tape

<UL>

ANR1162W Space reclamation skipping damaged file on volume CAFS15: Node SV001188, Type Backup, File space \sv001188d-

$, File name ..... file.PPT.

ANR1041I Space reclamation ended for volume CAFS15.

ANR4932I Reclamation process 10 ended for storage pool WINPOOL.

ANR0985I Process 10 for SPACE RECLAMATION running in the BACKGROUND completed with completion state FAILURE at 11:48:23.

ANR4936I Reclamation of storage pool WINPOOL has ended. Files reclaimed: 0, Bytes reclaimed: 0, Files reconstructed: 0, Unreadable files: 0.



[/list]



How can I recover from this?



Thanks on all comments.

 

[crazymad]

Hi,



try to list damaged files for the primary and for the copy pool by "Q CONTENT" and check whether are the same.



Files are marked as damaged and not deleted because there is a copy pool. Maybe if you mark the particular copy pool volume as "destroyed", the files will be deleted from the primary pool. But I'm not sure of that...



I'd try to "MOVE DATA" from the primary stgpool tape and check whether there remain only the damaged files (by "q cont"). If yes, you can delete the volume with the damaged files.



Hope it helps.

 

[gvieira]

Hi



Thank you. TSM is finally going on track... and I did loose some files

I'll try to improve my audits on data and run them more frequantly. What is your experiences? How often do you audit backups? Right after the backups?



Thanks again.