1. Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING) Click the link to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This message will disappear after you have made at least 12 posts. Thank you for your cooperation.

vss errors after client upgrade in a clustered envirnoment

Discussion in 'TSM Installation, Upgrade and Configuration' started by tsmhelum, May 29, 2008.

  1. tsmhelum

    tsmhelum New Member

    Joined:
    Aug 24, 2006
    Messages:
    450
    Likes Received:
    0
    Hello All,



    I just upgraded a Windows client Cluster Configured backup environment from 5.3.0.0. to 5.5.0.2. The upgrade seemed to go ok but the active node Cluster node backup failed to complete successfully. I found the errors below in the Clients dsm.errorlog. I know VSS Errors have been mentioned here before but I can’t find one that pertains to a Cluster ( If A Cluster backup is even relevant at all when it pertains to VSS errors. Has anyone seen this type of VSS error before?

    What is it with the 5.5.x.x Clients and these VSS errors anyway? I don’t believe these occurred with the earlier Clients that we had. Also are the VSS errors fixed in the latest 5.5.0.6 client? I would think/hope so If not is there something that I can put in the dsm.opt file to correct the issue? Or do I have to upgrade to 5.5.0.6? which makes me cringe (due to it being a Cluster and I just struggled a little through this first one.


    DSM.Error.log

    05/28/2008 00:13:34 VssRequestor::checkWriterStatus: VssRequestor::checkWriterStatus() failed with hr=VSS_E_WRITERERROR_NONRETRYABLE
    05/28/2008 00:13:34 ANS5269E The Microsoft Volume Shadow Copy Services writer 'Cluster Service Writer' current state (VSS_WS_STABLE) is not valid for the current operation or cannot be determined. The last error reported is '800423f4'.
    05/28/2008 00:13:34 ANS5271E A Microsoft Volume Shadow Copy Services writer is in an invalid state before snapshot initialization.
    05/28/2008 00:13:34 ANS1327W The snapshot operation for 'GOGICIANK1\SystemState\NULL\System State\SystemState' failed with error code: 4353.
    05/28/2008 00:13:34 ANS1228E Sending of object 'GOGICIANK1\SystemState\NULL\System State\SystemState' failed
    05/28/2008 00:13:34 ANS5258E An error occurred initializing a VSS request. For more information, see the TSM client error log.

    05/28/2008 00:13:34 ANS1375E The snapshot operation failed.

    05/28/2008 00:13:34 ANS5258E An error occurred initializing a VSS request. For more information, see the TSM client error log.

    05/28/2008 00:13:34 ANS1512E Scheduled event 'MIDNIGHT' failed. Return code = 12.


    05/28/2008 00:13:34 ANS1512E Scheduled event 'MIDNIGHT' failed. Return code = 12.
    05/29/2008 00:09:22 ANS1959W Removing previous incomplete group '\System State\0\SystemState' Id:0-783968980
    05/29/2008 00:10:11 VssRequestor::checkWriterStatus: VssRequestor::checkWriterStatus() failed with hr=VSS_E_WRITERERROR_NONRETRYABLE
    05/29/2008 00:10:11 ANS5269E The Microsoft Volume Shadow Copy Services writer 'Cluster Service Writer' current state (VSS_WS_STABLE) is not valid for the current operation or cannot be determined. The last error reported is '800423f4'.
    05/29/2008 00:10:11 ANS5271E A Microsoft Volume Shadow Copy Services writer is in an invalid state before snapshot initialization.
    05/29/2008 00:10:11 ANS1327W The snapshot operation for 'GOGICIANK1\SystemState\NULL\System State\SystemState' failed with error code: 4353.
    05/29/2008 00:10:11 ANS1228E Sending of object 'GOGICIANK1\SystemState\NULL\System State\SystemState' failed
    05/29/2008 00:10:11 ANS5258E An error occurred initializing a VSS request. For more information, see the TSM client error log.

    05/29/2008 00:10:11 ANS1375E The snapshot operation failed.

    05/29/2008 00:10:11 ANS5258E An error occurred initializing a VSS request. For more information, see the TSM client error log.

    05/29/2008 00:10:11 ANS1512E Scheduled event 'MIDNIGHT' failed. Return code = 12.



    Dsmsched.log shows

    05/28/2008 00:12:56 Performing a full, TSM backup of object 'COM+ REGDB Writer' component 'COM+ REGDB' using shadow copy.
    05/28/2008 00:13:05 Performing a full, TSM backup of object 'Registry Writer' component 'Registry' using shadow copy.
    05/28/2008 00:13:07 Performing a full, TSM backup of object 'System Writer' component 'System Files' using shadow copy.
    05/28/2008 00:13:33 Performing a full, TSM backup of object 'Cluster Service Writer' component 'Cluster Database' using shadow copy.
    05/28/2008 00:13:33 Performing a full, TSM backup of object 'Event Log Writer' component 'Event Logs' using shadow copy.
    05/28/2008 00:13:33 Performing a full, TSM backup of object 'WMI Writer' component 'Windows Managment Instrumentation' using shadow copy.
    05/28/2008 00:13:34 ANS1228E Sending of object 'GOGICIANK1\SystemState\NULL\System State\SystemState' failed
    05/28/2008 00:13:34 ANS5258E An error occurred initializing a VSS request. For more information, see the TSM client error log.


    05/28/2008 00:13:34 ANS1375E The snapshot operation failed.

    05/28/2008 00:13:34 ANS5258E An error occurred initializing a VSS request. For more information, see the TSM client error log.

    Thanks for any help in Advance!

    TSM Configuration

    TSM Server 5.5
    AIX 5.3
    Windows 2003 SP1/TSM Client 5.5.0.2
     
    Last edited: May 29, 2008
  2.  
  3. maheshl_1982

    maheshl_1982 New Member

    Joined:
    May 3, 2007
    Messages:
    30
    Likes Received:
    0
    Please check for VSS writers status. Go to command prompt and do the list vss writers and check all writers status. If there is a writers which are in error status then that writer should be fixed. Some times by registering the some vss related dll files could solve the problem.
     
  4. sgabriel62

    sgabriel62 Senior Member

    Joined:
    Apr 7, 2005
    Messages:
    1,359
    Likes Received:
    5
    Occupation:
    Principal of SGSolutions Inc.; Systems Architect &
    Location:
    Michigan
    VSS has been a continual issue for sometime now = each of us have experienced this.
    Yes - there is a strong potential the Dll files in the registry have become corrupt or removed. Since Vss issues are Microsoft issues - have your SA re-apply SP2 from CD, then re-register the necessary Dll files associated with System state writers - also = run a Ntbackup against system state. If all works then its a TSM issue. If you execute "vssadmin list writers" from the C prompt and it returns nothing - then you have to re-registry ALL the DLLs associated with system state.
    Good Luck
     
  5. Dyermull

    Dyermull New Member

    Joined:
    May 30, 2008
    Messages:
    1
    Likes Received:
    0
    I was having the exact same errors and problems when backing up SYSTEMSTATE:

    ANS1375E The snapshot operation failed.
    ANS5258E An error occurred initializing a VSS request. For more information, see the TSM client error log.


    Sgabriel's advice to register the VSS writers fixed the issue for me (Thanks!). Here are the steps on how to properly register the VSS writers in case you were wondering.

    You may or may not need the reboot at the end (I didn't).

    According to Microsoft Premier Support this is the correct procedure to follow when re-registering the Volume Shadow Copy service DLL's.

    Open a command prompt session

    CD to Windows\System32

    Net stop vss

    Net stop swprv

    regsvr32 ole32.dll

    regsvr32 vss_ps.dll

    Vssvc /Register

    regsvr32 /i swprv.dll

    regsvr32 /i eventcls.dll

    regsvr32 es.dll

    regsvr32 stdprov.dll

    regsvr32 vssui.dll

    regsvr32 msxml.dll

    regsvr32 msxml3.dll

    regsvr32 msxml4.dll

    Reboot your server.

    After reboot, open a command prompt and run:

    vssadmin list writers

    Ensure that all your writers are displayed w/o errors.
     
  6. sgabriel62

    sgabriel62 Senior Member

    Joined:
    Apr 7, 2005
    Messages:
    1,359
    Likes Received:
    5
    Occupation:
    Principal of SGSolutions Inc.; Systems Architect &
    Location:
    Michigan
    Let me know if any of you do these registrations - I too have the same issues on several 64 bit servers and Im about to get the nerve to ask our SAs if I can run these.
    Id like to know if its simple or not and if a reboot is required
    Thanks
     
  7. tsmhelum

    tsmhelum New Member

    Joined:
    Aug 24, 2006
    Messages:
    450
    Likes Received:
    0
    Thanks maheshl_1982 and sgabriel62

    I did the “vssadmin list writers” command as you suggested (see below) it returned no errors.

    Sgabriel62,

    I have never run a Ntbackup against the system state? Would that cause a problem with anything? How would that be done?

    Also, The backups were completing fine before the upgrade to 5.5.0.2 it didn’t start failing until after the upgrade. I noticed the Passive node of the Cluster is completing successfully I’m confused on that also, Unless it’s due to the “Cluster resources drives H and the Q (Quorum) being on the Active node which shouldn’t make a difference.

    Thanks Dyermull! we will see if it comes to that fix
     
    Last edited: May 30, 2008
  8. sgabriel62

    sgabriel62 Senior Member

    Joined:
    Apr 7, 2005
    Messages:
    1,359
    Likes Received:
    5
    Occupation:
    Principal of SGSolutions Inc.; Systems Architect &
    Location:
    Michigan
    Ntbackup against systemstate is no issue - simply launch the Ntbackup utility, from the GUI or wizard - simply choose you want to select what to back up - another window should appear - from there select systemstate - and the destination of the flatfile being generated - and thats it.

    Now Ntbackup - also uses VSS services - so if it "hangs" for more than 5 minutes - then kill it. Its not going to pass but simply time out.

    Interesting note about the upgrade vs the cluster cause my issues are cluster related as well and I'll look but I'm thinking a security update or a patch is the issue - but I could be wrong and overlooking something as well
     
  9. Efim

    Efim Senior Member

    Joined:
    Jun 26, 2006
    Messages:
    556
    Likes Received:
    22
    I had had very similar problem with backup SystemState. After trace the client i found that some files from system state (it is a set of files) didn't copy during vss backup, but i could copy it in explorer ;) . After run scandisk the problem was solved (i had to restart server for correct it). I didn't receive any errors in Windows event log.

    My way was the similar: i registered dll's, found a lot of patches for VSS, reinstall SP, install Resourse Kit for VSS......
    NTbackup use a bit different API than TSM as VSS requester. In my case NTbackup backup SystemState without errors.

    Efim
    My English is still bad. Sorry...
     
  10. sgabriel62

    sgabriel62 Senior Member

    Joined:
    Apr 7, 2005
    Messages:
    1,359
    Likes Received:
    5
    Occupation:
    Principal of SGSolutions Inc.; Systems Architect &
    Location:
    Michigan
    Efim
    It would be nice to know whichfiles did not get copied over during vss backup but you could copy it from explorer - Im assuming from the TSM client extract. How does scandisk have anything to do with it - interesting this.....
     
  11. sgabriel62

    sgabriel62 Senior Member

    Joined:
    Apr 7, 2005
    Messages:
    1,359
    Likes Received:
    5
    Occupation:
    Principal of SGSolutions Inc.; Systems Architect &
    Location:
    Michigan
    Update:
    Our local fix is to reapply the current VSS patch and reboot - once the writers come back and become stable - all is well.
    Still checking on the cluster quorum idea though.
     
  12. Efim

    Efim Senior Member

    Joined:
    Jun 26, 2006
    Messages:
    556
    Likes Received:
    22
    No, explorer = microsoft file explorer.
    It was only my advice - run scandisk.
    ok. I found all logs and traces. ;)
    It was two files diskpart.exe and ntmssvc.dll
    I ressived those errors: ANS5250E, ANS1950E, ANS1228E, ANS1375E.
    I hope i answered you.
    Efim
     
  13. tsmhelum

    tsmhelum New Member

    Joined:
    Aug 24, 2006
    Messages:
    450
    Likes Received:
    0
    UPDATE:

    I did the the “vssadmin list writers” command again and there is a error. Anyone how I would correct the issue?

    Thanks!

    Writer name: 'Cluster Service Writer'
    Writer Id: {41e12264-35d8-479b-8e5c-9b23d1dad37e}
    Writer Instance Id: {6017d216-3f66-472e-a5ca-c47d20b8e67b}
    State: [1] Stable
    Last error: Non-retryable error



    C:\>vssadmin list writers
    vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
    (C) Copyright 2001 Microsoft Corp.

    Writer name: 'System Writer'
    Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
    Writer Instance Id: {12a259f2-0085-43d6-bc40-a52d4642841f}
    State: [1] Stable
    Last error: No error

    Writer name: 'MSDEWriter'
    Writer Id: {f8544ac1-0611-4fa5-b04b-f7ee00b03277}
    Writer Instance Id: {b237d0c3-62da-49ef-9ea2-8cff01b5f935}
    State: [1] Stable
    Last error: No error

    Writer name: 'Event Log Writer'
    Writer Id: {eee8c692-67ed-4250-8d86-390603070d00}
    Writer Instance Id: {264710f2-31a6-4797-9161-55104945a645}
    State: [1] Stable
    Last error: No error

    Writer name: 'COM+ REGDB Writer'
    Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
    Writer Instance Id: {c8a26a46-3d08-4dd4-82af-17cebebce127}
    State: [1] Stable
    Last error: No error

    Writer name: 'Registry Writer'
    Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
    Writer Instance Id: {fddbd722-348a-47b4-836e-1ac56418396e}
    State: [1] Stable
    Last error: No error

    Writer name: 'Cluster Service Writer'
    Writer Id: {41e12264-35d8-479b-8e5c-9b23d1dad37e}
    Writer Instance Id: {6017d216-3f66-472e-a5ca-c47d20b8e67b}
    State: [1] Stable
    Last error: Non-retryable error

    Writer name: 'WMI Writer'
    Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
    Writer Instance Id: {f9eaec31-24e6-454f-bcee-92fc2dee87ba}
    State: [1] Stable
    Last error: No error
     
  14. sgabriel62

    sgabriel62 Senior Member

    Joined:
    Apr 7, 2005
    Messages:
    1,359
    Likes Received:
    5
    Occupation:
    Principal of SGSolutions Inc.; Systems Architect &
    Location:
    Michigan
    What you could do is two things - first have the SA look at it.
    Second - remove c:\adm.sys folder and its contents - remove the associated filespaces on the TSM server - and rerun your incremental again. If the error comes back after option two - is an OS issue
     
  15. tsmhelum

    tsmhelum New Member

    Joined:
    Aug 24, 2006
    Messages:
    450
    Likes Received:
    0
    sgabriel62 Thanks,

    The SA will look at it and let me know. Also, If I were to remove the c:\adm.sys folder and its contents along with the associated filespaces would that cause an issue? Also, I have the filespaces listed below I assume the ones that would be removed are the ones I have numbered excluding number 1?

    Filespace Name FSID
    1 Asr
    2 GOGICIANK1\Systemstate\NULL\SYSTEM\STATE\SYSTEMSTATE 3 SYSTEM SERVICES
    4 SYSTEM STATE
    \\gogiciank1\c$
    \\gogiciank1\\d$
    \\gogiciank1\\i$
    \\gogiciank1\\m$
    \\gogiciank1\\n$
    \\gogiciank1\\o$
     
    Last edited: Jun 4, 2008
  16. sgabriel62

    sgabriel62 Senior Member

    Joined:
    Apr 7, 2005
    Messages:
    1,359
    Likes Received:
    5
    Occupation:
    Principal of SGSolutions Inc.; Systems Architect &
    Location:
    Michigan
    No issues. A clean slate sort to speak. And yes FSID 2 3 4 will do.
    Also I may have mentioned this already - if I have sorry - but to test at the OS level - run NTBackup against system state.
    I would expect it to time out or hang in your current state
     
  17. scott-davidson

    scott-davidson New Member

    Joined:
    Oct 2, 2003
    Messages:
    8
    Likes Received:
    0
    What worked for Me.

    This post was very helpful in more than one way. When I ran the vssadmin list command, I noticed that I only had two writers working. I knew from past experience that there should have been more working.

    My next step was work through the many steps of re-registering the dll's. After trying to stop the vss service and it's companion service, I discovered that they were not running. So, rather than re-register all the dll's I elected to simply reboot the server. That fixed the issue.

    I am not going to try and figure out what caused these services to stop working. I would like to thank everyone for their contribution to this thread. VSS is good when it works. I just wish MS fix this service.
     
: vss

Share This Page