Page 1 of 2 12 LastLast
Results 1 to 24 of 30
  1. #1
    Newcomer
    Join Date
    Aug 2010
    Posts
    23
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Windows 2003 backup, missing system files [ntldr, boot.ini, etc.]

    Hi All,

    I've got problem backing up a Windows 2003 server. After running some tests it appeared that there are system files missing in node backup such as ntldr, boot.ini, ntdetect.com, ntoskrnl.exe and so on.

    Backup tasks are finishing successfully, files are not excluded, VSS support is enabled, dsmsched.log contains information about successfull incremental backup of SystemState (and other files), but in auditlog (full) there is no information about those files - except for ones in \\hostname\c$\WINDOWS\$NtServicePackUninstall$\ or \\hostname\c$\WINDOWS\ServicePackFiles\i386\


    TBMR restores end with unbootable system. (restore to another machine)

    -----------------------

    additionally i ran these command on dsmc clients with following results:

    query backup “{hostname\SystemState\NULL\System State\SystemState}\boot.ini”
    ANS1092W No files matching search criteria were found
    query backup “{hostname\SystemState\NULL\System State\SystemState\ntdetect.com”
    ANS1092W No files matching search criteria were found
    query backup “{hostname\SystemState\NULL\System State\SystemState}\ntldr”
    ANS1092W No files matching search criteria were found

    -----------------------

    Client OS is Windows2003 SE SP2, TSM B/A Client 6.1.3.1, TSM server 6.1.3.4
    -----------------------

    Does anyone have ever had similar problem?
    Any help would be appreciated.

  2. #2
    Senior Member
    Join Date
    Nov 2002
    Posts
    514
    Thanks
    2
    Thanked 4 Times in 4 Posts

    Default

    I would open a PMR with IBM regarding this. Also contact Cristie to see what they know about this problem.
    We did some testing recently with TSM version 6.2.1.1 on both Windows2003 and Windows2008 without any problem.
    I know your problem have been observed before, I do not know anything about what release or anything.

    Is this a test or is it a critical situation?

  3. #3
    Newcomer
    Join Date
    Feb 2009
    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Hi,

    It sounds like the process which enumerates VSS writers and files is not passing the correct file list to TSM. There are a couple of routes you can take to investigate this.

    From a windows command prompt run diskshadow /l c:\diskshadowout.txt

    this will enter a diskshadow prompt, from here enter.

    list writers detailed

    Once complete type exit on the prompt.

    In the output file used check for the missing files. If they are not there the problem would point to the VSS writer information passed to TSM being incorrect.

    You can raise a call with Microsoft about this, but also contact TSM support as they may be able help workaround this problem with the VSS writers list.

    Thanks

    Scott

  4. #4
    Newcomer
    Join Date
    Aug 2010
    Posts
    23
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Firstable, thank you for your quick response!

    @hogmaster:
    I've already contacted Cristie support, but as soon as it came out that some files are not in backup, they washed their hands and redirect me to IBM.
    Fortunatelly this are only tests, but they concerns critical servers so it is very important that we could restore it in minimum time.

    @ssterry:
    I tried do what you suggested, but there is no diskshadow utility on the system. Quick google search told me that this utility is available only in windows 2008, is there any other utility i could use to perform test that you recommended available in windows 2003 system?

  5. #5
    Newcomer
    Join Date
    Feb 2009
    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Try the Vshadow utility which is part of the Microsoft VSS SDK. The links to download the SDK, as well as full instructions on the Vshadow commands that need to be run can be found in the following DCF:
    http://www-01.ibm.com/support/docvie...id=swg21412250.

    If you have a base image you can play with it is worth removing applications to see which one is affecting the way VSS works. There are many services and applications which change the writers list. Once you know which one is causing your problems you at least know which vendor to work with and have the choice to remove it and allow your backups to complete until the problem can be fixed.

    Let me know how you get on or if you need more help.

  6. #6
    Newcomer
    Join Date
    Aug 2010
    Posts
    23
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    @ssterry:
    "The SDK needs to be installed to access the vshadow.exe file. The SDK does not need to be installed on the same system that will be analyzed since it is a stand-alone application but the copy of vshadow.exe needs to be from the correct operating system version [...]"

    If I understand correctly, I don't have to install VSS SDK on the machine i will analyze, but I can install it, on a different machine. Is that right?
    I'm asking this because affected server is very important machine running 24/7/365 - neither additional software installation nor restarts allowed.
    Last edited by ohwell; 11-09-2010 at 05:58 AM. Reason: additional info

  7. #7
    Member SpecialK's Avatar
    Join Date
    Jul 2009
    Posts
    46
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Have you tried running the DSM GUI, clicking on the Backup button and then browsing under local filesystems to these files to see if they are visible.

    They shouldn't be visible under the local filesystem (as they are systemstate), if they are visible then TSM doesn't recognize them as systemstate. If they are visible, do they have the "no entry" symbol next to them, suggesting that they are excluded by an include/exclude list?

  8. #8
    Member SpecialK's Avatar
    Join Date
    Jul 2009
    Posts
    46
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    The other thing you might want to try is run a command shell and change directory to %SystemRoot%\System32

    Then run the command

    vssadmin list writers

    You should get a list of 6 writers with no errors. 'System Writer', 'WMI Writer', 'Registry Writer', COM+ REGDB Writer', 'MSDEWriter' and 'Event Log Writer'.

  9. #9
    Newcomer
    Join Date
    Aug 2010
    Posts
    23
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    @SpecialK:
    Thank you for your reply! As for the vssadmin list writers I've checked it before and they're all stable showing no errors. However very interesting thing about those files - I ran dsm and checked local file system, and those files could indeed be seen there - ntldr, boot.ini, ntdetect.com - they are all there without exclude mark. To make sure i've checked in some random two others server and those file cannot be seen there.

    As you said it would indicate that those files are not recognized by TSM as system files, but if this is true, then why during normal backup (i performed a full audit) there is no entry about those files?
    And far more important, how to fix this?
    Have you ever had similar problem before?

  10. #10
    Member SpecialK's Avatar
    Join Date
    Jul 2009
    Posts
    46
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Do you have any HP software running on that server, for example "HP Performance Agent Software" ? Any 3rd party performance analysis software?

    I'm not sure if you can attach text files to these posts, but you could run the following command ( assuming your Windows is installed on C: )

    dsmc s c:\ntldr -traceflags=service -tracefile=dsmc.log

    Then try and post the dsm.log file.


    Last edited by SpecialK; 11-09-2010 at 06:52 AM. Reason: Additional information

  11. #11
    Newcomer
    Join Date
    Aug 2010
    Posts
    23
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    @SpecialK:

    As for third party software i could find only those as suspects (however no HP software installed):
    -Cimplicity HMI
    -iHistorian
    -PingPlotter
    -Symantec Enpoint Protection
    -Symantec PcAnywhere

    As attachment there is log you've asked me for, it's to big for a text file in this forum so i've packed it as .zip archive.
    (Please note that i didn't want to reveal any information about server, so i've changed hostname&nodename in that file - better safe than sorry)
    Attached Files Attached Files

  12. #12
    Member SpecialK's Avatar
    Join Date
    Jul 2009
    Posts
    46
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    If this isn't a production system, I'd suggest you try

    sfc /SCANNOW

    But you'd need the original Windows installation CD for it to run OK.

  13. #13
    Newcomer
    Join Date
    Aug 2010
    Posts
    23
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    @SpecialK
    Unfortunatelly this is one of the most critical production servers, so there is no way i could bring him down.
    Could you tell me a little more of what do you suspect? Why do you think it is neccessary or what could've happened?

  14. #14
    Member SpecialK's Avatar
    Join Date
    Jul 2009
    Posts
    46
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    I suspect that something has broken your VSS API. The usual suspects would be any 3rd party applications you have installed. The VSS writers are reporting themselves without errors, so whatever it is, it's subtle.

    It looks like TSM doesn't "see" your boot files as being part of systemstate, so they aren't backed up as part of systemstate. The TSM client determines what is system state by using one of the VSS APIs. The other part of the TSM client that backs up "normal" files, does "see" your boot files as being systemstate, so it quite rightly ignores them. Unfortunately, this means that the files are not backed up at all.

    I would suspect you have more files missing than the 3 boot files you mentioned.

    You could use NTbackup to backup those boot files, this may also be broken if it uses the same VSS APIs.

  15. #15
    Member SpecialK's Avatar
    Join Date
    Jul 2009
    Posts
    46
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    I did notice this in your log file though

    11/09/2010 13:02:53.812 [002896] [5464] : ntfileio.cpp (5494): fioScanDirEntry(): EXCL BootDir=bTrue && BootFile=bTrue: ntldr

    Do you have any server side excludes? Any "unusual" excludes in your dsm.opt file?

    One other thing you could try is to add the following option to your dsm.opt file.

    VSSUSESYSTEMPROVIDER YES

    Then re-run the dsmc command in post#10, without the trace flags.

    Then query the backup to see if ntldr is in it.
    Last edited by SpecialK; 11-09-2010 at 09:12 AM.

  16. #16
    Newcomer
    Join Date
    Aug 2010
    Posts
    23
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    @SpecialK:
    Yes, I have excluded all .mdf and .ldf files as there is also SQL server on that machine.
    please take a look at inclexcl_.txt in attachment.
    Attached Files Attached Files

  17. #17
    Newcomer
    Join Date
    Aug 2010
    Posts
    23
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Quote Originally Posted by SpecialK View Post
    [...]
    I would suspect you have more files missing than the 3 boot files you mentioned.
    [...]
    It is possible, the files i know for sure are missing are ntldr, boot.ini, ntdetect.com, ntoskrnl.exe but there maybe more...

  18. #18
    Member SpecialK's Avatar
    Join Date
    Jul 2009
    Posts
    46
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    One other thing you could try is to add the following option to your dsm.opt file.

    VSSUSESYSTEMPROVIDER YES

    Then re-run the dsmc command in post#10, without the trace flags.

    Then query the backup to see if ntldr is in it.

  19. #19
    Newcomer
    Join Date
    Aug 2010
    Posts
    23
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    @SpecialK

    I did as you requested however the result is the same.

    C:\Program Files\Tivoli\TSM\baclient>dsmc.exe s c:\ntldr

    [...]

    Selective Backup function invoked.

    Preparing to backup 'c:' using 'VSS' snapshot.
    Directory--> 0 \\hostname\c$\ [Sent]
    ANS1016I No eligible files were found.
    Selective Backup processing of '\\hostname\c$\ntldr' finished without failure.



    Total number of objects inspected: 1
    Total number of objects backed up: 1
    Total number of objects updated: 0
    Total number of objects rebound: 0
    Total number of objects deleted: 0
    Total number of objects expired: 0
    Total number of objects failed: 0
    Total number of subfile objects: 0
    Total number of bytes transferred: 325 B
    Data transfer time: 0.00 sec
    Network data transfer rate: 0.00 KB/sec
    Aggregate data transfer rate: 0.04 KB/sec
    Objects compressed by: 0%
    Subfile objects reduced by: 0%
    Elapsed processing time: 00:00:07



    tsm> query backup "{hostname\SystemState\NULL\System State\SystemState}\ntldr"

    ANS1092W No files matching search criteria were found

  20. #20
    Newcomer
    Join Date
    Aug 2010
    Posts
    23
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    I'll try to ask server team to run

    sfc /scannow

    or at least

    sfc /verifyonly

    so we could tell for sure if this is or isn't problem with corrupted files.
    However if you can think of any other possible reasons of this situation, then i would kindly ask you to share your ideas, so time needed for server team to run this command won't be wasted.

  21. #21
    Member SpecialK's Avatar
    Join Date
    Jul 2009
    Posts
    46
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Have you tried an incremental backup of your systemstate; then re-run the query to see if the boot files have been picked up this time?

    After that, I'd have to give it some serious thought.

  22. #22
    Newcomer
    Join Date
    Aug 2010
    Posts
    23
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Incremental backup of server (including systemstate) is performed every night, so I can already tell that it doesn't help.
    Of course if needed I may perform it again, but will it be any different than everynight incremental backup?

  23. #23
    Member SpecialK's Avatar
    Join Date
    Jul 2009
    Posts
    46
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    One other thing you can try is to add the following lines to your dsm.opt (Don't forget to remove them after your test!).

    TRACEFLAGS EXCLUDECACHE
    TESTFLAG SHOWEXCLUDECACHE
    TRACEFILE C:\dsmc.log

    Then run the dsmc command

    dsmc s c:\ntldr

    And post the trace log C:\dsmc.log if possible.
    Last edited by SpecialK; 11-10-2010 at 10:34 AM. Reason: Correct typo

  24. #24
    Newcomer
    Join Date
    Aug 2010
    Posts
    23
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Here is the dsmclog.txt from that try
    Attached Files Attached Files

Page 1 of 2 12 LastLast

Similar Threads

  1. Missing Files (\WINDOWS\Repair\Backup\BootableSystemState\Regist ry\default':)
    By esonnentag in forum Backup / Archive Discussion
    Replies: 3
    Last Post: 01-16-2011, 03:27 AM
  2. full backup of windows 2003 System State
    By nayeemag in forum TSM Client
    Replies: 3
    Last Post: 01-04-2010, 09:59 AM
  3. Files missing in Windows Server 2008 backup
    By sysarn in forum Backup / Archive Discussion
    Replies: 18
    Last Post: 05-11-2009, 10:00 AM
  4. Replies: 3
    Last Post: 07-10-2007, 02:35 AM
  5. Windows Server 2003: Unable to backup System Objects
    By rosmo01 in forum Backup / Archive Discussion
    Replies: 0
    Last Post: 09-29-2003, 09:50 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •