1. Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING) Click the link to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This message will disappear after you have made at least 12 posts. Thank you for your cooperation.

Slow TSM backup of one drive

Discussion in 'Backup / Archive Discussion' started by wankenobi, Nov 6, 2012.

  1. wankenobi

    wankenobi New Member

    Joined:
    Nov 6, 2012
    Messages:
    7
    Likes Received:
    0
    Dear,

    I have a problem with TSM backup .Backup taking more than 24 hours to finish .What I understand is that the issue due to millions of object on one drive (around 11'00'400 )
    I use tsm client version 6.3.0.18
    Server:Windows 2003 enterprise

    Any recomendation to troubleshoot this issue?

    Thanks

    Wan
     
  2.  
  3. MrOdysseus

    MrOdysseus New Member

    Joined:
    May 16, 2012
    Messages:
    34
    Likes Received:
    1
    Occupation:
    SAN/TSM Administrator
    Location:
    TSM World
    Hi Wan,

    This is definitely a job for Journaling. Check out the following URL http://pic.dhe.ibm.com/infocenter/t...ic=/com.ibm.itsm.client.doc/t_cfg_jbbwin.html

    Make sure to edit/setup the Journal config file to preserve DB on exit. There are times where the Journal will have to be rebuilt. (server Crash, or the Journal simply goes corrupt).

    We have a server with close to 20Million files, and the Journal may have to rebuild once a week. however subsequent backups complete in under 30 minutes picking up on avg 150,000 files worth of change.

    -Odysseus
     
    Last edited: Nov 6, 2012
  4. wankenobi

    wankenobi New Member

    Joined:
    Nov 6, 2012
    Messages:
    7
    Likes Received:
    0
    Hello Odysseus,

    Thanks for the tip .I have enabled journal for that problematic drive. Result I will update you after scheduled backup

    Thanks a lot
    Wan
     
  5. wankenobi

    wankenobi New Member

    Joined:
    Nov 6, 2012
    Messages:
    7
    Likes Received:
    0
    Journal technique did not help much .My backup still taking more than 24hours to finish .Issue is with one drive and folder in that have millions of small files .

    any other tip?
     
  6. LED888

    LED888 Moderator

    Joined:
    Oct 15, 2002
    Messages:
    724
    Likes Received:
    50
    Journaling is good when only about 10-12% of the data change.
    If you have more than 12% of the data changing, journaling is not going to help.

    or

    Do an image backup at the beginning of the month, and then do incremental backup.

    or

    Use the incrbydate parameters.
    There are cavates, review the client manual.

    or

    Create another node(s) to assist with the backup.
    I'm hoping that the os is 64bit. If the os is 32bit, then your going to run into the 4GB memory limitation.

    Good Luck,
    Sias
     
  7. wankenobi

    wankenobi New Member

    Joined:
    Nov 6, 2012
    Messages:
    7
    Likes Received:
    0
    Thanks Sias,

    My server is W2k3 64bit .By the way I will try incrbydate backup .

    Is this means that tsm will not support backing huge files? I have checked the drive and the size only 150GB but more than 1 million files !!
     
  8. GregE

    GregE Senior Member

    Joined:
    May 12, 2006
    Messages:
    2,100
    Likes Received:
    31
    When you enable Journaling, the first backup after that is going to take just as long as before. But this is because the first backup is populating the journal. Future backups should be shorter because your journal is populated and "tell" TSM server what files to backup without TSM scanning all of the files to find out what to backup.

    I have a system with over 60 million files on one drive. Approximately 2.3 TB of data Journaling keep this under control. But when I have to re-create the journal, it takes 3-4 days. Also on this one we do an image backup direct to tape once a week.
     
  9. wankenobi

    wankenobi New Member

    Joined:
    Nov 6, 2012
    Messages:
    7
    Likes Received:
    0
    It took 36 hours to finish the backup .I am waiting to see my next backup time .Will keep you post the result
     
  10. MrOdysseus

    MrOdysseus New Member

    Joined:
    May 16, 2012
    Messages:
    34
    Likes Received:
    1
    Occupation:
    SAN/TSM Administrator
    Location:
    TSM World
    Good luck with it.. Yes, Journaling will take some time the first time around. In our case, we have 21Million objects, all tiny files with a daily change of about 200,000 between expires and new files. This completes in about 30 minutes with Journaling running. For a incremental backup to re-create the Journal, It will take 12 hours or so.

    The Incrybdate is good. But again like LED888 said, read the manual about it. It may not apply to you.

    It also depends on your TSM servers, and Data Servers. If they are beefy. You can get a lot of data backed up in a short time as well.
     
  11. wankenobi

    wankenobi New Member

    Joined:
    Nov 6, 2012
    Messages:
    7
    Likes Received:
    0
    TSM journal helped for 2-3 days but now again the situation is back with same problem.backup taking long time .

    This is the current update!
     
  12. hogmaster

    hogmaster Senior Member

    Joined:
    Nov 9, 2002
    Messages:
    521
    Likes Received:
    20
    Occupation:
    Storage engineer
    Now is the time to try incrbydate. The during the weekend you run a normal inc and next week you run a incr by date.
    Also follw the advice of LED888 to run a image backup at regular intervals.


    Do all the files exist in the same directory?
    If not you can try to run a script that will traverse the directory tree and start a different backup for each directory without -subdir.
    then the script can monitor the number of active dsmc processes and spawn a new everytime the number of precces is les than say 5.

    Is there any pattern to the filenames when they are created? if the pattern is easy like todaysfile20120101_xxxxx then you can script a incremntal that will only target this months files like dsmc inc todaysfile2012011*


    Good Luck
     
  13. wankenobi

    wankenobi New Member

    Joined:
    Nov 6, 2012
    Messages:
    7
    Likes Received:
    0
    Thanks for your reply .
    by the way I forget to add
    I have seen a new error on the log
    ANS2079E Error processing '\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy271': error scanning snapshot .
     
  14. MrOdysseus

    MrOdysseus New Member

    Joined:
    May 16, 2012
    Messages:
    34
    Likes Received:
    1
    Occupation:
    SAN/TSM Administrator
    Location:
    TSM World
  15. hogmaster

    hogmaster Senior Member

    Joined:
    Nov 9, 2002
    Messages:
    521
    Likes Received:
    20
    Occupation:
    Storage engineer
    When you define the journal location make sure that you put it on a drive that have enough space. This could be one reason for your performance dropping again after a few days. the journal needs space.
    you can use vssadmin with Resize ShadowStorage option to tell vss where to write the chnages during the backup.
    do not forget to check you windows logs for clues to what is happening.
    It look like you are starting to get a grip on the situation now that you are starting to see proper error messages (good thing!)

    look for vss related issues in the windows eventlog.
     
: slow backup

Share This Page