Results 1 to 14 of 14
  1. #1
    Newcomer
    Join Date
    Nov 2012
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Slow TSM backup of one drive

    Dear,

    I have a problem with TSM backup .Backup taking more than 24 hours to finish .What I understand is that the issue due to millions of object on one drive (around 11'00'400 )
    I use tsm client version 6.3.0.18
    Server:Windows 2003 enterprise

    Any recomendation to troubleshoot this issue?

    Thanks

    Wan

  2. #2
    Member
    Join Date
    May 2012
    Location
    TSM World
    Posts
    34
    Thanks
    3
    Thanked 1 Time in 1 Post

    Default

    Hi Wan,

    This is definitely a job for Journaling. Check out the following URL http://pic.dhe.ibm.com/infocenter/ts...fg_jbbwin.html

    Make sure to edit/setup the Journal config file to preserve DB on exit. There are times where the Journal will have to be rebuilt. (server Crash, or the Journal simply goes corrupt).

    We have a server with close to 20Million files, and the Journal may have to rebuild once a week. however subsequent backups complete in under 30 minutes picking up on avg 150,000 files worth of change.

    -Odysseus
    Last edited by MrOdysseus; 11-06-2012 at 09:21 AM.

  3. #3
    Newcomer
    Join Date
    Nov 2012
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Hello Odysseus,

    Thanks for the tip .I have enabled journal for that problematic drive. Result I will update you after scheduled backup

    Thanks a lot
    Wan

  4. #4
    Newcomer
    Join Date
    Nov 2012
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Journal technique did not help much .My backup still taking more than 24hours to finish .Issue is with one drive and folder in that have millions of small files .

    any other tip?

  5. #5

    Default

    Journaling is good when only about 10-12% of the data change.
    If you have more than 12% of the data changing, journaling is not going to help.

    or

    Do an image backup at the beginning of the month, and then do incremental backup.

    or

    Use the incrbydate parameters.
    There are cavates, review the client manual.

    or

    Create another node(s) to assist with the backup.
    I'm hoping that the os is 64bit. If the os is 32bit, then your going to run into the 4GB memory limitation.

    Good Luck,
    Sias

  6. #6
    Newcomer
    Join Date
    Nov 2012
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Thanks Sias,

    My server is W2k3 64bit .By the way I will try incrbydate backup .

    Is this means that tsm will not support backing huge files? I have checked the drive and the size only 150GB but more than 1 million files !!

  7. #7
    Senior Member GregE's Avatar
    Join Date
    May 2006
    Posts
    2,100
    Thanks
    9
    Thanked 17 Times in 16 Posts

    Default

    Quote Originally Posted by wankenobi View Post
    Journal technique did not help much .My backup still taking more than 24hours to finish .Issue is with one drive and folder in that have millions of small files.
    When you enable Journaling, the first backup after that is going to take just as long as before. But this is because the first backup is populating the journal. Future backups should be shorter because your journal is populated and "tell" TSM server what files to backup without TSM scanning all of the files to find out what to backup.

    I have a system with over 60 million files on one drive. Approximately 2.3 TB of data Journaling keep this under control. But when I have to re-create the journal, it takes 3-4 days. Also on this one we do an image backup direct to tape once a week.

  8. #8
    Newcomer
    Join Date
    Nov 2012
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    It took 36 hours to finish the backup .I am waiting to see my next backup time .Will keep you post the result

  9. #9
    Member
    Join Date
    May 2012
    Location
    TSM World
    Posts
    34
    Thanks
    3
    Thanked 1 Time in 1 Post

    Default

    Good luck with it.. Yes, Journaling will take some time the first time around. In our case, we have 21Million objects, all tiny files with a daily change of about 200,000 between expires and new files. This completes in about 30 minutes with Journaling running. For a incremental backup to re-create the Journal, It will take 12 hours or so.

    The Incrybdate is good. But again like LED888 said, read the manual about it. It may not apply to you.

    It also depends on your TSM servers, and Data Servers. If they are beefy. You can get a lot of data backed up in a short time as well.

  10. #10
    Newcomer
    Join Date
    Nov 2012
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    TSM journal helped for 2-3 days but now again the situation is back with same problem.backup taking long time .

    This is the current update!

  11. #11
    Senior Member
    Join Date
    Nov 2002
    Posts
    516
    Thanks
    2
    Thanked 4 Times in 4 Posts

    Default

    Now is the time to try incrbydate. The during the weekend you run a normal inc and next week you run a incr by date.
    Also follw the advice of LED888 to run a image backup at regular intervals.


    Do all the files exist in the same directory?
    If not you can try to run a script that will traverse the directory tree and start a different backup for each directory without -subdir.
    then the script can monitor the number of active dsmc processes and spawn a new everytime the number of precces is les than say 5.

    Is there any pattern to the filenames when they are created? if the pattern is easy like todaysfile20120101_xxxxx then you can script a incremntal that will only target this months files like dsmc inc todaysfile2012011*


    Good Luck
    Windows, does it come with a gui?

  12. #12
    Newcomer
    Join Date
    Nov 2012
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Thanks for your reply .
    by the way I forget to add
    I have seen a new error on the log
    ANS2079E Error processing '\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy271 ': error scanning snapshot .

  13. #13
    Member
    Join Date
    May 2012
    Location
    TSM World
    Posts
    34
    Thanks
    3
    Thanked 1 Time in 1 Post

    Default

    Good luck trying out hogmaster recommendation.. for the error above it has to deal with Microsoft Volume Shadowcopy Service (VSS)


    http://pic.dhe.ibm.com/infocenter/ts...30%37%39%22%20

  14. #14
    Senior Member
    Join Date
    Nov 2002
    Posts
    516
    Thanks
    2
    Thanked 4 Times in 4 Posts

    Default

    When you define the journal location make sure that you put it on a drive that have enough space. This could be one reason for your performance dropping again after a few days. the journal needs space.
    you can use vssadmin with Resize ShadowStorage option to tell vss where to write the chnages during the backup.
    do not forget to check you windows logs for clues to what is happening.
    It look like you are starting to get a grip on the situation now that you are starting to see proper error messages (good thing!)

    look for vss related issues in the windows eventlog.
    Windows, does it come with a gui?

Similar Threads

  1. Slow backup in TSM 6.1
    By Masonit in forum Performance Tuning
    Replies: 4
    Last Post: 10-20-2009, 06:18 AM
  2. TSM backup very slow
    By SajidSurti in forum Backup / Archive Discussion
    Replies: 8
    Last Post: 05-07-2009, 06:27 AM
  3. rman backup slow after tsm 5.2 upgrade to 5.2.8.0??
    By fwang in forum Performance Tuning
    Replies: 1
    Last Post: 02-20-2008, 11:49 PM
  4. slow writing to tape drive
    By elam in forum Tape / Media Library
    Replies: 1
    Last Post: 09-05-2007, 10:40 AM
  5. TSM backup of unix files to itself is slow
    By nonamire in forum Backup / Archive Discussion
    Replies: 5
    Last Post: 07-27-2006, 02:05 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •