Results 1 to 14 of 14
  1. #1
    Member
    Join Date
    Feb 2011
    Posts
    306
    Thanks
    14
    Thanked 0 Times in 0 Posts

    Post How to find how much of the backup of a file system is completed

    Hi all,

    Anyway to find out how much of the filesystem is been backedup ?

    I am having a full backup job of 22TB filesystem running from about 73 days now. Still not sure how many more days this is going to take ???

  2. #2
    Member
    Join Date
    Feb 2011
    Posts
    306
    Thanks
    14
    Thanked 0 Times in 0 Posts

    Default

    Hi all,

    Does "bytes recvd" in the output of query session command will say anything about how much of the backup job is complete ?
    Currently this query on the backup job says that it received 11.3TB so far, does this mean it completed backup of 11.3TB of data of total 22TB ?

    Code:
    q se 18626 f=d
    
                  Sess Number: 18,626
                 Comm. Method: Tcp/Ip
                   Sess State: Run
                    Wait Time: 0 S
                   Bytes Sent: 3.6 M
                  Bytes Recvd: 11.3 T
                    Sess Type: Node
                     Platform: Linux86
                  Client Name: Linux09
          Media Access Status: Current output volumes:  BKUPJA,C01218,(6449 Seconds)
                    User Name:
    Date/Time First Data Sent: 08/30/12   00:12:05
       Proxy By Storage Agent:

  3. #3
    Senior Member Trident's Avatar
    Join Date
    Apr 2007
    Location
    Oslo, Norway
    Posts
    262
    Thanks
    2
    Thanked 19 Times in 19 Posts

    Default

    If this is the first backup, and you do not have client compression enabled, then this is true.
    -= Trident =-

  4. #4
    Senior Member
    Join Date
    Dec 2003
    Location
    Greater Toronto Area Canada
    Posts
    408
    Thanks
    4
    Thanked 7 Times in 7 Posts

    Default

    You have to admit that this is a pretty poor transfer rate for the backup. It works out to around 154 GB per day.

  5. #5
    Member
    Join Date
    Feb 2011
    Posts
    306
    Thanks
    14
    Thanked 0 Times in 0 Posts

    Default

    Thanks Trident.

    Hi rallingham, yes this is very poor transfer rate.
    How can i improve things here ?
    What are the things i should be checking to improve the TSM performance ?

  6. #6
    Senior Member
    Join Date
    Dec 2003
    Location
    Greater Toronto Area Canada
    Posts
    408
    Thanks
    4
    Thanked 7 Times in 7 Posts

    Default

    Hard to tell right now. I would suspect your network connection as the transfer rate is so poor. But it is impossible to tell without a lot of details. For example are the connections through GB ethernet or lower?

  7. #7
    Member
    Join Date
    May 2012
    Location
    TSM World
    Posts
    34
    Thanks
    3
    Thanked 1 Time in 1 Post

    Default

    You are also backing up straight to tape it seems. A few factors to consider
    Do you have Collocation defined for that node?
    Are there are processes using drives?
    The size of the data is a factor as well. Are these millions of objects or a few big files
    ... and so on

    You can determine how many tapes you've used so far by trying something like this, fill in for X

    select count(volume_name) as vol_count from volumeusage where node_name='Linux09' and stgpool_name='XXXX'

    You can also try

    Query OCCupancy node_name

  8. #8
    Senior Member Trident's Avatar
    Join Date
    Apr 2007
    Location
    Oslo, Norway
    Posts
    262
    Thanks
    2
    Thanked 19 Times in 19 Posts

    Default

    If you have the possibility, you can start multiple threads from the client.

    dsmc i d:\dir1\* -subdir=yes
    dsmc i d:\dir2\* -subdir=yes
    ..

    And so on.

    For small files, direct them to a DISK type devclass.

    There are aslo some tuning parameters dsm.opt that can be used.

    My favorite is to set TCPWindowSize to 0 (let the os handle windowscaling).
    -= Trident =-

  9. #9
    Member
    Join Date
    Feb 2011
    Posts
    306
    Thanks
    14
    Thanked 0 Times in 0 Posts

    Default

    thanks rallingham and Trident,

    Hi MrOdysseus,

    looks like data is going straight to tapes only (from what i see of q stg commmand's output; the HI and LOW mig of stgpool are being reset to 0 and 0)

    How to check if collocation is defined ?

    Can you explain little more on "any processess using drives" ? you mean tape drives ? i don't see any process related this particular bacup job but there is a stgpool backup running at the moment and q mount command shows that 7 out of 8 tape drives are being used.

    select command shows zero count
    Code:
    select count(volume_name) as vol_count from volumeusage where node_name='Linux09' and stgpool_name='BKUPJA'
    
      VOL_COUNT
    -----------
              0
    but below is the q occ for this filespace

    Code:
    query occ Linux09 /DATA
    
    Node Name      Type     Filespace       FSID     Storage        Number of      Physical       Logical
                            Name                     Pool Name          Files         Space         Space
                                                                                   Occupied      Occupied
                                                                                       (MB)          (MB)
    ----------     ----     ----------     -----     ----------     ---------     ---------     ---------
    Linux09     Bkup     /DATA         34     BKUPPOOL            72         69.79         69.79
     
     Linux09     Bkup     /DATA         34     BKUPJA         5,012,713     18,976,99     18,972,23
                                                                             9.96          4.40
    
    Linux09     Bkup     /DATA         34     CPOFFJA      4,378,495     16,968,44     16,967,68
                                                                             7.93          8.60
     
    Linux09   Bkup     /DATA         34     CPONJA       4,718,526     17,742,62     17,741,86
                                                                              9.29          9.97

  10. #10
    Member
    Join Date
    May 2012
    Location
    TSM World
    Posts
    34
    Thanks
    3
    Thanked 1 Time in 1 Post

    Default

    Hi there...
    Yes, I wanted to ask if there was anything else using the Tape Drives. You could end up with tape contention, slowing down the backup. I'm not sure if that is the case.

    Can you try the other storage pool names? (CPOFFJA, and CPONJA). This will only give you an idea of how many tapes you are using for that node, and check if that number grows.

    I take it CPOFF is the offsite copy and CPON is the onsite.

    Is BKUPJA the disk storage pool?

    It does seem like you've backed up 18TB already.

    The way you do collocation is on the Offsite/Onsite copies You would either define to collocate by node or group, those storage pool wide settings.

    When you do Q STG CPOFFJA you can see a option for

    Collocate?:

    Then you can define collocation groups and add members to those groups

    q collocg or
    q collocg f=d

    If the stg is defined to collocate by node, the backups to tape could go faster. I don't recommend making changes as that could lead to heavy reclaims etc... this could be something to watch out for down the road.


    Do you have enough scratch tapes? or tapes allocated to the storage pool. It does seem like a very long time to backup 22TB
    Is the server itself using on a 100MB NIC card instead of 1GB NIC?


    Hope this info helps.

  11. #11
    Member
    Join Date
    Feb 2011
    Posts
    306
    Thanks
    14
    Thanked 0 Times in 0 Posts

    Default

    Hi MrOdysseus,

    yes, CPOFFJA and CPONJA are offsite and onsite copy pools.
    BKUPJA is a primay tape pool and BKUPPOOL is the disk pool.
    When i query BKUPJA stgpool, looks like collocation is set to "group".
    yes, there are few scratch tapes (12 now). i will always try to keep atleast 10 scratch tapes available.

    Does backing up data directly to tape take long time ?
    I suspect that the data is going directly to tape instead of first going to disk pool BKUPPOOL (because the HI pct migr and LO pct migr values are being reset every night to 0 and 0. This causing "migration" process to run all the time and also using the tape drives. Every morning i am changing HI and LO values of migration to 90 and 10. I am having hard time to find why this is being reset 0 and 0)

  12. #12
    Member
    Join Date
    May 2012
    Location
    TSM World
    Posts
    34
    Thanks
    3
    Thanked 1 Time in 1 Post

    Default

    Hi UnixandTSM ...

    Sounds like a scheduled task is resetting it for you.. It could either be an external task or a administrative one.

    try q event * t=a or

    q event * t=a begind=-1

    You could also dig through the logs and see what changing those thresholds for you

    q act begint=-24 search=HIghmig


    If youh have tons of scripts, you can output everything into a file and search that file.

    q script * > c:\scripts.txt

  13. #13
    Member
    Join Date
    Feb 2011
    Posts
    306
    Thanks
    14
    Thanked 0 Times in 0 Posts

    Default

    Hi MrOdysseus,

    Just an hour ago i updated the stgpool to values HI=90 and LO=10 but then again it reset.
    below is the actlog query.
    Code:
    tsm>q actlog begint=-1 search=HIghmig
    
    Date/Time                Message
    --------------------     ----------------------------------------------------------
    11/12/12   10:19:51      ANR1000I Migration process 31403 started for storage pool
                              BACKUPPOOL automatically, highMig=0, lowMig=0,
                              duration=No. (PROCESS: 31403)
    Not sure what is making the migration to run automatically?

  14. #14
    Senior Member
    Join Date
    Dec 2009
    Location
    Brno
    Posts
    112
    Thanks
    1
    Thanked 19 Times in 18 Posts

    Default

    q act s=BACKUPPOOL
    q act s="update stg"
    q act s="PROCESS: 31403"
    and look what/who updated the stg pool

Similar Threads

  1. Trying to find a backup file without success
    By etchingsj in forum TSM Operation
    Replies: 6
    Last Post: 08-14-2012, 03:52 PM
  2. How to find retained backup file versions
    By tasr in forum Backup / Archive Discussion
    Replies: 2
    Last Post: 11-17-2008, 02:19 PM
  3. Replies: 4
    Last Post: 03-19-2008, 11:33 AM
  4. Splitting a file system or file for simultaneous backup to 2 devices
    By uucp in forum Backup / Archive Discussion
    Replies: 3
    Last Post: 04-13-2005, 01:45 AM
  5. Good backup, but can't find file.
    By augfirst in forum Backup / Archive Discussion
    Replies: 4
    Last Post: 06-20-2003, 08:52 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •