1. Community Tip: Please Give Thanks to Those Sharing Their Knowledge.
    If you receive helpful answer on this forum, please show thanks to the poster by clicking "LIKE" link for the answer that you found helpful.
  2. Community Tip: Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING)
    Click the link above to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This notice will disappear after you have made at least 3 posts.

Slow backup stg to AWS

Discussion in 'Others' started by gernthefish, Aug 15, 2016.

  1. gernthefish

    gernthefish ADSM.ORG Member

    Joined:
    Mar 3, 2005
    Messages:
    98
    Likes Received:
    0
    I created a 10TB volumes in AWS, mounted it as an ext4 filesystem on our linux TSM server, and created a seq file stg copypool w/ 35 200gb volumes. For some reason, the initial backup stg ran at a constant ~14mb/sec. We are copying from 1 LTO6 tape at a time, but this still seems VERY slow. Based on several metrics we're looked at, I'm suspecting TSM as the bottleneck, but can't pinpoint anything. Any thoughts?
     
  2.  
  3. marclant

    marclant ADSM.ORG Moderator

    Joined:
    Jun 16, 2006
    Messages:
    2,559
    Likes Received:
    358
    Occupation:
    Accelerated Value Specialist for Spectrum Protect
    Location:
    Canada
    I suggest you try to manually copy a large amount of date to AWS outside TSM and see what kind of throughput you are getting. That will help you determine if the bottleneck is outside or inside TSM.
     
  4. gernthefish

    gernthefish ADSM.ORG Member

    Joined:
    Mar 3, 2005
    Messages:
    98
    Likes Received:
    0
    Thanks marclant, we have tried copying data from outside TSM and got much higher throughput. Any ideas on what could be limiting this within TSM?
     
  5. marclant

    marclant ADSM.ORG Moderator

    Joined:
    Jun 16, 2006
    Messages:
    2,559
    Likes Received:
    358
    Occupation:
    Accelerated Value Specialist for Spectrum Protect
    Location:
    Canada
    No. Probably need a deep performance analysis to identify the bottleneck. It's never as easy as enabling the option: go_fast=yes
     
  6. TykneeTym

    TykneeTym ADSM.ORG Member

    Joined:
    Oct 1, 2009
    Messages:
    34
    Likes Received:
    0
    By AWS do you mean S3? I don't think AWS provides block or file level storage out to non-EC2 systems. If this is the case you're doing a translation from File to Object (S3 is object storage) that may be part of your problem. What hardware/software are you using to do this translation?
     
  7. gernthefish

    gernthefish ADSM.ORG Member

    Joined:
    Mar 3, 2005
    Messages:
    98
    Likes Received:
    0
    Yeah, we came to the same conclusion on the object stg, but AWS now has an S3 file gateway that appears to be a much better solution. This allows us to mount a bucket as an NFS share. Testing has shown improved but inconsistent performance #s so far. Still trying to get a better handle on this.
     
  8. ch02all

    ch02all Newcomer

    Joined:
    Jan 13, 2012
    Messages:
    2
    Likes Received:
    0
    Hi Did you ever perform the deep performance analysis? If so what was the conclusion? Also, what improvement did you see using the NFS gateway? you say performance was inconsistent but was the improvement consistent writing to NFS.
     
  9. gernthefish

    gernthefish ADSM.ORG Member

    Joined:
    Mar 3, 2005
    Messages:
    98
    Likes Received:
    0
    No, we haven't gotten to the deep analysis yet. Amazon claims file gateway max rates for upload is 125 mb/sec and 20mb/sec for download, but we're still seeing problems. On a linux NFS client host, we are getting about 60+ mb/sec uploading a single file using the AWS CLI (bypasses the gateway), but much slower results when we use standard cp copy of the same file (uses gateway). With cp, smaller files typically get 20-30 mb/sec, but larger files (>20gb or so) are VERY slow. Like 1 mb/sec or something. Amazon is claiming a fix is in the works, so we'll see what happens.
     
  10. gernthefish

    gernthefish ADSM.ORG Member

    Joined:
    Mar 3, 2005
    Messages:
    98
    Likes Received:
    0
: AWS, Amazon

Share This Page