1. Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING) Click the link to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This message will disappear after you have made at least 12 posts. Thank you for your cooperation.

NetApp deduplication

Discussion in 'TSM Operation' started by stephrf, Jan 15, 2013.

  1. stephrf

    stephrf New Member

    Joined:
    Nov 20, 2003
    Messages:
    30
    Likes Received:
    0
    Occupation:
    Storage Administrator
    Location:
    UK
    Hi, is anyone using Netapp FAS to do the deduplication for TSM storage pools?

    This is more from a management aspect:

    1. are you using "thin provisioning" to fool both TSM and the OS that it has more space?

    2. or do you allocate 2 TB to a stgpool (eg file) and then increase the maxscratch to use freed up space?
    Or, won't this work?

    I guess either is change in the way management stgpools is done.

    Any thoughts, ideas would be appreciated.

    thanks Rob
     
  2.  
  3. jonathan274

    jonathan274 New Member

    Joined:
    May 7, 2009
    Messages:
    49
    Likes Received:
    0
    I did some quick research on this out of curiosity. There isn't much out there and it doesn't sound like a recommended approach. See below.

    8 DEDUPLICATION AND TIVOLI STORAGE MANAGER (TSM)
    If Tivoli Storage Manager (TSM) and NetApp deduplication for FAS will be used together, the following should be taken into consideration: Deduplication savings with TSM will not be optimal due to the fact that TSM does not block align data when it writes files out to its volumes. The net result is that there are less duplicate blocks available to deduplicate. TSM compresses files backed up from clients to preserve bandwidth. Compressed data does not usually yield good savings when deduplicated. TSM client-based encryption will result in data with no duplicates. Encrypted data does not usually yield good savings when deduplicated. TSM’s progressive backup methodology backs up only new or changed files, which reduces the number of duplicates, since there are not multiple full backups to consider.

    Source: Netapp deduplication for FAS http://contourds.com/uploads/file/tr-3505.pdf

    As such I would personally suggest utilizing TSM's native deduplication if possible.
     
  4. stephrf

    stephrf New Member

    Joined:
    Nov 20, 2003
    Messages:
    30
    Likes Received:
    0
    Occupation:
    Storage Administrator
    Location:
    UK
    Jonathan, thanks for your advice. I did see good dedupe / compression savings for certain data types. However, unless the data could be kept on the on the FAS in terms of primary and copy pools the time overhead in migration to tape was too slow: rehydration I guess. That coupled with the management headache meant we decided not to use it for this TSM instance.
    However, I am planning to use it for another TSM instance where we will have enough space to keep all the data on the FAS. I'll let you know if the savings make it worthwhile.
    regards Rob
     
  5. cmoeller

    cmoeller New Member

    Joined:
    Aug 17, 2011
    Messages:
    35
    Likes Received:
    0
    I kind of doubt that. Using the filer's deduplication means, that TSM knows nothing about it, right. Copying or migrating data to a tape pool (via TSM that is) for example would indeed require a form of rehydration - but on a storage level, where it's just reading different pointers from the disks. There is a hit on performance, but it decreases even sequential reads not in a manner where it would cause trouble. You can -easily- max out several LTO-5 streams even with a small SATA-only filer using deduped data.

    And regarding the use of a NA filer als storage for TSM - absolutely fine. I'd suggest using CIFS however, that coupled with a 10GbE connection is both easy to use and manage and performs pretty decently, no iSCSI or FC required. Thin provisioning is standard for us anyways ...

    Deduplication and compression are features TSM does have, yes - but I much prefer it right down on the storage level for TSM usage.
     

Share This Page