• Please help support our sponsors by considering their products and services.
    Our sponsors enable us to serve you with this high-speed Internet connection and fast webservers you are currently using at ADSM.ORG.
    They support this free flow of information and knowledge exchange service at no cost to you.

    Please welcome our latest sponsor Tectrade . We can show our appreciation by learning more about Tectrade Solutions
  • Community Tip: Please Give Thanks to Those Sharing Their Knowledge.

    If you receive helpful answer on this forum, please show thanks to the poster by clicking "LIKE" link for the answer that you found helpful.

  • Community Tip: Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING)

    Click the link above to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This notice will disappear after you have made at least 3 posts.

NetApp deduplication

stephrf

Active Newcomer
#1
Hi, is anyone using Netapp FAS to do the deduplication for TSM storage pools?

This is more from a management aspect:

1. are you using "thin provisioning" to fool both TSM and the OS that it has more space?

2. or do you allocate 2 TB to a stgpool (eg file) and then increase the maxscratch to use freed up space?
Or, won't this work?

I guess either is change in the way management stgpools is done.

Any thoughts, ideas would be appreciated.

thanks Rob
 

jonathan274

ADSM.ORG Member
#2
I did some quick research on this out of curiosity. There isn't much out there and it doesn't sound like a recommended approach. See below.

8 DEDUPLICATION AND TIVOLI STORAGE MANAGER (TSM)
If Tivoli Storage Manager (TSM) and NetApp deduplication for FAS will be used together, the following should be taken into consideration: Deduplication savings with TSM will not be optimal due to the fact that TSM does not block align data when it writes files out to its volumes. The net result is that there are less duplicate blocks available to deduplicate. TSM compresses files backed up from clients to preserve bandwidth. Compressed data does not usually yield good savings when deduplicated. TSM client-based encryption will result in data with no duplicates. Encrypted data does not usually yield good savings when deduplicated. TSM’s progressive backup methodology backs up only new or changed files, which reduces the number of duplicates, since there are not multiple full backups to consider.

Source: Netapp deduplication for FAS http://contourds.com/uploads/file/tr-3505.pdf

As such I would personally suggest utilizing TSM's native deduplication if possible.
 

stephrf

Active Newcomer
#3
Jonathan, thanks for your advice. I did see good dedupe / compression savings for certain data types. However, unless the data could be kept on the on the FAS in terms of primary and copy pools the time overhead in migration to tape was too slow: rehydration I guess. That coupled with the management headache meant we decided not to use it for this TSM instance.
However, I am planning to use it for another TSM instance where we will have enough space to keep all the data on the FAS. I'll let you know if the savings make it worthwhile.
regards Rob
 

cmoeller

ADSM.ORG Member
#4
However, unless the data could be kept on the on the FAS in terms of primary and copy pools the time overhead in migration to tape was too slow: rehydration I guess.
I kind of doubt that. Using the filer's deduplication means, that TSM knows nothing about it, right. Copying or migrating data to a tape pool (via TSM that is) for example would indeed require a form of rehydration - but on a storage level, where it's just reading different pointers from the disks. There is a hit on performance, but it decreases even sequential reads not in a manner where it would cause trouble. You can -easily- max out several LTO-5 streams even with a small SATA-only filer using deduped data.

And regarding the use of a NA filer als storage for TSM - absolutely fine. I'd suggest using CIFS however, that coupled with a 10GbE connection is both easy to use and manage and performs pretty decently, no iSCSI or FC required. Thin provisioning is standard for us anyways ...

Deduplication and compression are features TSM does have, yes - but I much prefer it right down on the storage level for TSM usage.
 

tpesselier

ADSM.ORG Member
#5
Hello

I have primary storagepool on fas i know that you should not enable tsm deduplication and netapp deduplication at the same time ok.

But in my case i don't use TSM deduplication so it's possible to just enable NettApp deduplication ? it's work ?

Best
 

moon-buddy

ADSM.ORG Moderator
#6
I have been using Data Domain for back end storage for over 9 years now and the setting is set to TSM no compression so Data Domain does the de-duplication.

If you set the de-duplication on the TSM side, you will end up using more space (with Data Domain) on your back end storage. Netapp, which we also have, does not de-duplicate as much but still has a little edge over native TSM de-duplication.

Do not turn ON back end storage de-duplication if TSM de-duplication is ON. This is a waste of processing cycles and the back end will not de-duplicate any further. Least to say, been there and had done it.

Personally, if one is certain to use native TSM de-deduplication, I would not go for expensive back end storage like Netapp or Data Domain. I will just buy high reliability, replicating and cluster-aware JBOD. SAN arrays are possible candidates.
 
Last edited:

tpesselier

ADSM.ORG Member
#7
I know what you mean moon but I do not choose my infrastructure :-(

After done several tests, the inline deduplication of netapp works very badly with a storagepool tsm.

The deduplication nettapp is a schedule and it needs the volume not in use

I also have a datadomain, even with the best tuning is very slow because the speed of each stream is limited with datadomain, moreover you can not make instant restore with vmware because it is too slow for me

I think source dedup with tsm is the best, the object storage appliances are not bad either if you use storage contenair
 

Advertise at ADSM.ORG

If you are reading this, so are your potential customer. Advertise at ADSM.ORG right now.

UpCloud high performance VPS at $5/month

Get started with $25 in credits on Cloud Servers. You must use link below to receive the credit. Use the promo to get upto 5 month of FREE Linux VPS.

The Spectrum Protect TLA (Three-Letter Acronym): ISP or something else?

  • Every product needs a TLA, Let's call it ISP (IBM Spectrum Protect).

    Votes: 8 23.5%
  • Keep using TSM for Spectrum Protect.

    Votes: 17 50.0%
  • Let's be formal and just say Spectrum Protect

    Votes: 5 14.7%
  • Other (please comement)

    Votes: 4 11.8%

Forum statistics

Threads
30,926
Messages
131,573
Members
21,207
Latest member
Nur03