Page 2 of 2 FirstFirst 12
Results 25 to 33 of 33
  1. #25
    Senior Member Jeff_Jeske's Avatar
    Join Date
    Jul 2006
    Location
    Stevens Point, WI
    Posts
    485
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default

    We have a NAS, TAPE, VTL, and SATA and we need more TSM storage. I have been recommending we purchase an EMC EDL over DD, NAS, or SATA.

    We currently have a VTL and it has been 100% rock solid from day 1. No issues of any kind and we REALLY work it over. The compression is excellent and the the recovery of small files from multiple volumes is almost instant. In terms of recovery, the appliance driven VTL is significantly quicker than windows managed SATA disk.

    The CDL is also once removed from the OS system administrators. I don't like that my entire SATA file pool could be whacked by a click happy admin.

    Since our business is insurance 90% of our data is database data that arrives in TSM precompressed and sometimes encrypted. Furthermore our database retention is only 10 days. If you ask an honest data domain or other dedup engineer what the likely return on investment would be on 10 day old, compressed, and encrypted data they will tell you not to waste your time. This is without any form of software dedupe.

    Appliance dedup today is filling a void that is rapidly being replaced or eliminated by encryption and software dedup. It may have some short term rewards (but not without up front cost) for some but it will be superseded software technology.

    Quick and dirty.... I feel the best approach is to buy large quantities of disk that can store any form of one and zero that you put there. I believe large quantities of inexpensive disk will be much more beneficial to a backup engineer than a medium supply of expensive deduped disk.

    I believe I could run our entire backup platform with only physical LTO5 for big files and VTL devices for little files. In the backup world I feel an abundance of capacity outweighs an abundance of complexity.

  2. #26

    Default Real world stats

    This article has some interesting statistics related to TSM de-dupe. http://bit.ly/bqgYnj
    Ken Bury
    Client Technical Specialist
    Tivoli Storage Software
    IBM Corporation

    Twitter: http://www.twitter.com/KenBury
    Linkedin: http://linkedin.com/in/KenBury
    Blog: http://thinkstoragemodel.blogspot.com/
    Tivoli Request for Enhancement (RFE) site: http://www.ibm.com/developerworks/rfe/?BRAND_ID=90

  3. #27
    Senior Member Jeff_Jeske's Avatar
    Join Date
    Jul 2006
    Location
    Stevens Point, WI
    Posts
    485
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default

    After nearly two years....I'd like to bring this back as hopefully more people are familiar with dedup operations and such.

    We recently shifted to occupancy based licensing providing us unlimited usage of the SQL TDP. Therefore my primary objective is to leverage the TDP for SQL backups WITH CLIENT SIDE COMPRESSION.

    I have deployed the TDP and configured the TDPSQL folder dsm.opt file to use the dedup managment classes and deduplication yes options. The client is configured with dedupe client or server option. My question is ... are there any other options that need to be configured. If not, how do I determine if client side dedupe is working or how well the data is deduplicating? I can't seem to find a log that provides me with this information.

    I can see that the data eventually is deduplicated but I'm not certain where that is taking place. Can I kill identify duplicates and still get client side deduplication?



    The biggest consumer of my Storage and NIC resources are my locally stored SQL flatfile backups.

    Jeff Jeske - Sentry Insurance Storage Engineer
    IBM Certified Advanced Deployment Professional
    SNIA Certified Storage Engineer
    EMC Proven Professional
    ITIL Certified

    Sometimes the more you know... the harder it is to make a good decision!

  4. #28
    Member
    Join Date
    May 2006
    Posts
    197
    Thanks
    3
    Thanked 6 Times in 5 Posts

    Default

    I have switched companies since this thread started. Now I have an opportunity to work with Data Domain and TSM. I hope to have some test hardware soon where I can try some TSM backups without any other data on the array to try to get some more pure numbers. It looks like TSM writing to Data Domain will be in the 3:1 to 7:1 range for unstructured data. This is very similar to what I had seen previously with ProtecTIER.

    I had the opportunity to attend the Pulse conference this year. I found the information from other users there on TSM deduplication to be disappointing. Several speakers talked about getting 30% - 60% reduction in their backup data footprint. Compared to TSM 5.5 they reported their DB grew by a factor of 10 or more on the servers where they achieved these numbers. One presenter said their TSM 5.5 DB was 45 GB and once the clients were moved to TSM 6.3 utilizing client/server deduplication their new DB was around 600GB.

    Does this match others experiences? Really makes me want to shy away from TSM Dedup.

    -Rowl

  5. #29
    Senior Member Jeff_Jeske's Avatar
    Join Date
    Jul 2006
    Location
    Stevens Point, WI
    Posts
    485
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default

    Well... much has changed since I started this thread... but here are my findings:

    For our storage purchase we decided to buy a pure EMC CX4 sata frame and leverage client side dedupe and compression. We ended up moving about 1000 clients from six TSM 5.5 CDL instances to two TSM 6.2 file device class type nodes. On 5.5 we tried to keep our TSM database down to about 100GB and on 6.2 we are aiming for 200GB. This limit was set to allow us to quickly recover the TSM DB in the event we need to recover it. All of our TSM servers are on Windows 2008.

    Initially we tested server side and client side deduplication and that was fairly disappointing. I was expecting to dedupe the crap out of the OS and full SQL DB exports that are run each night but it simply didn't play out that way. On the bright side all of our SATA clients are using client side compression and we are seeing 2:1 on unstructured data and sometimes 6:1 on SQL DB data. Be advised we saw the same level of compression with version 5.5.

    Now... We do see some server side deduplication but the question is whether its worth it or not. Our TSM databases reside on very expensive EMC VMAX FC (about $35k per TB) storage and our backups go to inexpensive SATA disk (about $2K per TB). Is it really worth adding the complexity and database growth of dedupe? So far for us it doens't look good for dedupe. This is even more true when you think how much compressed data you can store on an LTO4 tape!!!

    Overall I am very happy with TSM 6.2 as it allows us to register and store tons more data per instance. We plan to consolidate about eight TSM 5.5 instances into four TSM 6.2 instances and have capacity to spare. Do do not plan to leverage Dedupe unless we find candidates that TRULY dedupe well as in 10:1. So far we are happy with what compression gives us. (For example: I have a 320GB SQL database that compresses down to 60GB using the TDP with compression.)

    My recommendation would be to leverage client side compression wherever you can. Then test sample clients for dedupe and compare that to what you would achieve with compression alone. Lastly we did test running compression and dedupe together and it really didn't provide a tangible benefit.

    As for other threads on this topic.... I found quite a few but when I asked deeper questions no one was able to provide the answers I was looking for. This leads me to believe most admins really don't understand what is going on behind the scenes.

    Jeff Jeske - Sentry Insurance Storage Engineer
    IBM Certified Advanced Deployment Professional
    SNIA Certified Storage Engineer
    EMC Proven Professional
    ITIL Certified

    Sometimes the more you know... the harder it is to make a good decision!

  6. #30
    Moderator chad_small's Avatar
    Join Date
    Dec 2002
    Location
    Gilbert, AZ
    Posts
    2,191
    Thanks
    1
    Thanked 25 Times in 24 Posts

    Default

    Currently I work in an environment with TSM 6.2.3 servers backing up to Data Domain storage for dedupe. Now the problem with any dedupe storage solution is trying to create the offsite while the dedupe disk is trying to compress the data. The whole offsite tape creation can be unbearably slow. Luckily we have a remote site that we replicate our Data Domain's too so I don't have to worry about offsite tape creation. So if you are looking into a dedupe solution I would recommend you identify who you plan to keep data in an offsite location. Trying to generate tapes could be time and cost prohibitive.

    As for my compression ratio: 6.2 (160TB precompression 25.9TB post compression)

    That's just from one of my 5 data domains.
    Chad Small
    IBM Certified Deployment Professional
    chadsmal@gmail.com
    http://www.tsmadmin.com

  7. #31
    Member jbastl's Avatar
    Join Date
    Jun 2006
    Location
    Czech Republic
    Posts
    16
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default

    Ho Timgren

    Deduplication works well on databases. (Oracle, MS sql),
    if you are using only incremental backups of file systems, dedup benefits are litle bit better then compression.
    Some data are not deduplicateble ;p

    HW black box vtl with dedup is great, but very expensive. Data are deduplicated on disks only. On tapes thy are stored in full size.

    So - if you are planning to use deduplication (hw or sw) think about type of your data. before bying some dedup solution.

  8. #32
    Member
    Join Date
    Dec 2002
    Location
    St. Louis
    Posts
    47
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Wow. I started this thread 06-26-2007. I was shocked to login and see it still alive and well. Amazing.

    Let me do a quick follow up.
    We have two DD880's, and DD640's (i think that's the model) We get about 5:1 - 6:1 dedup rates with TSM 6.23, which is pretty typical. Were fiddling with TSM dedup, but haven't really spent much time with it. Initial tests haven't been impressive enough to invest a whole lot into it. It works ...but you get what you pay for.

    Based the original pitch, dedup certainly isn't the "golden child" vendors liked to claim regardless of the creative marketing TCO they presented. We have yet to "save" anything other then sq ft in the data center. In fact, If we offloaded the entire DD880 to raw SATA drives with NO compression whatsoever, we'd still come out ahead - money wise. The "savings" come down to less spinning physical disk (I see this as a vendor benefit, not a customer), less data center space, less power costs, and ease of management. Actual storage costs are close to a wash using disk storage. (i.e. 5:1 dd rates.... at 5x the total cost) If evaluated against LTO5's, tape will win every time, hands down, no contest.

    The DD880 is VERY easy to use, very stable, and has performed very well for us. I have no real complaints about the product or the support. It makes managing TSM storage a breeze. This of course has value too - but it's not value that a financial auditor will ever realize.

    The problems we realized early on using the DD880 for direct rman backups was that it presented the DBA's with then entire 70TB filesystem, and no real way to limit to it's use to a set quota. I don't know about your shop... but here... when DBA's see storage -- they fill it up completely! And ONLY when it's absolutely full and unusable do they figure out how to clean it out or expire old data-sets. So unless DBA's were provided with their very own DD880 - direct DB backups on a system shared by TSM isn't an option. I've also found no real value in doing the recommended "full rman backups" daily just so the dedup ratio increases. I find that this number has NO value in a fixed cost environment. Sending 7x the data instead of 2x the data to dedupped storage appliance doesn't "save" anything, nor does it add any additional protection.

    We're currently evaluating the Quantum DXi8500 which promises the same capacity at about 1/2 the cost and with a more reasonable annual maintenance fee AND a VM solution that looks promising. Of course.. when it comes to vendor claims... I'll believe it when i see it.

  9. #33

    Default

    My whole enviorment is De-Dup. I am getting almost 70% compression and dedup ratio. I have roughly about 420 Nodes. It has been working great. I had some issue. TSM 6.3.3 resolved some of the issue. You need to upgrade to TSM 6.3.3.100

    Thanks,

Page 2 of 2 FirstFirst 12

Similar Threads

  1. Replies: 2
    Last Post: 02-04-2007, 08:20 AM
  2. Difference between "move data" & "move nodedata"?
    By cheffern in forum Backup / Archive Discussion
    Replies: 2
    Last Post: 12-11-2006, 09:30 AM
  3. TDP for SQL: Backups changing DB recovery mode from "normal" to "simple"?
    By c.j.hund in forum TDP/Application Layer Backup
    Replies: 1
    Last Post: 09-12-2006, 04:29 AM
  4. accidental "delete filespace" and "remove node"
    By rogwall in forum Administrative Client
    Replies: 2
    Last Post: 05-09-2006, 02:57 PM
  5. Replies: 1
    Last Post: 11-19-2003, 10:05 AM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •