Results 1 to 16 of 16
  1. #1
    Member
    Join Date
    May 2007
    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Disk to disk backups on TSM + compression and dedup advice.

    Hello All,

    As a proof of concept I need to make some modifications to our TSM setup so some selected clients are backed up to disk (primary pool).

    Currently our primary pool is a VTL with dedup so that pool is set up as a standard library, but since the license price of additional capacity is absurd we are thinking of backing up straight to disk and put in either compression or deduplication.

    As a backend for this disk pool we will be using one or more HDS AMS2100's with either SATA or SAS disks (dependant on the performance I get from my proof of concept).

    Our server is running on Solaris and has a Veritas Enterprise installed. What would be the best way to plan this setup ?

    • Devclass file or disk ? What is the optimum size ?
    • For devclass disk, do I use the RAW device given to the OS or would I be better of creating volumes in Veritas and using the raw devices from there ?
    • Any other recommendations or experiences ?

    I'm a bit rusty concerning devclasses , mgt classes ets so a little help with the setup would be appreciated

    Also on this disk based backup, what would you recommend regarding compression or dedup ? Is it for ex. better to compress a database or dedupe it ? What about filesystems ? Etc etc ... what are your experiences with both ?

    Thanks a lot all !

    Maxx

  2. #2
    Newcomer
    Join Date
    Sep 2010
    Posts
    50
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Hi Maxx

    Starting with compressions : for databases allow the database to compress itself for ex .RMAN, this removes all white space at the client before it gets to the TSM server, same with file backups allow the client to compress before transmit.

    If using TSM 5.5 or lower raw devices will give you better performance. For the devclass for your disk are you intending to allow multiple clients backup in parallel or sequential ?

  3. #3
    Member
    Join Date
    May 2007
    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Hello BPHealy !

    Thx for your feedback ... Since we will be upgrading to 6.2 very soon I would be better off using a devclass of file then I guess. What is my best best then ?
    - 1 file per storage lun of 100 Gb
    - 10 files of 100 Gb per lun of 1Tb ?
    I am guessing the first would perform slightly better sine it can allocate more SCSI queue_depths

    We currently have about 200 Clients (Windows, SUN, AiX + TDP's for Exchange, MSSQL, Oracle, Informix, ...), so yes, during the backup windows we always have multiple client backups running.

    On the storage box I would use for the proof of concept I would be using dedicated storage space in fixed raidgroups. On the AMS2100 box I would dedicate for the disk backups I will be using Dynamic Provsioning with striping over multiple raidgroups in order to optimise performance.

  4. #4
    Senior Member DanGiles's Avatar
    Join Date
    Oct 2002
    Location
    Toronto, Ont. Canada
    Posts
    545
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Setting the devclass to FILE or DISK will depend on the clients. If you have lots of smaller clients, random access could be better. You probably won't get much deduplication out of them either. For large backups, the sequential nature of FILE could server you better, and since these get periodic full backups, they could dedup fairly well. If you won't have much disk and plan to migrate the data to tape after just a couple of days, I wouldn't bother with dedup at all!

    Either way, I probably wouldn't want to throw all the clients into a single storage pool, as restores would probably suck big time.

    Note I use 'could' and 'probably' -- you'll have to test on your own data!
    Daniel R. Giles
    Sileg Consulting Inc.
    416.402.9744 dan.giles@sileg.com
    http://www.linkedin.com/in/sileg

  5. #5
    Member
    Join Date
    May 2007
    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    The goal would be to use the DISK or FILE pool as a permanent primary storage pool, replacing over time our VTL. Our secondary pool remains on physical tapes as is. Data will only be removed as it gets expired.

    Our current setup holds 280 Nodes, but we still need to add the online database and SAP backups of our new environment, so I am expecting approx. 320 Nodes.
    This represents 300 Tb of total data stored in TSM (both primary and secondary pool), but we expect this to grow exponentially to a few Peta bytes in a very near future as soon as we will pump data in our SAP databases. We would therefore rely heavily on dedup (as we do now on our VTL also).
    Most of our data comes from Informix and Oracle databases or from SAP databases on Oracle, so I am expecting this to dedup fairly well (on our VTL we have a dedup rate of 6.. --> 6.8 that is

    We do use several pools currently and plan to continue to do so (DATABASE pool, UNIX pool, WINTEL pool, MAIL pool, WORM pool + all respective secondary pools).

    Any suggestions or remarks are always welcome !

  6. #6
    Senior Member DanGiles's Avatar
    Join Date
    Oct 2002
    Location
    Toronto, Ont. Canada
    Posts
    545
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    If you are going to be using deduplication, you HAVE to use FILE device class. Note that TSM will not dedup across storage pools!

    Personally, I'd shy away at putting all that on disk storage pool, but before I could give any type of suggestion, I'd need to know how you plan to do DR. Are you planning to replicate electronically? I'd look at active data pools for most critical servers and use tape/truck for the rest.

    A couple of notes, however: your database backups should dedup fairly well, depending on how often you do full backups, and how many you keep. Transaction log, however, usually do not dedup that well. If you use (or are going to use) the new Windoze Office application, their files are 'compressed' already, so that will not compress much further, and probably will not dedup that well.
    Daniel R. Giles
    Sileg Consulting Inc.
    416.402.9744 dan.giles@sileg.com
    http://www.linkedin.com/in/sileg

  7. #7
    Senior Member Jeff_Jeske's Avatar
    Join Date
    Jul 2006
    Location
    Stevens Point, WI
    Posts
    485
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default

    We have 4 rather large TSM servers attached to file device class storage. We have 500GB front end diskpool that migrates to a 20TB primary file pool. We kept the size down to 20TB because for us that is the point that the TSM database hits 80GB. Our file pool volume size is 50GB and we run aggressive reclamation on the file pools.

    We are using client side compression on all SATA clients and it works well. The only issue is that some under powered VM nodes will spike to 100% utilization during the backup.

    Another thing we learned is that you need to retain a blend of large files and small files. At first we sent all of our large DB backup files to physical tape and all system files, web server files, and file servers to SATA. This caused the TSM DB to grow quickly without using much storage. We no longer separate file types and seem to get more miles out of our DB before it can be considered large.

    Also.... remember that moving to SATA provides the OS access to both the primary and copy pool volumes. If something unusual is encountered like a virus, click happy unqualified admin, or data corruption ... it can whack both copies.

    Jeff Jeske - Sentry Insurance Storage Engineer
    IBM Certified Advanced Deployment Professional
    SNIA Certified Storage Engineer
    EMC Proven Professional
    ITIL Certified

    Sometimes the more you know... the harder it is to make a good decision!

  8. #8
    Senior Member DanGiles's Avatar
    Join Date
    Oct 2002
    Location
    Toronto, Ont. Canada
    Posts
    545
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Jeff. SATA = Serial ATA. How does the type of disk play into things? What difference does it make to TSM whether the disk is SATA, SAS, or fibre? I do not understand your last paragraph.
    Daniel R. Giles
    Sileg Consulting Inc.
    416.402.9744 dan.giles@sileg.com
    http://www.linkedin.com/in/sileg

  9. #9
    Senior Member Jeff_Jeske's Avatar
    Join Date
    Jul 2006
    Location
    Stevens Point, WI
    Posts
    485
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default

    SATA isn't as good at random reads and writes as FC. 15K RPM FC disks are about 2x as fast on random I/O (read or write). Throughput per drive on sata is only near FC on linear reads, which is a very rare scenario in any shared environment.

    SATA is very good at streaming linear data so it works great as a landing platform for large files or migrations from a diskpool. If you are currently using physical tape or VTL windows doesn't "SEE" the data. It only sees the tapes and drives. With a file device class it will be able to see and touch the volumes (data). If I went in to windows explorer I could simply right click and delete both the pri and copy pool data.
    Last edited by Jeff_Jeske; 11-15-2010 at 11:19 AM.

    Jeff Jeske - Sentry Insurance Storage Engineer
    IBM Certified Advanced Deployment Professional
    SNIA Certified Storage Engineer
    EMC Proven Professional
    ITIL Certified

    Sometimes the more you know... the harder it is to make a good decision!

  10. #10
    Senior Member DanGiles's Avatar
    Join Date
    Oct 2002
    Location
    Toronto, Ont. Canada
    Posts
    545
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Ah, but you can see the TSM volumes through the OS on DISK volumes as well, and it doesn't matter the type of disk. But, we're getting off the topic of Maxx's original question...
    Daniel R. Giles
    Sileg Consulting Inc.
    416.402.9744 dan.giles@sileg.com
    http://www.linkedin.com/in/sileg

  11. #11
    Senior Member Jeff_Jeske's Avatar
    Join Date
    Jul 2006
    Location
    Stevens Point, WI
    Posts
    485
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default

    I agree.... and that's my point disk is more dangerous or accident prone than tape. It's just something to keep in mind.

    Jeff Jeske - Sentry Insurance Storage Engineer
    IBM Certified Advanced Deployment Professional
    SNIA Certified Storage Engineer
    EMC Proven Professional
    ITIL Certified

    Sometimes the more you know... the harder it is to make a good decision!

  12. #12
    Member
    Join Date
    May 2007
    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Hello All !

    Sorry for my late reply to this thread but I notice all of you have been quite busy ever since.
    I am aware disk is more error-prone that Tape, that is why we use a physical library in a remote location for data protection. The disk-based backups will be running of an HDS AMS2x00 series in a Raid6 setup, so normally the risk should be neglectable (we use this type of boxes on our Tier class 3 and 4 clients also for OS and data volumes).
    When used as the device holding our primary backup pool, we would configure it as a dynamic pool allowing us to stripe the data over multiple RAID6 raid groups. As mentionned before, we are still hesitant about moving to SATA 2Tb disks as our current VTL is running on FC15k disks, so this might prove slower then before, and since our database volumes are growing exponentially, our aim is to reduce the backup frame. Chances are I can convince management to fill the box with SAS15k 600 Gb drives; but of course this all depends on the results of my proof of concept.

    Our type of date is mainly oracle : we are expecting about 30 Tb worth of online databases, The rest is just a few Tb of windows servers, a couple of Tb of Unix filespaces etc ...

    The retention of the above filespaces and databases will vary from 3 versions for offline backups to 180 days for Tier 1 online backups.

    I was not aware that dedup doesn't work on DISK devclass, thanks for the hint ! I will therefore favor the FILE devclass !

  13. #13
    Senior Member Jeff_Jeske's Avatar
    Join Date
    Jul 2006
    Location
    Stevens Point, WI
    Posts
    485
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default

    Are you sure TSM extended edition is really that expensive? Have a sit down with your IBM rep, tell them your concerns and intentions unless they can move on the licensing fees. Everything is negociable. Our IBM reps bend over backwards to keep us happy, often providing free licensing for some items to keep us happy.

    If you have an infrastructure inplace that is working fine, I really can't believe that a rip and replace solution will be any cheaper.

    Jeff Jeske - Sentry Insurance Storage Engineer
    IBM Certified Advanced Deployment Professional
    SNIA Certified Storage Engineer
    EMC Proven Professional
    ITIL Certified

    Sometimes the more you know... the harder it is to make a good decision!

  14. #14
    Member
    Join Date
    May 2007
    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    I'm not sure where TSM EE comes into place into this thread, but we do have a licensed EE setup (as we do have the licenses for all TDP's in use - Oracle, Exchange, Informix, SQL, Sharepoint, ...). As mentionned in the openingpost, the goal is to prove to management I can get better performance using direct to disk backups instead of our very expensive VTL which is at the end of it's capacity (it currently holds 22 Tb out of 30 Tb with a dedup factor of 6.8 --> +/- 150 Tb).

    We are currently in a major overhaul, where we will be moving from Solaris to AiX, from Informix to Oracle, add an SAP layer, etc etc .. adding in total 150 physical servers. Since this totally explodes our current VTL we are due to a replacement very soon.

    Basically we expect to cross the Pb of data backed up in a VERY near future, which is why we are looking into other and better ways to backup these volumes. First comes into play the medium we will store these backups on for the primary pool (remember our copy pool uses physical tapes). Afterwards the biggest servers database servers will be using either dedup at the client side, or if that doesn't pull it, a storage agent will be implemented (we now are using basic clients everywhere).

    Snapshots are not an option unfortunately; but we do use dataguard very intensly for our Oracle Databases ..... but now we are elaborating a bit too much also ... back to disk based backups

  15. #15
    Senior Member
    Join Date
    Nov 2002
    Posts
    516
    Thanks
    2
    Thanked 4 Times in 4 Posts

    Default

    My 2 Cents from a very similar setup
    Use disk for 80% of the files/databases and logs -> VTL -> Tape, or exhange the VTL for a
    fildevice class with dedupe, except for the fast chagning data like logfiles.
    Use stg agents for the large databases, send them directly to Tape use 2 or 3 tape drives
    per stg agent and speread the ful backups evely during the week. most large databases can do full backups even when producing logfiles at several gigabytes per minute
    Windows, does it come with a gui?

  16. #16
    Senior Member Jeff_Jeske's Avatar
    Join Date
    Jul 2006
    Location
    Stevens Point, WI
    Posts
    485
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default

    I guess I misunderstood your licensing concern in the original post. Sorry about that.

    Why aren't you streaming large DB backups to tape at your primary site? We have VTL, SATA, and physical tape.... no matter how many spindals we throw at backups they cannot compete with directly streaming to tape. We have all of our small file system files land in a diskpool and files of size larger than 5GB going directly to tape..... this provides fantastic performance.

    Jeff Jeske - Sentry Insurance Storage Engineer
    IBM Certified Advanced Deployment Professional
    SNIA Certified Storage Engineer
    EMC Proven Professional
    ITIL Certified

    Sometimes the more you know... the harder it is to make a good decision!

Similar Threads

  1. TSM database move from local disk to SAN disk
    By akilap in forum Performance Tuning
    Replies: 3
    Last Post: 01-23-2009, 05:30 PM
  2. Migrated TSM disk pool to new disk array PROBLEM
    By dexter1972 in forum TSM Server
    Replies: 2
    Last Post: 05-07-2008, 05:09 PM
  3. TSM DISK STGpool devclass type (file or disk)
    By rmarkarian in forum TSM Server
    Replies: 2
    Last Post: 02-06-2008, 10:36 PM
  4. TSM Database Backups on disk
    By khurram in forum Backup / Archive Discussion
    Replies: 2
    Last Post: 06-15-2005, 12:59 AM
  5. TSM - DISK Tuning for DB, LOG, DISK Storage Pools
    By smourylev in forum Performance Tuning
    Replies: 2
    Last Post: 02-23-2005, 01:15 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •