While we don't do deduplication (tests show we gain less than 25% from it),
we also split our DB2 instances across multiple, physically-separate
volumes. The one thing to note is that you have to dump and restore the
database to spread existing data across those directories if you add them
post-installation.

On Fri, Dec 20, 2013 at 02:23:34PM -0500, Marouf, Nick wrote:
> I can second that Sergio,
>  Backup stgpools to copy tapes is not pretty, and is an intensive process to 
> rehydrate all that data.
>
> The one extra thing I did was split the database across multiple folder for 
> parallel I/O to the Database. That has worked out very well, and I currently 
> have it setup to span across 8 folders, with an XIV backend that  can take a 
> beating.
>
>
> -----Original Message-----
> From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf 
> Of Sergio O. Fuentes
> Sent: Friday, December 20, 2013 12:04 PM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: Re: [ADSM-L] Deduplication "number of chunks waiting in queue" 
> continues to rise?
>
> Client-side dedup and simultaneous-write to a copy pool are mutually 
> exclusive.  You can't do both, which is the only theoretical way to enforce 
> deduprequiresbackup with client-side dedup.  I suppose IBM could enhance TSM 
> to do a simultaneous-like operation with client-side dedup, but that's not 
> available now.  So, I'm not sure how the TSM server enforces 
> deduprequiresbackup with client-side dedup.  Ever since 6.1 I have always set 
> that to NO anyway.  I have dealt with the repercussions of that as well.  
> Backup stgpool on dedup'd stgpools is not pretty.
>
> I have made some architectural changes to the underlying stgpools and the 
> 'backup stgpools' run pretty well, even with 1TB SATA drives.  Two things I 
> think helped quite a bit:
>
> 1.  Use big predefined volumes.  My new volumes are 50GB.
> 2.  Use many filesystems for the devclass.  I have 5 currently.  I would use 
> more if I had the space.
>
> Thanks!
>
> Sergio
>
>
> On 12/20/13 11:35 AM, "Prather, Wanda" <Wanda.Prather AT icfi DOT com> wrote:
>
> >Woo hoo!
> >That's great news.
> >Will open a ticket and escalate.
> >
> >Also looking at client-side dedup, but I have to do some architectural
> >planning, as all the data is coming from one client, the TSM VE data
> >mover, which is a vm.
> >
> >Re client-side dedup, do you know if there is any cooperation between
> >the client-side dedup and deduprequiresbackup on the server end?
> >I have assumed that the client-side dedup would not offer that protection.
> >
> >W
> >
> >-----Original Message-----
> >From: Sergio O. Fuentes [mailto:sfuentes AT umd DOT edu]
> >Sent: Friday, December 20, 2013 10:39 AM
> >To: ADSM: Dist Stor Manager
> >Cc: Prather, Wanda
> >Subject: Re: [ADSM-L] Deduplication "number of chunks waiting in queue"
> >continues to rise?
> >
> >Wanda,
> >
> >In trying to troubleshoot an unrelated performance PMR, IBM provided me
> >with an e-fix for the dedupdel bottleneck that it sounds like you're
> >experiencing.  They obviously will want to do their due-diligence on
> >whether or not this efix will help solve your problems, but it has
> >proved very useful in my environment.  They even had to compile a
> >solaris e-fix for me, cause it seems like I'm the only one running TSM
> >on Solaris.  The e-fix was very simple to install.
> >
> >What you don't want to do is go to 6.3.4.2, unless they tell you to
> >because the e-fix is for that level (207).  Don't run on 6.3.4.2 for
> >even a minute. Only install it to get to the e-fix level.
> >
> >Dedupdel gets populated by anything that deletes data from the stgpool,
> >I.e. move data, expire inv, delete filespace, move nodedata, etc.
> >
> >We run client-side dedupe (which works pretty well, except when you run
> >into performance issues on the server) and so our identifies don't run
> >very long, if at all.  It might save you time to run client-side dedupe.
> >
> >BTW, when I finally got this efix and TSM was able to catch-up with the
> >deletes and reclaims as it needed to, I got some serious space space
> >back in my TDP Dedup pool.  It went from 90% util to 60% util (with
> >about 10TB of total capacity).  What finally really got me before the
> >fix was that I had to delete a bunch of old TDP MSSQL filespaces and it
> >just took forever for TSM to catch up.  I have a few deletes to do now,
> >and I'm a bit wary because I don't want to hose my server again.
> >
> >I would escalate with IBM support and have them supply you the e-fix.
> >6.3.4.3 I don't think is slated for release any time within the next
> >few days, and you'll just be struggling to deal with the performance issue.
> >
> >HTH,
> >Sergio
> >
> >
> >
> >On 12/19/13 11:35 PM, "Prather, Wanda" <Wanda.Prather AT ICFI DOT COM> wrote:
> >
> >>TSM 6.3.4.00 on Win2K8
> >>Perhaps some of you that have dealt with the dedup "chunking" problem
> >>can enlighten me.
> >>TSM/VE backs up to a dedup file pool, about 4 TB of changed blocks per
> >>day
> >>
> >>I currently have more than 2 TB (yep, terabytes)  of volumes in that
> >>file pool that will not reclaim.
> >>We were told by support that when you do:
> >>
> >>SHOW DEDUPDELETEINFO
> >>That the "number of chunks waiting in queue" has to go to zero for
> >>those volumes to reclaim.
> >>
> >>(I know that there is a fix at 6.3.4.200 to improve the chunking
> >>process, but that has been APARed, and waiting on 6.3.4.300.)
> >>
> >>I have shut down IDENTIFY DUPLICATES and reclamation for this pool.
> >>There are no clients writing into the pool, we have redirected backups
> >>to a non-dedup pool for now to try and get this cleared up.
> >>There is no client-side dedup here, only server side.
> >>I've also set deduprequiresbackup to NO for now, although I hate doing
> >>that, to make sure that doesn't' interfere with the reclaim process.
> >>
> >>But SHOW DEDUPDELETEINFO shows that the "number of chunks waiting in
> >>queue" is *still* increasing.
> >>So, WHAT is putting stuff on that dedup delete queue?
> >>And how do I ever gain ground?
> >>
> >>W
> >>
> >>
> >>
> >>**Please note new office phone:
> >>Wanda Prather  |  Senior Technical Specialist  |
> >>Wanda.Prather AT icfi DOT com
> >>|  www.icfi.com
> >>ICF International  | 443-718-4900 (o)
> >
> >

--
-- Skylar Thompson (skylar2 AT u.washington DOT edu)
-- Genome Sciences Department, System Administrator
-- Foege Building S046, (206)-685-7354
-- University of Washington School of Medicine