Sergio and Wanda,
Thanks for your posts!  I opened PMR 10702,L6Q,000 a couple weeks ago
for slow performance [recently completely fell off the cliff!] with our
SRV3 TSM
v6.3.4.200 service that *was* successfully doing client+server deduplication
for 72TB BackupDedup STGpool on NetApp FC [soon to be 3par] FC disks.

I did not previously know about this command...
SHow DEDUPDelinfo now shows >7M enqueued dedupdel chunks @ SRV3 TSM.
I just requested escalation to consider whether TSMv6.3.4.207 efix will
help us.
Thanks again... hoping to re-post with better performance results soon!

jim.owen AT yale DOT edu   (w#203.432.6693, c#203.494.9201, h#203.387.3030)

On 12/20/2013 10:38 AM, Sergio O. Fuentes wrote:

Wanda,

In trying to troubleshoot an unrelated performance PMR, IBM provided me
with an e-fix for the dedupdel bottleneck that it sounds like you're
experiencing.  They obviously will want to do their due-diligence on
whether or not this efix will help solve your problems, but it has proved
very useful in my environment.  They even had to compile a solaris e-fix
for me, cause it seems like I'm the only one running TSM on Solaris.  The
e-fix was very simple to install.

What you don't want to do is go to 6.3.4.2, unless they tell you to
because the e-fix is for that level (207).  Don't run on 6.3.4.2 for even
a minute. Only install it to get to the e-fix level.

Dedupdel gets populated by anything that deletes data from the stgpool,
I.e. move data, expire inv, delete filespace, move nodedata, etc.

We run client-side dedupe (which works pretty well, except when you run
into performance issues on the server) and so our identifies don't run
very long, if at all.  It might save you time to run client-side dedupe.

BTW, when I finally got this efix and TSM was able to catch-up with the
deletes and reclaims as it needed to, I got some serious space space back
in my TDP Dedup pool.  It went from 90% util to 60% util (with about 10TB
of total capacity).  What finally really got me before the fix was that I
had to delete a bunch of old TDP MSSQL filespaces and it just took forever
for TSM to catch up.  I have a few deletes to do now, and I'm a bit wary
because I don't want to hose my server again.

I would escalate with IBM support and have them supply you the e-fix.
6.3.4.3 I don't think is slated for release any time within the next few
days, and you'll just be struggling to deal with the performance issue.

HTH,
Sergio



On 12/19/13 11:35 PM, "Prather, Wanda" <Wanda.Prather AT ICFI DOT COM> wrote:

TSM 6.3.4.00 on Win2K8
Perhaps some of you that have dealt with the dedup "chunking" problem can
enlighten me.
TSM/VE backs up to a dedup file pool, about 4 TB of changed blocks per day

I currently have more than 2 TB (yep, terabytes)  of volumes in that file
pool that will not reclaim.
We were told by support that when you do:

SHOW DEDUPDELETEINFO
That the "number of chunks waiting in queue" has to go to zero for those
volumes to reclaim.

(I know that there is a fix at 6.3.4.200 to improve the chunking process,
but that has been APARed, and waiting on 6.3.4.300.)

I have shut down IDENTIFY DUPLICATES and reclamation for this pool.
There are no clients writing into the pool, we have redirected backups to
a non-dedup pool for now to try and get this cleared up.
There is no client-side dedup here, only server side.
I've also set deduprequiresbackup to NO for now, although I hate doing
that, to make sure that doesn't' interfere with the reclaim process.

But SHOW DEDUPDELETEINFO shows that the "number of chunks waiting in
queue" is *still* increasing.
So, WHAT is putting stuff on that dedup delete queue?
And how do I ever gain ground?

W



**Please note new office phone:
Wanda Prather  |  Senior Technical Specialist  | Wanda.Prather AT icfi DOT com
|  www.icfi.com
ICF International  | 443-718-4900 (o)