Hi Wanda,
I'm using Deduplication and have found that tsm life would be much easier if
the stg pool was kept smaller under 3TB in size. I haven't done enough testing
with this, and I know it is slightly counterproductive to achieve the highest
deduplication savings. But it sure does make the administrative side much
cleaner and easier to work. Keeping the storage pool smaller does create
offsite copies and reclamation faster. It is a divide and conquer.
I do have a storage pool that does take longer to reclaim, and this seems to
clear up every Sunday as we generally have very little incoming client backups
on that day. I'm using client side deduplication to leverage the client
processing; and In order to protect against stale or duplicate chunk collisions
I keep the cache databases set very low where the clients on average have to
reset that cache every few days.
Deduplication is very important for me, Please keep me in the loop when you
come closer to a resolution.
tsm: TSMC04P>SHOW DEDUPDELETEINFO
****Dedup Deletion General Status****
Number of worker threads : 8
Number of active worker threads : 1
Number of chunks waiting in queue : 1967534
tsm: TSMC04P>q db f=d
Database Name: TSMDB1
Total Size of File System (MB): 1,148,760
Space Used by Database(MB): 304,105
Free Space Available (MB): 6,755,930
Total Pages: 33,625,117
Usable Pages: 33,623,517
Used Pages: 33,620,309
Free Pages: 3,208
Buffer Pool Hit Ratio: 98.0
Total Buffer Requests: 21,032,020,059
Sort Overflows: 0
Package Cache Hit Ratio: 99.8
Last Database Reorganization: 12/16/2013 18:21:53
Full Device Class Name: 3592DEV
Number of Database Backup Streams: 1
Incrementals Since Last Full: 0
Last Complete Backup Date/Time: 12/19/2013 10:12:53
-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of
Erwann Simon
Sent: Friday, December 20, 2013 12:33 AM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] Deduplication "number of chunks waiting in queue"
continues to rise?
Hi Wanda,
Expire Inventory is queuing chunk for deletion.
See the Q PR output when, at the end of the expire inventory process, the total
numbers of nodes have been reached. No more deletion of objects occurs, but
SHOW DEDUPDELETEINFO shows that the deletion threads are still working, queuing
and deleting chunks. This activity does not appear "externally" and consumes
most of the expire inventory time.
Let's try with deduplication disabled (dedup=no) for that pool (?).
Regards,
Erwann
"Prather, Wanda" <Wanda.Prather AT ICFI DOT COM> a écrit :
>TSM 6.3.4.00 on Win2K8
>Perhaps some of you that have dealt with the dedup "chunking" problem
>can enlighten me.
>TSM/VE backs up to a dedup file pool, about 4 TB of changed blocks per
>day
>
>I currently have more than 2 TB (yep, terabytes) of volumes in that
>file pool that will not reclaim.
>We were told by support that when you do:
>
>SHOW DEDUPDELETEINFO
>That the "number of chunks waiting in queue" has to go to zero for
>those volumes to reclaim.
>
>(I know that there is a fix at 6.3.4.200 to improve the chunking
>process, but that has been APARed, and waiting on 6.3.4.300.)
>
>I have shut down IDENTIFY DUPLICATES and reclamation for this pool.
>There are no clients writing into the pool, we have redirected backups
>to a non-dedup pool for now to try and get this cleared up.
>There is no client-side dedup here, only server side.
>I've also set deduprequiresbackup to NO for now, although I hate doing
>that, to make sure that doesn't' interfere with the reclaim process.
>
>But SHOW DEDUPDELETEINFO shows that the "number of chunks waiting in
>queue" is *still* increasing.
>So, WHAT is putting stuff on that dedup delete queue?
>And how do I ever gain ground?
>
>W
>
>
>
>**Please note new office phone:
>Wanda Prather | Senior Technical Specialist | Wanda.Prather AT icfi DOT com
> | www.icfi.com
>ICF International | 443-718-4900 (o)
--
Erwann SIMON
Envoyé de mon téléphone Android avec K-9 Mail. Excusez la brièveté.
|
