|
Re: [Networker] ZFS deduplication.
2010-03-18 15:34:49
I have recently made similar tests to see what dedup ratios I can get
when writing to dedup enabled ZFS filesystems as a B2D target.
I have discovered the same behavior, e.g. NetWorker saveset streams do
not really get deduplicated. Changing the underlying ZFS dedup blocksize
doesn't have any positiv effect, even if I go down to 8k blocksize.
The only explanation I can think of is, as Yaron already pointed out,
block boundaries are aligned differently because of the different
meta-data which is part of the savesets.
It seems, at current point of the code implementation the dedup can not
deal with different block bounderies and as such, not effectively usable
for deduplicating such streams.
BTW I experienced the same thing when using NetBackup as the backup
application.
regards -attila
********************************************************************
Attila Mester 5 Digit Sun internal: x62534
Data Protection Architect Tel: (+49 89) 46 008 2534
Sun Microsystems GmbH Fax: (+49 89) 46 008 2583
Sonnenallee 1 Mobil: +49 172 812 5947
85551 Heimstetten / Germany mail: attila.mester AT sun DOT com
********************************************************************
Terry Lemons schrieb:
Hi Yaron
I know that NetWorker has a default block size for each of its output devices.
This default block size can be overridden; details on this are in the NetWorker
Administration Guide.
Does ZFS have a specific block size that it deduplicates? If so, could it be
that the NetWorker default block size for the AFTD device is not the same as,
or a multiple of, the ZFS default block size?
tl
-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On
Behalf Of Yaron Zabary
Sent: Sunday, March 14, 2010 3:59 PM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: [Networker] ZFS deduplication.
A few days ago I read a post to EMC's Networker forum by Nicholas
Bone (https://community.emc.com/thread/99839?tstart=0). He reported a
test he performed with AFTD which was running on top of ZFS with dedup.
Unfortunately, he wasn't able to get any reasonable dedup ratios (1.03
for three full savesets of the the same file system). My conclusion was
that Networker does not align files at block level, which confuses the
ZFS dedup code. Is anyone familiar with some flag or any configuration
option which will convince save or AFTD to do the right thing so that
ZFS will be able to find identical blocks ?
To sign off this list, send email to listserv AT listserv.temple DOT edu and type
"signoff networker" in the body of the email. Please write to networker-request
AT listserv.temple DOT edu if you have any problems with this list. You can access the
archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
|
|
|
