Not quite.
A checkpoint forces a break in the data stream; on tape for will get a EOF mark written; the tape will stop; the media server has a bit of a chat with the
Master Server, flushing out data about the files backed up in the file just closed, etc. That is how it knows if there is a failure and restart, what is still to back up.
If you are writing to a disk target you will again split the backup into another fragment I think. So if you say set a 32GB fragment size, the on-disk files
are split every 32GB and also I believe when a checkpoint is hit, every 15 minutes or whatever you configure.
As to effect on deduplication % I would not expect it to be huge, though it will force the software to start calculating hashes from the start of each new
file.
William D L Brown
From: veritas-bu-bounces AT mailman.eng.auburn DOT edu [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu]
On Behalf Of scott.george AT parker DOT com
Sent: 08 September 2011 12:19
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Dedupe ratios and checkpoint backups
I think the question was: where is that file located when it is created, and does it then get backed up by the job doing the backup itself?
From:
|
David Stanaway <david AT stanaway DOT net>
|
To:
|
veritas-bu AT mailman.eng.auburn DOT edu
|
Date:
|
09/07/2011 09:11 PM
|
Subject:
|
Re: [Veritas-bu] Dedupe ratios and checkpoint backups
|
Sent by:
|
veritas-bu-bounces AT mailman.eng.auburn DOT edu
|
Checkpoints create a new file for each checkpoint.
On 9/7/2011 1:17 PM, Rusty Major wrote:
I have always understood it that Checkpoints were saved in a log on the client and, therefore, wouldn’t affect dedupe ratios at all. I haven’t ever verified that, nor did a quick
search yield anything.
-Rusty
From:
veritas-bu-bounces AT mailman.eng.auburn DOT edu [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu]
On Behalf Of Lightner, Jeff
Sent: Tuesday, September 06, 2011 3:23 PM
To: scott.george AT parker DOT com;
veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Dedupe ratios and checkpoint backups
Haven’t done any comparisons but we use checkpoint for our big ERP DB (near 6 TB) backup and still get good compression ratios. There is nothing that has made me think I need to
look at it or tweak it to get better. I’d say the benefit of not having to restart a huge backup from scratch offered by checkpoints would outweigh deduplication ratio issues unless you have infinite time to run backups.
From:
veritas-bu-bounces AT mailman.eng.auburn DOT edu [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu]
On Behalf Of scott.george AT parker DOT com
Sent: Tuesday, September 06, 2011 3:06 PM
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Dedupe ratios and checkpoint backups
To answer your question, we tested both the NBU 5000 (Symantec) and the DD860 (Data Domain) and the differences couldn't be more stark. I was using a 30 minute checkpoint interval. I never achieved
anything better than 15:1 from the NBU5000 (which isn't bad). The DD860 hit 39:1 at the time I disabled all of the policies. This was running daily full backups on an array of different servers (DB2, Windows file servers, UNIX file servers, and Siebel app
servers) over the course of 2 months.
The NBU5000 is a fixed block device, the DD860 a variable block device. There is no way of knowing if checkpoints were the culprit for the NBU5000 getting the lesser ratio, but it does present a plausible theory.
If at all, this would probably affect fixed-block solutions more than the variable-block ones. The variable-blocked solutions would continue to look for identical blocks, but in different positions of the data stream.
Just wondering if anyone has done testing or seen documentation (from any dedupe vendor) regarding the usage of enabling checkpoints on backups that are being deduplicated? I would think that the introduction of checkpoints every X minutes into the datastream
would interrupt the continuity of the data and make it seem more unique thus negatively affecting dedupe ratios but I’m wondering by how much. Most, if not all, of the variable length guys have the ability to ‘re-align’ themselves to the start of the files
so I would think it might be more pronounced on large files vs your average server but I’m just thinking out loud.
Anyone seen a recommendation or actually tested themselves?
_______________________________________________________
Barclays
www.barclaycardus.com
_______________________________________________________
This e-mail and any files transmitted with it may contain confidential and/or proprietary information. It is intended solely for the use of the individual or entity who is the intended recipient. Unauthorized use of this information is prohibited. If you have
received this in error, please contact the sender by replying to this message and delete this material from any system it may be on.
_______________________________________________
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu_______________________________________________
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Proud partner. Susan G. Komen for the Cure.
Please consider our environment before printing this e-mail or attachments.
----------------------------------
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential information and is for the sole use of the intended recipient(s). If you are not the intended recipient, any disclosure, copying, distribution, or use of the contents of this information
is prohibited and may be unlawful. If you have received this electronic transmission in error, please reply immediately to the sender that you have received the message in error, and delete it. Thank you.
----------------------------------
_______________________________________________
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
_______________________________________________
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu