Veritas-bu

Re: [Veritas-bu] Tapeless backup environments

2007-10-26 12:10:18
Subject: Re: [Veritas-bu] Tapeless backup environments
From: "Curtis Preston" <cpreston AT glasshouse DOT com>
To: <Mark.Donaldson AT cexp DOT com>, <Len.Boyle AT sas DOT com>, <veritas-bu AT mailman.eng.auburn DOT edu>
Date: Fri, 26 Oct 2007 11:53:17 -0400
Hmmm..  I need to look into this further.  I could have sworn that it
stores a checksum per file backed up, and that it used that checksum
when it restored the file to see if the restored file is the same as the
backup.

I wonder if we can get an authoritative answer on this from a Symantec
lurker.

---
W. Curtis Preston
Backup Blog @ www.backupcentral.com
VP Data Protection, GlassHouse Technologies 

-----Original Message-----
From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of
Mark.Donaldson AT cexp DOT com
Sent: Thursday, October 25, 2007 12:26 PM
To: Len.Boyle AT sas DOT com; veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Tapeless backup environments

Nope - I don't think Netbackup is making checksums.

Tape hardware seems to be reasonably adept at detecting big tape errors,
though.  This, of course, goes away with disk based backups.

bpverify is just a check of the tape contents vs the media catalog.  It
does read the tape blocks so it may allow the drive to detect a media
error but it's not a verification of the block integrity vs some stored
checksum.


DESCRIPTION
     bpverify verifies the contents of one  or  more  backups  by
     reading  the backup volume and comparing its contents to the
     NetBackup catalog. This operation does not compare the  data
     on the volume with the contents of the client disk. However,
     it does read each block in the image,  thus  verifying  that
     the  volume  is readable. NetBackup verifies only one backup
     at a time and tries to minimize media mounts and positioning
     time.

-M

-----Original Message-----
From: Len Boyle [mailto:Len.Boyle AT sas DOT com] 
Sent: Monday, October 22, 2007 6:37 PM
To: Donaldson, Mark - Broomfield, CO; veritas-bu AT mailman.eng.auburn DOT edu
Subject: RE: Re: [Veritas-bu] Tapeless backup environments

Hello Mark,

Did I read in this list that netbackup was supposed to do some kind of
checksum on the data written to tape?
If so would a bpverify check this. I would assume that if netbackup does
this it would find the error.
because netbackup would do it's calc before passing the block to the
dedupe hardware/software. And the block that it gets back from the
dedupe hardware/software would be different.

Of course the brings on the question with the Symantec/Veritas pure disk
product or emc's as the netbackup and the dedupe parts are merged one
would not have this double check. At least I would not think that one
would.

len

-----Original Message-----
From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of
Mark.Donaldson AT cexp DOT com
Sent: Monday, October 22, 2007 4:52 PM
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Tapeless backup environments

I think that part of the problem is that a hash duplication is nearly
undetectable until you have restored and tested it as false.

We all know that 99.999% of what we back up is never restored.  It just
ages gracefully on media and is expired.  If any of that .001% is
restored and is damaged due to a tape fault (and we've all had it
happen) then we all know that we can usually reach back to a different
version or different tape and we'll be close enough to make the user go
away and let us return to our coffee and surfing.

I think a big part of the worry of a hash collision is that the restore
seems to happen, the file restores flawlessly, and it'll not be
detectable unless someone can checksum the whole file or it's a binary
or similar that simply refuses to work.

Again, restoring from a different tape, different version may be
ineffective depending on where the hash collision occurred and for what
reason.  Every version may use this same unchanging block which is
restore incorrectly due to an invalid hash match.

I know the odds are astronomical but I still remember that even though
the odds are 150 million to one I'll win the lottery, I still see
smiling faces on TV holding giant checks.

It's a bet, like all other restore techniques, and I'm going to make
sure management has full knowledge of the risks before we implement it
here (which is likely).

-M

-----Original Message-----
From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Jeff
Lightner
Sent: Monday, October 22, 2007 10:28 AM
To: Austin Murphy; veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Tapeless backup environments

This paper looks to be 5 years old (based on newest references it cites
- it actually cites others that go back nearly 10 years).  It would be
interesting to see his take on current deduplication offerings to see if
the other checks they contain over simple hashing were enough to allay
his concerns.

One thing I've not seen in all this discussion is anyone saying they've
actually experienced data loss as a result of commercial deduplication
devices.  Can anyone here claim that?

-----Original Message-----
From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Austin
Murphy
Sent: Monday, October 22, 2007 10:47 AM
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Tapeless backup environments

Here is some required reading on the topic from Val Henson, a noted
academic/storage-guru.

An Analysis of Compare-by-hash
www.nmt.edu/~val/review/hash.pdf

Of particular interst is why hardware error rates can't be compared
with deterministic software errors.

Austin
_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
----------------------------------
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or
confidential information and is for the sole use of the intended
recipient(s). If you are not the intended recipient, any disclosure,
copying, distribution, or use of the contents of this information is
prohibited and may be unlawful. If you have received this electronic
transmission in error, please reply immediately to the sender that you
have received the message in error, and delete it. Thank you.
----------------------------------

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu