Not really sure why you would need +24 hours to handle copying data to your 
secondary site.

With features like:

- Copy storage pool feature for client backups
- Copy storage pool feature on migration
- Normal backup storage pool

With the first 2, there shouldnt really be much data left to copy for the 3rd 
one. I've seen sites handling up to 35TB per day using these features and they 
never had to spend 24 hours doing backup storagepool.

Ofc, it also depends on the throughput your getting. There are VTL's out there 
that can handle up to 45TB/hour that costs alot less than 1 million. However, 
it does require the infrastructure (fiber) connections to reach those numbers.

As with the hash conflict, the DD uses SHA-1 with a variable block length for 
deduplication. Theoretically, there is a 2^160 chance it will happen. Doesnt 
seem to be that bad, but your first hash collision is randomly more likely to 
happen than that number suggests.

Best Regards

Daniel


Daniel Sparrman
Exist i Stockholm AB
Växel: 08-754 98 00
Fax: 08-754 97 30
daniel.sparrman AT exist DOT se
http://www.existgruppen.se
Posthusgatan 1 761 30 NORRTÄLJE


-----"ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> skrev: ----- 
Till: ADSM-L AT VM.MARIST DOT EDU
Från: Shawn Drew 
Sänt av: "ADSM: Dist Stor Manager" 
Datum: 10/05/2011 00:14
Ärende: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: 
Re: [ADSM-L] Ang: Re: [ADSM-L] vtl versus file systems for pirmary pool

We were using the copystg feature of the storage pools before, and even 
then our TSM cycle was growing, growing and eventually passed 24 hours. 
For us, it wasn't 4 hours vs 1 hour.  It was 24+hours+finger crossing vs 
one hour.  We reached the point where we had to kill the backup-stg's and 
try and catch up on the weekends.

As far as the hash conflict, I think all of these systems use 
cryptographic level hashes (MD5/SHA1). From my understanding, the chances 
of a collision are way, way, exponentially lower than .1%.  Where did that 
number come from, Is that a commonly accepted number for this issue? Maybe 
I'm not understanding the proposed error. Are you referring to a 
cryptographic hash collision? or something else? 

In order to update our environment to handle the traditional TSM copypool 
architecture, we would have had to spend an additional million dollars (at 
a minimum) and many more expensive WAN links.   Considering this is for 
short-term data (30 days or less) , we still have weekly tape backups, and 
I've never seen anything to convince me that the chances of these specific 
errors are anywhere near the quoted .1%,  I'm still confident this was the 
right move for us. 

I do keep looking for where I could be wrong, so I want to make sure I 
completely understand what you are saying.

Regards, 
Shawn
________________________________________________
Shawn Drew





Internet
daniel.sparrman AT EXIST DOT SE

Sent by: ADSM-L AT VM.MARIST DOT EDU
10/04/2011 04:44 PM
Please respond to
ADSM-L AT VM.MARIST DOT EDU


To
ADSM-L
cc

Subject
[ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: 
[ADSM-L] Ang: Re: [ADSM-L] vtl versus file systems for pirmary pool






Not entirely true since you dont have todo backup stgpool as a only resort 
with TSM 6.2. Since the simultaneous copy feature is now not only 
available for client, but for server processes, there is nothing saying 
you have to copy all the data by using backup stgpool.

And it's not a matter if it's a risk or not getting a logical error. The 
risk is there, and if it hits you, you dont loose a few hours of data. A 
hash conflict would strike you across both primary and secondary storage, 
and it could strike you for a huge amount of data if you're unlucky. You 
get a hash conflict on a common hash key, and it could strike you all 
across the board.

So, having your offsite replicated within the hour instead of 4 hours with 
the risk of loosing most of it, or letting it take 4 hours, but be sure 
you'll be able to recover?

The descriptions around here sometimes scares me (not in particular this 
description). Some people seem to think that 0.1% chance of loosing it all 
is worth it for other benefits.

If you told your boss (not your IT manager, but your business developer 
for example) that there's a 0.1% chance you'll loose all your backups and 
all your versions, think he'd be ok with it? My guess? I'd say he wouldnt 
even answer it...

If you feel replication is such a good way of going, you can always go 
sync or async mirroring instead. You're doing just the same (since you can 
actually have verification on your mirroring). You're syncing/replicating 
bits and bytes, but there's no way for the mirroring / replication to be 
sure it's actually readable for the app. Unlike having your app creating a 
2ndary copy.

Best Regards & nice evening

Daniel


Daniel Sparrman
Exist i Stockholm AB
Växel: 08-754 98 00
Fax: 08-754 97 30
daniel.sparrman AT exist DOT se
http://www.existgruppen.se
Posthusgatan 1 761 30 NORRTÄLJE


-----"ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> skrev: ----- 
Till: ADSM-L AT VM.MARIST DOT EDU
Från: Shawn Drew 
Sänt av: "ADSM: Dist Stor Manager" 
Datum: 10/04/2011 20:17
Ärende: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] 
Ang: Re: [ADSM-L] vtl versus file systems for pirmary pool

The other side of this is that with 3rd party deduplicated replication, my 

offsite copy is rarely more than one hour behind the source copy.  Before 
I moved to this, we would schedule our backup-stg pools to run once a day, 

and they would have to run several hours before it was in sync.   If there 

was a real DR situation, we would lose much more data by being up to 
24-hours out of sync with the old "backup-stg" solution.   And it's always 

recent "just backed-up" data that is the most desirable after a DR 
situation. 

 The chances of a real DR situation seem higher to me than a logical 
error.  Perhaps just feeling like that after seeing a minor earthquake and 

a hurricane within a couple weeks of each other. 

 I have seen a few logical errors and a few DR situations in my career and 

even in my conservative bank environment, a 
single-pool-3rd-party-replicated is lower risk than the slow-backup-stg 
solution.  We still keep periodic longer term backups on Tape, so we would 

have a "last resort" restore source if there was a logical error, but it 
is still not a daily backup.

In the end, its a risk trade-off decision that you have to make for 
yourself.  And decide if you can afford the time that a backup-stg takes 
each day. 


Regards, 
Shawn
________________________________________________
Shawn Drew





Internet
steven.langdale AT GMAIL DOT COM

Sent by: ADSM-L AT VM.MARIST DOT EDU
10/04/2011 02:57 AM
Please respond to
ADSM-L AT VM.MARIST DOT EDU


To
ADSM-L
cc

Subject
Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: 
Re: [ADSM-L] vtl versus file systems for pirmary pool






The "logical error" question has come up before.  With no TSM managed copy
pool you are perhaps at a slightly higher risk.

An option is to still have a copy pool, but on the same DD.  So little 
real
disk usage, but some protection from a TSM logical error.  That obviously
does not protect you from a DD induced one.

FWIW, when we implement VTL's, and if the bandwidth allows, we use TSM to
create the copy.  Small sites with limited bandwidth, we rely on the
appliance.

Steven



On 4 October 2011 06:41, Daniel Sparrman <daniel.sparrman AT exist DOT se> 
wrote:

> > If someone puts a high-caliber bullet through my Gainesville DD, then
> > I recover it from the replicated offsite DD, perhaps selecting a
> snapshot.
> >
> > If someone puts a high-caliber bullet through both of them, then I
> > have lost my backups of a bunch of important databases.
>
> And if you have a logical error on your primary box, which is then
> replicated to your 2nd box? Or even worse, a hash conflict?
>
> I dont consider someone putting a bullet through both the boxes a high
> risk, I do however consider other errors to be more of a high risk.
>
> Best Regards
>
> Daniel
>
>
>
>
>
> Daniel Sparrman
> Exist i Stockholm AB
> Växel: 08-754 98 00
> Fax: 08-754 97 30
> daniel.sparrman AT exist DOT se
> http://www.existgruppen.se
> Posthusgatan 1 761 30 NORRTÄLJE
>
>
> -----"ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> skrev: -----
> Till: ADSM-L AT VM.MARIST DOT EDU
> Från: "Allen S. Rout"
> Sänt av: "ADSM: Dist Stor Manager"
> Datum: 10/03/2011 23:38
> Ärende: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: [ADSM-L] Ang: Re: 
[ADSM-L]
> vtl versus file systems for pirmary pool
>
> On 09/28/2011 02:16 AM, Daniel Sparrman wrote:
>
> > In this mail, it really sounds like you're using your DD as both
> > primary storage and for TSM storage.
>
> I am, right now, using the DD as a target for direct-written database
> backups, only.  So that's not really "primary storage", as I think
> about it.
>
>
> > If the DD box fails, what are your losses?
>
> If someone puts a high-caliber bullet through my Gainesville DD, then
> I recover it from the replicated offsite DD, perhaps selecting a 
snapshot.
>
> If someone puts a high-caliber bullet through both of them, then I
> have lost my backups of a bunch of important databases.
>
>
>
> > Sorry for all the questions, I'm just trying to get an idea how
> > you're using this box.
>
> No problem. Our conversation is fuzzed by the fact that I am also
> talking about how one _might_ use it for TSM storage.  I'm
> contemplating it, but not doing it at the moment.
>
> > [ ... if you lose a DD, then ... ] you have to restore the data from
> > somewhere else (tape?).
>
>
> In my planning, the DD gets copied / offsited to a remote DD, so
> that's the somewhere else.
>
> - Allen S. Rout
>



This message and any attachments (the "message") is intended solely for 
the addressees and is confidential. If you receive this message in error, 
please delete it and immediately notify the sender. Any use not in accord 
with its purpose, any dissemination or disclosure, either whole or 
partial, 
is prohibited except formal approval. The internet can not guarantee the 
integrity of this message. BNP PARIBAS (and its subsidiaries) shall (will) 

not therefore be liable for the message if modified. Please note that 
certain 
functions and services for BNP Paribas may be performed by BNP Paribas 
RCC, Inc.


This message and any attachments (the "message") is intended solely for 
the addressees and is confidential. If you receive this message in error, 
please delete it and immediately notify the sender. Any use not in accord 
with its purpose, any dissemination or disclosure, either whole or partial, 
is prohibited except formal approval. The internet can not guarantee the 
integrity of this message. BNP PARIBAS (and its subsidiaries) shall (will) 
not therefore be liable for the message if modified. Please note that certain 
functions and services for BNP Paribas may be performed by BNP Paribas RCC, Inc.