Data Retention Best Practices

[mclawler]

I've been searching the web for a best practice document from someone on Data Retention Best Practices....and they seem to be all over the place, if I may ask, what is the general consensous for backup retention periods out in the 'real' world?

Daily?
Weekly?
Monthly?
Yearly?

Currently my company has what I consider to be an incredibly long retention period (daily 1 month, Weekly 2 months, monthly 1 year and yearly forever). What are you guys seeing/doing out therE?

 

[gimlet]

My experiences have mostly been along the lines of "We can only spend $X on backups. Try to keep as much as you can."

I'd advise against it.

 

[mclawler]

Luckily we're not running into that scenario, it's more of a 'are we keeping to much, and how do we prove it to the company' thing that we're looking for. Not that $$ savings isn't good, but if we need to spend more on backups we will. I'm lucky enough to have a manger that believes backups are one of our most important functions in IT.

 

[chad_small]

Depends on the company. When I was with Honeywell we had to keep some data as long as the airplane instruments were still in use (decades in most cases). I've worked some places where they need monthlies and yearlies (IMHO weeklies are a joke just keep deleted data longer). We just created a monthly and yearly node name for each server and had them perform an incremental (NOT A FULL) on the scheduled time. If you collocate these pools then its more cost effective and if you can dedupe...even better.

 

[mclawler]

I've been considering setting up several different retention groups for each node...

Daily - 7 day retention
Weekly - 31 day retention
Monthly - 1095 day retention (3 years - the company is dictating this)
Yearly - forever (the customers are dictating this, and the company is in agreement with it)

What I've been doing is 31 days worth of daily backups, then create a backupset of everything for long term storage (we're capacity licensed, and backupsets don't count against the capacity). What I'd like to do is setup 4 nodes per server (node_daily, node_weekly, node_monthly, node_yearly) and run an incremantal for each dedepending on schedules....everything would dump to the dedup_disk pool and get deduplicated (if its not doing client side dedupe alreadY) from there I'd get the advantages of deduplication against he data, no need to mess with backupsets (doing nodegroup backupsets of several hundred servers is a major PITA).

Anyone else doing incrementals like this? as always, your input is appreciated.

 

[rowl]

I am curious, have you considered using an archive instead of a backup for the 3 year and forever backups? Also, do you have any details on what data actually needs these longer retentions? Not sure on your platform, but do you need to keep the OS backups for 3+ years, or some subset of the data?

 

[mclawler]

We have considered the archives, but have not yet put them into play. I have orders from previous incarnations of backups to 'backup everything' including the c:\ drives...

With dedup however I'm not overly worried about the c:\ drives, it gets really good dedup rates since just about all of our c:\ drives are the same.

As for what data needs to be kept longer or shorter, we have tried to go back to the user community and request some info on what they feel needs to be kept for how long, and the standard answer is 'if I need it, I'll need it, so keep everything we do forever'....which is crazy, so we stopped asking them.

I have some leeway in changing our retention policies, but for the most part the yearly will always be forever (our customers can come back at us at any time to fight something, so we have to keep all of our records....gotta love teh Auto industry and recalls....).

So, to make it easy, we have to keep everything for these time frames, and we include the c:\ just for ease of system restores.

In the 8 years I've been here I've only been asked to do it 3 times, and i've only had to go back to the users once and tell them that it was outside our window...but that one time cost us quite a bit.

 

[domino]

Standard Policies >

Daily - 14 day retention -Node_0014_CO - domain PD_0014_CO
Weekly - 40 day retention -Node_0040_CO ...
Monthly - 400 day retention -Node_0400_CO ...
Yearly - 4000 day retention -Node_4000_CO ...

 

[MikeLee]

Best practice would be indefinite storage but that comes at a price
and is best outsourced to a data center solution company that
has a server park where you can easily store your data.

 

[cwilloug]

The longest we keep deleted data is 30 days......That does make things simple and easy.