Hello everyone!
This may sound a bit like a rant but it is not.
I am really trying to get something done here and I'd like your opinion(s) on how I could proceed.
This will be quite a long post but I have a feeling many TSM users are in a similar situation.
First allow me to state the premise :
I am trying to implement a strategy to satisfy some business requirements that say we must be able to recover our data 'as it was' as far back as 5 years ago. Mostly this is for financial, medical and PR data.
What I need is not assurance that the data has been backed up ans is recoverable. What I need is assurance that we can go back in the past should legal issues arise and so forth. There is no possibility to build this functionnality into the applications because of the risk on intentional fraud/destruction/alteration ans so on.
I know it is impossible to 'catch' every single datum but with other media/job/policy based backup software it is at least possible for everyone in the shop to agree on key dates/days and to run monthly/yearly jobs at those times with say a 5 year retention and make the auditors/users/management happy but... TSM is data based and manages its sauce using file versions associated expiration rules. Thats superb for everyday operations, protecting data on the short term and recovering systems in a DR scenario but that way of doing things, at least for me, really falls on its face when it comes to long-term retention.
So I took a look at how I could achieve what is expected of our data protection solution using TSM.
I tried Backupsets, Archives and I am now gearing up to try "cheatin' it".
Backupsets
Sounds great on paper. Create some nodegroups, generate backupsets on those nodegroups, use the needed retention, problem solved!
But I crossed out that possibility because...
TSM will not generate 'a' backupset from the required data... It will generate individual backupsets for each and every node in the group, sequentially, loading each source tape multiple times for every node as it walks through its data. You need TOC pools. Backupsets are not tracked by the DRM and TSM does not usa scratch pools so offsite'ing media requires cleverness to prevent orphaned tapes. Generating the backupsets interferes with daily operations since the generation process seems to, for lack of a better way of saying it, 'freeze' the data/nodedata it is working on and this leads to endless loop migration processes and other nastiness. The output of Q BACKUPSETS quickly becomes cluttered and management scripts are needed because no human can makes heads or tails of it after a few monthlys or yearlys have been run.
Archives
Again sounds great on paper. Define archive copygroups and pools, thats easy. I can even dedicate a library to that task so as no to impact daily operations. Sweet !
But I crossed out that possibility because...
TSM does not allow to archive the domain: thats strictly for incrementals. So you cannot archive the same data you backup. TSM expects a user sitting at the node, typing 'archive /thisfile.please' but what we have is one or two backup admins and a bunch of schedules. Scheduling archive jobs for the whole shop would be challenging to say the least but you could distribute local scripts on every node that interpret the contents of dsm.opt and generate a tag with the current date and call dsmc toARGH!MYHEADHURTS!.
cheatin' it
So this is where I'm at. Coaxing the product into giving me what I need.
In order to provide the same protection for everyone I have to consider the worst case scenario of a file getting updated/changed each day so in order to provide the requested protection 'The TSM Way' I'd need to build copygroups with VEREXISTS, VERDELETED, RETEXTRA and RETONLY set to 1825. No way this can fly.
The least painful way I can imagine doing something like this is creating two nodes for each computer I need to protect with TSM, using the first node as intended and 'cheating' second one. By 'cheating' I mean using the same domain as the 'straight' node but having a schedule back it up only once a month to a copygroup with VEREXISTS=12 RETEXTRA=365 VERDELETED=12 RETONLY=400. Then add a third node with a yearly schedule backing up to something like VEREXISTS=5 RETEXTRA=1825 VERDELETED=5 RETONLY=1825.
This might work, I might get away with doing just the yearlys if management agrees but I'm still going to have a hard time moving this offsite unless I back it up to a copy pool in order to use DRM.
Has anyone 'cheated' it' with any measure of success?
Any other ideas on how to approach this issue/requirement ?
This may sound a bit like a rant but it is not.
I am really trying to get something done here and I'd like your opinion(s) on how I could proceed.
This will be quite a long post but I have a feeling many TSM users are in a similar situation.
First allow me to state the premise :
I am trying to implement a strategy to satisfy some business requirements that say we must be able to recover our data 'as it was' as far back as 5 years ago. Mostly this is for financial, medical and PR data.
What I need is not assurance that the data has been backed up ans is recoverable. What I need is assurance that we can go back in the past should legal issues arise and so forth. There is no possibility to build this functionnality into the applications because of the risk on intentional fraud/destruction/alteration ans so on.
I know it is impossible to 'catch' every single datum but with other media/job/policy based backup software it is at least possible for everyone in the shop to agree on key dates/days and to run monthly/yearly jobs at those times with say a 5 year retention and make the auditors/users/management happy but... TSM is data based and manages its sauce using file versions associated expiration rules. Thats superb for everyday operations, protecting data on the short term and recovering systems in a DR scenario but that way of doing things, at least for me, really falls on its face when it comes to long-term retention.
So I took a look at how I could achieve what is expected of our data protection solution using TSM.
I tried Backupsets, Archives and I am now gearing up to try "cheatin' it".
Backupsets
Sounds great on paper. Create some nodegroups, generate backupsets on those nodegroups, use the needed retention, problem solved!
But I crossed out that possibility because...
TSM will not generate 'a' backupset from the required data... It will generate individual backupsets for each and every node in the group, sequentially, loading each source tape multiple times for every node as it walks through its data. You need TOC pools. Backupsets are not tracked by the DRM and TSM does not usa scratch pools so offsite'ing media requires cleverness to prevent orphaned tapes. Generating the backupsets interferes with daily operations since the generation process seems to, for lack of a better way of saying it, 'freeze' the data/nodedata it is working on and this leads to endless loop migration processes and other nastiness. The output of Q BACKUPSETS quickly becomes cluttered and management scripts are needed because no human can makes heads or tails of it after a few monthlys or yearlys have been run.
Archives
Again sounds great on paper. Define archive copygroups and pools, thats easy. I can even dedicate a library to that task so as no to impact daily operations. Sweet !
But I crossed out that possibility because...
TSM does not allow to archive the domain: thats strictly for incrementals. So you cannot archive the same data you backup. TSM expects a user sitting at the node, typing 'archive /thisfile.please' but what we have is one or two backup admins and a bunch of schedules. Scheduling archive jobs for the whole shop would be challenging to say the least but you could distribute local scripts on every node that interpret the contents of dsm.opt and generate a tag with the current date and call dsmc toARGH!MYHEADHURTS!.
cheatin' it
So this is where I'm at. Coaxing the product into giving me what I need.
In order to provide the same protection for everyone I have to consider the worst case scenario of a file getting updated/changed each day so in order to provide the requested protection 'The TSM Way' I'd need to build copygroups with VEREXISTS, VERDELETED, RETEXTRA and RETONLY set to 1825. No way this can fly.
The least painful way I can imagine doing something like this is creating two nodes for each computer I need to protect with TSM, using the first node as intended and 'cheating' second one. By 'cheating' I mean using the same domain as the 'straight' node but having a schedule back it up only once a month to a copygroup with VEREXISTS=12 RETEXTRA=365 VERDELETED=12 RETONLY=400. Then add a third node with a yearly schedule backing up to something like VEREXISTS=5 RETEXTRA=1825 VERDELETED=5 RETONLY=1825.
This might work, I might get away with doing just the yearlys if management agrees but I'm still going to have a hard time moving this offsite unless I back it up to a copy pool in order to use DRM.
Has anyone 'cheated' it' with any measure of success?
Any other ideas on how to approach this issue/requirement ?
