Networker

Re: [Networker] True Disaster Recovery

2006-10-03 16:28:56
Subject: Re: [Networker] True Disaster Recovery
From: Stan Horwitz <stan AT TEMPLE DOT EDU>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Tue, 3 Oct 2006 16:20:41 -0400
On Oct 3, 2006, at 12:57 PM — 10/3/06, Albert Eddie Contractor AFRPA CIO/IT wrote:

I read (with interest) the thread about disaster recovery. (Recovering
tapes on Server B if Server A dies and you can not change Server B name
because it is in use as a full time server)...

Our disaster plan has to be a little more "NEW ORLEANS" (what if the
site is completely wiped out!?)

I love reading other people's disaster plans even while I am refine my
own. Do any of you have disaster plans you could share with me?

Of particular interest is complete SERVER ROOM disaster (aka no other
server in room can be used)....

Some of the best plans (on a budget) that I have read have included
LEASING a server that meets the H/W & S/W requirements on a month by
month basis while you rebuild your server room with PURCHASED H/W.

Would love to hear your thoughts.

Our DR plan here at Temple U. is to classify each of our servers into different levels such as 0, 1, 2. Level zero servers get top priority for recovery in the event of a major disaster, al la Katrina. The higher the level number, the lower the priority. Level zero servers must have a duplicate server at one of our two DR sites. One is owned by Temple and is on Temple's main campus. The other site is rented from Sunguard, but it is still here in Philadelphia. The Sunguard site is where our old mainframe iron's DR stuff is located. We also have a small LDAP farm, DR web server, and email server (for executive board members) at Sunguard.

Our on campus DR site is about 1/4 mile from our main data center. Our only NetWorker server is located in the on-campus DR site. We also keep some small administrative servers there that are DR level 0 and level 1 servers in our data center.

We have quarterly rehearsals that involve random server audits to ensure that fail over to our DR stuff works. As far as the DR stuff that's backed up by the NetWorker server that I manage (which is the majority of our data), we have a 100% success rate in meeting DR requirements. Our mainframe DR is handled outside of NetWorker and is not within my purview ... yet.

We ship mainframe data off-site six days a week. All data that's backed up via NetWorker sits on tapes in our Sony PetaSite and those tapes are never removed from the tape library, nor do we copy them. This is because the library is already outside the building where the primary copies of the data are maintained. Plans are also in progress to do a major revamp of how our mainframe data (e.g., student grades, payroll, etc.) are backed up, so our mainframe DR strategy is likely to change.

Our NetWorker server consists of a storage node and a main server, both running NSR 7.2.1. The server has a huge Sony PetaSite connected to it, which is on its on fibre SAN. Due to the high cost of purchasing our Sony tape library, we have not taken steps to replicate it anywhere at all. If our NetWorker server goes up in smoke though, we lose no production data because 100% of the production data is in a different building. Our data center and on- campus DR center are well protected structurally from environmental disasters and Temple can provide its own power in an emergency. The DR site also is guarded by a bunch of automated protections from fire and unauthorized intrusion, as is our primary data center. As a result, the level of exposure to business discontinuity if our NetWorker server died is very remote.

We are shopping for a new Chief Information Security Officer. Once a new CISO is hired, it would not surprise me at all if having DR plans for our NetWorker server enters the picture, but I think management will be shellshocked at the cost to protect our NetWorker server and our main tape library. In terms of hardware cost, our NetWorker server and its tape library are the most expensive server Temple owns, by far and I would say that licensing it with EMC probably ranks second or third, so I would be surprised if we actually take any real steps to protect our backups from a Katrina style disaster due to the high cost, but I might be wrong and I do at least expect this question to get on the radar screen at some point this coming year.

We also have one storage node and and old tape library still online, but out of production. We are seriously considering using that hardware for our mainframe backups outside of NetWorker (lack of IBM mainframe support), but that issue is still undecided.

To sign off this list, send email to listserv AT listserv.temple DOT edu and type 
"signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu 
if you have any problems
wit this list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

<Prev in Thread] Current Thread [Next in Thread>