Disaster Recovery When a Disaster Strikes, Will You be Ready?

Transcription

1 Disaster Recovery When a Disaster Strikes, Will You be Ready? Larry Mattox,VC3 November 14, 2013 MHRA Annual Meeting Common Disasters More than Hurricanes! External Disasters Tornado Snow/ice Flood Fire Chemical spill Internal Disasters Server hardware failure Disk drive failure Data corruption issues (viruses, controller, application, database, etc.) 1

2 Real World DR Examples City of Mullins 2011 and 2012 flood damage to Police Department and City Hall Fire in adjoining non-city building impacted City Hall Fortunately Mullins was part of the MASC Agility program! TJCOG Water damage to GIS room Kirby Bowers, Executive Director, I m glad I moved to the cloud! Fellow tenant (of VC3) water pipe broke Damaged equipment resulting in no HVAC for entire building for one week (20 story building) North Carolina Company Air conditioner condensation pump above server room flooded Hurricane the next month Have You Thought About? Impact of a disruption on local businesses and citizens? Potential economic impact on city? Applications most reliant on? How long could you provide a consistent level of service without those applications? Which applications are critical vs. supportive? Effects of Disruption Operational Impacts Customer service Quality of support Operational backlog Dependence on information Intangible Impacts Public opinion Social issues Employee morale/confidence Financial Impacts Lost revenue/reimbursement Fines and penalties Uninsured record loss 2

3 When a Disaster Strikes, Can You? Provide your services When your citizens need them the most? At the level needed by your citizens? Recover your communications system? Do You Have..? A Disaster Recovery Plan? A document that defines the resources, actions, tasks and data required to manage IT operations in the event of a disruption. A Business Continuity Plan? A comprehensive written plan to maintain or resume business in the event of a disruption. The 4 Key Elements of a DR Plan? 1. Solid daily data backups 2. Offsite backup 3. Access to your data 4. Written Plan 3

4 Do You Have a Solid Procedure? Daily, weekly, monthly backups Person in charge (and back-up) Technology that works Data stored offsite (secure site) Regularly test backups Run periodic checks of process Review, update and improve your plan Do You Have an Offsite Backup Plan? Data stored offsite (away from municipality) At secure, non-city location True data center Iron Mountain, etc. To different geographic location Coastal to inland Sister city What Your DR Plan Must Do for You! Replicate data nightly to offsite facility Replicate system configuration nightly Full recovery at remote site within 24 hours Conduct annual test of recovery capabilities 4

5 The Cloud Removes DR Concerns Included in most cloud solutions = lower cost Recovery is immediate! Backs-up more than traditional DR solution Frees-up resources formerly dedicated to DR Cloud-Based Backups & DR Eliminates process that leads to human error Automated backups Done by DR professionals Accessible in a disaster Data stored in secure offsite data center Does not require replacement equipment Data easy to access remotely New Options Because of the Cloud Hosted desktop Staff can work from any location with internet connection No loss of data Zero down-time from disaster to recovery Hosted phones Automatically re-route to number not affected by disaster Turn your home phone, cell phone or even hotel room phone into your office phone 5

6 Where to begin? 5 important questions to answer about your Disaster Recovery Plan. 1. How Is Your Municipality s Data Backed Up? Not being backed up Critical problem! USB Drive Tape Disc Cloud Not satisfactory! Very vulnerable. Lack of back up space. Loss/damage to USB. Can you access in a disaster? Human error issues. Can you access in a disaster? Human error issues. Accessible in a disaster. Eliminates process that leads to human error. 2. Who Is In Charge of the Backup Process? Take real ownership? What happens when they aren t there? Partner in the process? Conducting periodic tests to ensure data is backed up? 6

7 3. Where Is Your Data Stored? Onsite (In same building) Near the server Locked room or vault Offsite (In same city area) Bank vault Employee home Other city site Offsite (In different geographical area) Sister city Enterprise class data center 4. Can You Recover Data in a Disaster? Does it require replacement equipment? Can it be restored to you remotely? How long will either option take? 5. Do You Have a Business Continuity Plan? 1. Should be a written document 2. Plan should be accessible in the event of a disaster 3. Defined location(s) for key people during/after a disaster 4. Calling Tree 5. Key staff - access their data 6. Designate a staff member to update the municipal website with alerts for your residents 7

8 Definitions Business Continuity - Resumption and recovery of business activities to pre-disaster state. Business Continuity Plan A comprehensive written plan to maintain or resume business in the event of a disruption. When the Lights Go Out Do you have a plan? Can you find it or access it? But Before the Lights Go Out You have the opportunity to create a Business Continuity Plan 24 8

9 Business Continuity Plan A comprehensive written plan to maintain or resume business in the event of a disruption. Maintain & Modify Your Plan Working Document Establish Periodic Review Cycle Systems Procedures Personnel Changes Failure to Plan = Planning to Fail Your Mission Find out If you have a documented Disaster Recovery or Business Continuity Plan Yes: Who knows the most about it? What does it cover? Has it been tested and when? Or can it be tested? 3 rd Party review No: Why not? What will be done to put one in place? Assistance putting one together 9

10 Larry Mattox, VC3 10