9/3/2009. Information Systems Disaster Recovery. Learning Objectives. Why have a plan? unexpected? APPA-Institute for Facilities Management

Transcription

1 Information Systems Disaster Recovery APPA-Institute for Facilities Management J. Craig Klimczak, D.V.M., M.S. Vice-Chancellor for Technology St. Louis Community College 300 South Broadway St. Louis, MO Learning Objectives To identify and describe the key components of an information system disaster recovery plan (DRP) To understand the relationship between information system disaster recovery (DR) planning and business continuity (BC) To identify and assess information system vulnerabilities and risks To evaluate various strategies for information system post disaster recovery and business restoration Why have a plan? C ti i t th Can you anticipate the unexpected? 1

2 Hurricane Katrina at Category 5 2

3 Today s Activities Components of a IS DR Plan IS DR Planning Process Table-Top Top Exercises Risk/Vulnerability Analysis Business Impact Analysis Develop Recovery Strategies Crisis Response-After the Disaster Summary and Closing Remarks The Plan DR Planning FEMA Establish a Planning Team Analyze Capabilities and Hazards Develop the Plan Develop the Plan Implement the Plan 3

4 DR Plan Components-FEMA Executive Summary Emergency Management Elements Emergency Response Procedures Support Documents IS Disaster Recovery Plan: Components Crisis Response Team Critical Person Inventory Notification Checklist Business Recovery Work Area Checklist Critical System Recovery Priority Contingency Plans Critical Vendor Inventory Off Site Stored Materials Service Level Agreements Disaster Recovery Institute DRI International was founded in 1988 as the Disaster Recovery Institute in order to develop a base of knowledge in contingency planning and the management of risk, a rapidly growing profession. DRII administers the industry s premier educational and certification programs for those engaged in the practice of BC planning and management. 4

5 DR Planning Process 10 Steps from DRII Project Initiation and Management Risk Evaluation and Control Business Impact Analysis Developing Business Continuity Strategies Emergency Response and Operations Developing and Implementing Business Continuity Plans Awareness and Training Programs Maintaining and Exercising Business Continuity Plans Public Relations and Crisis Coordination Coordination with Public Authorities Risk Evaluation and Control Determine what business interruption events (risk) and the probability of each that can adversely affect your organization and what controls are currently in place to eliminate or mitigate each risk. Risk/Vulnerability Analysis: Exercise List various types of disasters and crises that might disrupt business operations. For each item on the list, classify the scope of impact on operations. Scope: Regional, Campus wide, Limited, etc. Finally, rank the incidents by probability of the incident occurring. 5

6 Business Continuity vs. Disaster Recovery Business Continuity Management Defines holistic management processes that identify potential impacts that threaten an organization and provides a framework for building resilience with the capability for an effective response that safeguards the interests of its key stakeholders, reputation, and value creating activities. Business Continuity Management The Primary Objective To allow business operations to continue under adverse conditions, by the introduction of appropriate resilience strategies, recovery objectives, business continuity and crisis management plans in collaboration with, or as a key component of, an integrated risk management initiative. Business Impact Analysis (BIA) Estimates the financial and operational impact all credible disruptive events would have on your day-to-day business. It also identifies critical processes/applications/ systems, the people who use and support them, and the risk tolerance for each. The BIA allows you to define the target of your BC/DR Plan. 6

7 (BIA) - continued Identifying your BC/DR Plan Target Locate and identify key stakeholders Identify and prioritize critical applications & data Identify the impact (in $$$) of the loss of critical applications & data over time. Identify the level of risk assumption for each critical application and data Business Impact Analysis: Exercise On your Business Impact Analysis Worksheet, list the systems and functions that are routinely used in the operation of your organization. For each system and function, classify the function as either: Critical, Essential, Necessary, or Desirable. Estimate the daily cost to your organization if this function were not performed following a disaster. BIA Recovery Strategy Identify the Recovery Time Objective (RTO), i.e., the maximum amount of time allowed without access to each critical application & data. Identify the Recovery Point Objective (RPO), i.e., the maximum amount of data loss (usually measured in time) allowed for each critical application. RTO and RPO can be put on a time vs. $$ business impact sliding scale, and a time vs. $$ BC/DR cost sliding scale. 7

8 (BIA) - continued (BIA) - continued (BIA) - continued 8

9 Develop Recovery Strategies: Exercise From your list of systems and functions, choose several high priority systems and develop a list of recovery strategies for each of them. Redundant HOT Site Leased Cold Site Replacement Hardware Contract Service Level Agreement Developing Business Continuity Strategies Identify the technical infrastructure (h/w & s/w) that supports each critical application in the plan and any prepositioned resources that may be required Define a backup and off-site scheme for all critical application data and identify resources that maybe be required to meet your RPO. Emergency Response and Operations Notification Sequence Who pulls the trigger any business interruption that can not be handled by normal operations procedures should be evaluated as a reason to execute your BC/DR Plan. Alternative Command Centers What kind of notification system(s) will you use? Who gets notified first, second, etc. 9

10 Emergency Response and Operations - continued Example notification sequence/breakdown Disaster Management Team - first Disaster Assessment Team second (get pre-event clearance) Technical team (on-alert vs. declared disaster) Local police and medical team (make sure they know about your plan Executive Team Emergency Response and Operations Example notification continued Pre-positioned recovery services providers Public Relations Team User Management (stakeholders) Vendor Team Developing and Implementing Business Continuity Plans FILL IN THE DETAILS! Assign team members and backups Define a notification sequence with actual names Identify in detail the technical infrastructure (h/w & s/w) that supports each critical application in the plan and the staff & backups who will recreate it if the plan is executed remember RTO 10

11 Developing and Implementing Business Continuity Plans Define operation procedures (if different) once recovery is complete. Develop pyour Test Plan Develop your Maintenance Plan Awareness Programs and Training Create and maintain an institutional awareness program. Success depends upon broad based awareness and understanding di of the plan Create and maintain training program for all team members and their backups. Maintaining and Exercising the Business Continuity Plans Maintaining Your Plan Define review periodicity Maintain good change control procedures Maintain good change control procedures Keep good inventories h/w (ser. #s), s/w levels/keys, firmware versions, etc. Password management root/admin 11

12 Maintaining and Exercising the Business Continuity Plans Exercising / Testing Your Plan Without exercising/testing you do not have a plan Establish a Program for exercising/testing E.g., types of tests, periodicity, reporting, etc. Define success criteria Ways to Exercise Your Plan Table Top Exercise Gather the emergency response team Role playing meetings Functional Exercise Ex. Recover data from tape Ex. Pull the power to test generator Full Scale Ex. Bring online a DR site or system Maintaining and Exercising the Business Continuity Plans Use the results to build on strengths and improve weak areas. All exercises and tests are good regardless of results. Use the results to update your Plan as needed 12

13 Crisis Communications How do you communicate internally? Includes team members, employees, management, customers (stakeholders), vendors, suppliers, and the media What kind of notification systems to use? Must be exercised! Coordination with External Agencies Become familiar with the Incident Command System (ICS) in your area. Make sure ICS knows about you and get pre- event clearance for your Disaster Assessment Team Many ICS groups will review your BC/DR plan/strategy. Maintain current knowledge of laws and regulations concerning Emergency Management Immediate Response Take stock of damage It is not just physical Research capacity Recovery capacity (housing, etc.) Take stock of human resources Who is available? What is their work capacity? Take stock of outside resources Who can help? 13

14 Lawson - Lessons Learned Communicate, Communicate, Communicate! Don t wait for others to do it for you. Take control of your destiny. Use outside firms who are experienced and will jump-start the process. FEMA and Government will be extremely slow to reimburse. Prepare your Board for the long haul. Lawson Lessons Learned was brought on-line only for a limited subset of administrators and faculty/staff. Number one internal nal complaint Web site crucial. Employee registration important (lets them know you care, lets you know where resources are). Set application recovery priorities. Lawson- Lessons Learned Advice to DR Planners & CIOs: and the web have become the life blood, know their status at all times. Haveathickskinandasoftheart. a and a soft heart. You will be the hero and the goat at the same time. Don t be upset at the use of outside resources, facilitate it! Don t be surprised when old partners disappoint you, while others support you. 14

15 Lawson Lessons Learned IT is all about people. Know what happened to your people. Be sensitive to their needs but be honest and forthcoming about the capacity of the institution to recover. Remember the stress they are under but help them remember the larger picture of the institution (a delicate balance) Summary BC/DR Planning is a complex endeavor requiring commitment of time and resources to adequately prepare for the unexpected. BC/DR Planning is all about making as many decisions (pre-event) as possible in a calm and thoughtful manner for a time (post-event) when chaos will rule and time is of the essence. The Plan Ours/Yours 15

16 Resources and References bizindst.pdf John Lawson-CIO Tulane University Information Systems Disaster Recovery APPA-Institute for Facilities Management J. Craig Klimczak, D.V.M., M.S. Vice-Chancellor for Technology St. Louis Community College 300 South Broadway St. Louis, MO