BUSINESS CONTINUITY PLANNING GUIDELINES Washington University in St. Louis The purpose of this guide is to serve as a tool to all departments, divisions, and labs across the University in building a Business Continuity Plan (BCP) to guide recovery operations. This will lead toward a resilient culture that sustains the critical operational functions of WUSTL.
Business Continuity Planning Purpose Severe disruptions to operations can result from a variety of situations. Washington University in St. Louis (WUSTL) Emergency Management officials has identified dozens of possible hazards to the different campus areas with the potential to cause severe, negative impacts. Such hazards include tornadoes, earthquake, fire, severe thunderstorms, severe blizzard/snow fall, building flood, utility failures, pandemic flu, hacking, data breach, IT equipment failure, or workplace violence. While departments, divisions, and labs at WUSTL may not be able to prevent these from occurring, effective planning minimizes the number of ad hoc decisions that must be made following a severe disruption and enables areas to resume essential operations more rapidly. As a world-renowned University, business continuity planning becomes a critical endeavor for long-term sustainment of vital operations that have local, regional, national, and international impacts. Vital University achievements include: Over 14,000 students, with over 40% that study abroad Approximately 4,000 degrees awarded annually More than 1,500 courses taught Over 4,500 students living on campus More than 3,000 research projects each year with annual support close to $550 million Over 1,200 full-time medical faculty treating more than 430,000 children and adults Over 900,000 outpatient visits annually. The regional impact of WUSTL includes: That s my life s work and I don t have another life Researcher from CSU, Northridge watching his office burn after the 1994 earthquake (Source: Academic Aftershocks) Third largest employer in St. Louis Students from outside the region add approximately $144 million to economy Total amount of goods and services purchased locally is greater than $230 million annually Approximately 26% of WUSTL alumni live and work in St. Louis area. The purpose of this guide is to serve as a tool to all departments, divisions, and labs across the University in building a Business Continuity Plan (BCP) to guide recovery operations. This will lead toward a resilient culture that sustains the critical operational functions of WUSTL. This first edition for the BCP guide outlines the action steps towards business continuity. It is anticipated that future editions will include supportive tools and templates to compliment the steps outlined. Page 1
Two Critical Emergency Plans As a basic outline, there are primarily two categories of emergency plans: 1) Emergency Action Plans and 2) Business Continuity Plans. A. Emergency Action Plans Emergency Action Plans (EAPs) are built for the immediate life safety and the property protection of all University assets. Actions outlined within an EAP are life safety actions written specifically to faculty, staff, students, and visitors for immediate actions. These actions occur from the moment there is a hazard impacting the University, or an imminent threat of a hazard, until the immediate threat to life has passed. B. Business Continuity Plans Business Continuity Plans (BCPs) typically activate after EAPs have been implemented and focus on the recovery aspects of the disaster. BCPs consist of two primary phases: 1. quick and temporary resumption of critical, time-sensitive services and operations; 2. complete restoration of all business functions, returning back to normal. A systematic BCP does not focus the planning efforts on every possible hazard scenario that may occur but rather it identifies the common impact elements in any disaster: i.e., loss of information, loss of personnel, loss of equipment, loss of access to information and facilities, and seeks to design the plan to protect against these possible impacts. While it is important to have an Emergency Action Plan, this guide is designed to focus only on the Business Continuity Plan. The two plans should seamlessly complement each other in guiding all faculty, staff, and students to safely respond and recover from any disaster incident which impacts the campus. Completing a business continuity plan can be done over a series of several steps. The primary steps towards developing and maintaining an effective plan include: School/Department Initiation Conduct Risk Evaluation Conduct Business Impact Analysis Develop Business Continuity Strategies Write and Finalize Business Continuity Plans Train and Exercise the Business Continuity Plans While each step is critical, many of the steps may be conducted simultaneously. The pace for the plan development will be dependent on the effectiveness of the team members involved in the process. Page 2
Building a Business Continuity Plan A. School/Department Initiation 1. Establish senior management support Success or failure for any part of the BCP process will depend on the level of senior management support of each school/department within the University. It is critically important that Deans, Directors, and Department Heads clearly a. outline BCP expectations for all members within the school/department, with specific assignments to key personnel and b. ensure continual progress in planning, training, exercising, and evaluating. 2. Identify unit planning level The Unit for a school/department is the level at which it has been identified that a BCP needs to be created. All schools/departments across the University vary in size and scope. Senior management for each school/department must determine, at the beginning, the appropriate Unit level for building a BCP. Smaller schools/departments may be considered as one Unit and build only one BCP. For larger schools/departments with multiple divisions and/or labs, one BCP may be inadequate to address all the personnel and operational functions. For such larger schools/departments, multiple Units (i.e. each division, each lab, etc.) will need to be defined and the BCP process applied to each Unit. Key criteria to consider when identifying the appropriate Unit level for a BCP include: a. Size of the school/department s organizational structure and the appropriate school/department level that can effectively implement a recovery plan to ensure quick and efficient actions with all employees addressing all core, critical functions. b. School/department personnel with authority to address core, critical functions identified within the school/department. For example, a Principal Investigator is ultimately responsible for the continuity of critical core functions for specific laboratory areas. A BCP addressing the key personnel and critical core functions for those specific laboratory areas would be vital for effective recovery of operations. Larger schools/departments need to determine if such laboratory core functions could be adequately addressed in a BCP combined with general policies and functions or if a separate and individual BCP would be of most benefit. Page 3
For larger schools/departments that identify more than one planning Unit and thus more than one BCP, the BCP development outlined in this guide will apply to each individual Unit level as well as at the top School/Department level for services that apply to all Units. Connectivity and collaboration between the BCPs will be critical. 3. Identify a Business Continuity Coordinator The Business Continuity Coordinator (BCC) is the person selected to lead the planning effort within each identified Unit and work with Emergency Management personnel to ensure plans are completed, exercised, and updated regularly. If multiple plans are going to be developed within a school/department, the BCC for the School/Department BCP should be the lead coordinator for all Unit BCP activities, coordinating with other Unit BCCs identified within the school/department. Key criteria in identifying a Coordinator that will be successful include: a. A position with mid-level management authority or above, b. A willingness to receive additional training related to BCP, c. A commitment of time to get the plan written, followed up by adequate training, exercising, and evaluation, d. Interpersonal communication skills that encourage input and team coordination towards the successful implementation of a BCP program. 4. Organize a planning team At a minimum, every school/department needs to establish a business continuity planning team that includes key people that know the school/department well enough to understand its functions and priorities. Larger schools/departments will likely require more people to participate on the team. For large Page 4
schools/departments with large divisions/units, teams within each division/unit may be considered necessary to support the coordinator appointed to lead the planning effort for that division/unit. Business continuity planning team members may include: a. Lead BCC for the school/department b. Continuity coordinators for other Units within the school/department c. Key management personnel representing all areas of school/department d. Internal Information Technology personnel e. Internal Human Resources personnel 5. Outline work plan & schedule to build BCP With regular internal meetings, the business continuity planning process should take approximately three to four months. Every school/department may create time frames that are the most manageable for the staff involved. The four primary steps to building the BCP include: Conduct a Risk Evaluation (2 weeks) Conduct a Business Impact Analysis (3-4 weeks) Develop Business Continuity Strategies (3-4 weeks) Write and Finalize BCP (2-3 weeks) If BCP coordinators are able to make the planning process a priority with a strong commitment toward its completion, a base BCP can be completed within the time frame outlined. The plan can then be updated and modified as it is exercised. Responsive and active planning team members will facilitate an effective use of time and an efficient planning process. B. Conduct Risk Evaluation Identify possible disruptions The goal in this step is to identify the risks/threats that can adversely affect the unit, its resources, and/or image. Once identified, threats are assessed as to their potential level of impact that would result. The information gathered and learned here is carried to the next step, Business Impact Analysis, to apply the impacts of the disruptions to critical functions. a. Risks/Threats WUSTL Emergency Management officials have conducted extensive hazard assessments of higher priority risks that could adversely affect Page 5
personnel and operations on campus. Some potential risks that could impact WUSTL include: tornado fire severe thunderstorm severe blizzard/snow fall building flood temperature extreme utility failures earthquake hazards materials spill pandemic flu workplace violence civil unrest hacking data breach IT equipment failure The expectation of each unit developing a BCP is to review the potential risks to WUSTL and identify any additional risks to your area that may differ from those identified. These risks will guide the development of the potential impacts that could alter unit operations. b. Impacts Identifying the hazards that may cause any type of catastrophic event is a critical element of any emergency planning. However, the more critical use for this data is to identify the potential level Expect the best, plan for the worst, and prepare to be surprised. Denis Waitley of impact that can adversely affect resources when such a hazard becomes a reality. This step is also known as the what if scenarios, or identifying what could happen. As worse case scenarios can easily be scripted, it is more effective to plan, to the best extent possible, for the likely level of impact from the disruptions identified. In evaluating the different hazards, common elements or impacts from many or all of the hazards can be noted. For example, a tornado, severe winter storm, or earthquake all likely would cause power outages. So when evaluating the impact of risks, every unit should outline how the hazards of highest concern will affect: availability of personnel availability of information technology availability of communications status of infrastructure (facility, utilities, transportation). Page 6
By defining how the hazards could affect each of the bullet areas is building the potential impacts of disasters against a unit s resources. This data will be carried into the next step, Business Impact Analysis. C. Conduct Business Impact Analysis 1. Identify critical business functions The first step in the Business Impact Analysis (BIA) is to identify and prioritize the business functions and processes that are most vital to operations. In each Unit, a BCP needs to specifically define the core business function (s) that is essential to their operations and must continue throughout an incident or resume rapidly thereafter. Criteria to consider when evaluating critical business functions are those that play a primary role in supporting: a. life safety b. customer support c. finance vitality and sustainment d. regulatory mandates e. academic/clinical/research operations or f. department/university reputation. As the majority of Unit functions likely fall within the criteria listed, it is important to define the essential functions that must be implemented quickly to sustain basic operations. Critical resources and services that support the essential functions identified will be reviewed later in the process under Dependencies. 2. Prioritize functions and processes based on level of criticality After identifying the essential core functions, it is necessary to prioritize the level of criticality for each of the functions defined by a Recovery Time Objective (RTO). The RTO is the amount of time between the disaster impacts to the time when the function is operational at a minimum acceptable level. This is also known as the tolerable length of unavailability. For each function identified, departments need to rate the level of criticality as a: a. Critical 1: RTO less than 24 hours b. Critical 2: RTO 24 72 hours c. Critical 3: RTO greater than 72 hours. An example may be that Clinical Care is identified as a critical function that must be operational within 24 hours. This would be defined as a critical 1 Page 7
function. A second example may be Classroom instruction is identified as a critical function that must be operational within 72 hours. This would be defined as a critical 2 function. 3. Identify potential impacts on critical functions With the potential impact on resources identified along with the critical functions, it is now time to bring the two sets of data together. The Unit s BCP planning team evaluates each critical function and identifies how the potential impact on resources (personnel, IT, communications, and infrastructure) could negatively impact the operations of each one. For example, if a pandemic flu could likely decrease staffing resources by 40%, how will clinical care or classroom instruction be impacted? 4. Identify critical function vulnerabilities In addition to identifying the potential impacts of disasters, it is at this stage of the BCP process that vulnerabilities are often discovered within any workplace. Single points of failures, minimal redundancies, etc. often will be exposed at this point. For example, only one person may have access to key areas or only the administrative assistant is able to reference personal contact information of other staff members. It is important to note vulnerabilities that exist, or weak planning areas within the unit that would allow disasters to be more destructive. Strategies for addressing these vulnerabilities will be outlined in the next section. 5. Identify dependencies and peak periods Two key characteristics associated with critical functions that must be assessed are dependencies and peak periods. a. Dependencies University operations can be very complex with multiple interdependencies among internal and external processes and systems. It is vitally important for each unit to identify critical dependencies (services and resources) that can affect the successful implementation of their critical functions listed. The three common areas of interdependencies to evaluate include, but are not limited to: Communications/Network/Data Centers o Central IT services o Telecommunications Facility Corporation (TFC) Page 8
Internal University Systems and Processes o Human Resource Management and Self Service System (HRMS), o Student Information System (SIS) o Financial Information Systems (FIS) o Clinical Systems o Grants Management o Payroll o Human Resource (HR) Services o Resource Management Services o Facility Operations Services (including utility resources) 3 rd party providers, key suppliers, business partners It is critical to note that detailed, emergency coordination agreements with 3 rd party providers are vital to ensure adequate support in times of crisis. In discussing emergency response capabilities with current and/or potential vendors needed in an emergency, it is important to analyze response capabilities including local/regional/national presence, redundancies, business disaster plans, resource prioritization with other customers, etc. In identifying the dependencies that support critical business functions for each area, it is necessary for BCP committee members to communicate and coordinate with the appropriate liaison for each of the centralized services or resources. b. Peak Periods These are periods of high activity specific to a critical function. Units should identify any time periods during the year when it would be expected to have an especially high level of activity in accomplishing a particular function i.e.: class schedules; annual fiscal closing, etc. Page 9
D. Develop Business Continuity Strategies 1. Addressing identified vulnerabilities The first order of business in outlining action strategies is to prevent or minimize the potential damage that could occur. This is Luck is what happens when also known as mitigation. Often preparation meets opportunity times, many of the vulnerable Seneca, Roman Philosopher, areas identified previously can be 1 st Century AD mitigated prior to an incident to lessen the impact of disasters. When outlining pre-disaster improvement measures, there are two types of measures that can be used to address the vulnerabilities exposed: a. Preventive controls: controls or actions that can be taken to completely inhibit the impact exposures (i.e. buy a generator). b. Reactive controls: controls or actions that can be planned and instituted to compensate or minimize the impact of exposures (i.e. establish a staff call-up process). 2. Identify the critical function requirements With the knowledge of what can happen and the functions that must be operational during or immediately following a disaster event, the next step is to outline the actions to be taken during the recovery period. a. Define the minimum resources needed: Units need to analyze each critical function and identify what options and/or minimum resources would be needed in order to meet the RTO determined by the planning team. For example, if addressing a critical function of continuing clinic care operations within 24 hours, the minimum number of skilled/credential personnel necessary to carry out basic/urgent clinical care services would need to be identified. Another example would be electrical power, identified as critical 1, to support refrigerant equipment storing vital research specimen. It would be necessary for the BCP planning team to identify the maximum amount of time the equipment can sustain adequate coolant without electricity. If the equipment is supported by generator power, the run Page 10
time for the generator prior to refueling should be defined. b. Find the resources needed: After identifying the minimum resources required to carry out the critical functions, it is necessary to identify all the possible options currently available, or could be made available, in order to acquire the needed resources. If considering centralized resources as a resource, involve them in the planning to make sure your expectations, or RTOs as identified above, can be met. For example, if you have a RTO for accessing a network for critical documents in less than 24 hours, ensure that your Unit and/or Central IT personnel know and have the resources and planning in place to meet this RTO for your critical documents. All possible alternative continuity options should be considered. Alternative continuity options could include: develop manual workaround procedures, develop reciprocal agreements with other departments and/or outside institutions, identify internal and/or external alternate work site, contract third party service providers/outsourcers, suspend operations for specified amount of time, arrange for staff to work from home. c. Outline the strategies to implement the resources: This step can include identifying trigger points for activating alternate resources, methods for activation, and logistical detail for sustainment of the alternate resources during the recovery phase. 3. Vital records management WUSTL records that are designated as vital to University operations and which if destroyed would seriously impair normal University affairs, or which by their loss might place the University in a state of legal or fiscal jeopardy, should be secured. Vital records should be restricted to those records that really are crucial for University continuity of operations following a disaster. a. Identify vital records Vital records may be in any format or medium (paper, electronic, microfilm, etc.). The vital records copy may not be the original the Page 11
information contained, not the medium, is most important. b. Define schedule for duplication and transferring of records The three key principles to protecting vital records is the duplication, transferring, and safe storage of the records. With a clear understanding of which documents need to be protected, the unit must determine what regularly scheduled activities need to be performed to duplicate, transfer, and store new records. The schedule can vary depending on the type of record and resources available to perform the necessary tasks. c. Select the protection methods for those vital records identified. The third key principle, safe storage, must be carefully evaluated so risks to the vital records are minimized to the greatest extent possible. Electronic vital records required for supporting WUSTL departments critical functions, systems, data center operations, and other priority functions, and procedures needed to recover them and/or reconstruct lost data are developed. In addition, procedures to establish and maintain offsite backups are completed and/or reviewed. Safe storage options for vital records may include: on-site with fire-resistant cabinets, off-site within a low-risk, protected area, third party disaster recovery firms. E. Write and Finalize Business Continuity Plans The effective compilation of the data gathered throughout the BCP process is critical to ensure optimal execution. The plan must serve as a reference guide that is user friendly and accessible to all that are impacted by the actions outlined in the BCP. At this point, WUSTL officials are currently evaluating BCP support tools that will support the compilation and organization of departmental BC information. Such a tool will require the information gathered up to this point. Effective and applicable information gathered throughout the process will provide the substance to make a written plan useful to the personnel it is designed to protect. F. Train and Exercise the Business Continuity Plans Once the BCP has been created, it is of little value unless all users and benefactors of the plans are trained on the BCP and it is exercised. This should be done initially after the Page 12
BCP approval and then on a regular cycle for review. 1. Training on BCP All staff impacted by the BCP should have access to view the plan and be trained on roles and responsibilities expected. This training should take place in two methods: a. Formal training: A formal training session focused on reviewing the key principles of the BCP is the primary way to kick off exposure of the plan to staff. Depending on the size of the group, this training can take place in several sessions. b. Regular review: In addition to formal training sessions, it is a best practice to review different elements of the plan in regularly scheduled sessions such as staff meetings or department trainings. This review session can be no more than 5 minutes, following up on previous BCP assignments or reviewing a key element of the BCP such as the strategy to address the loss of a department critical function. 2. Exercising the BCP As complete as a plan may seem when originally written, there are always areas for improvement and corrections will be needed. The primary method for identifying improvements in the plan is exercising it on a regular basis by conducting exercises with the individuals expected to execute the plan following a disaster. It is preferable to identify weak areas of a plan in an exercise environment when the situation is controlled. The more you sweat in practice, the less you bleed in battle. Chinese Proverb Each unit should identify the frequency and most appropriate time frame for exercising the BCP. Outside normal business hours to avoid operational disruption is not always the best time for testing the plan. The closer to reality an exercise is conducted, the more likely it is to identify challenges that will appear during the real event. The entire plan does not need to be exercised every time. It is often more effective to focus on specific elements of the plan for each exercise. These exercises can take on the form of sitting around a table and discussing a scenario that would require BCP implementation. Participants discuss their way through Page 13
all the potential issues identified. A more involved method is to put the concepts of the BCP into practice. For example, if a strategy to respond to loss of email was to communicate with official department head memos, then one exercise may be to spend one day requiring all staff to only utilize the department memos supplied. G. Maintenance and Audit of the Business Continuity Plans All emergency planning documents, including a BCP, are considered living documents. As such, they are in continual need for revision and updating. Revisions to the plan most commonly come from 1) suggestions from personnel during training, 2) weaknesses identified while exercising the plan, and/or 3) lessons learned during real events. No matter the method for identifying improvements, review and updates to the plan should be conducted annually at a minimum. The BCC is responsible for ensuring the regular review and updates of the plan. The final BCP for each Unit should be shared with school/department senior management and public safety officials. This will ensure system-wide continuity for the University by encouraging integration among key partners and stakeholders throughout. Page 14