Business Continuity Planning Principles and Best Practices Tom Hinkel and Zach Duke

Transcription

1 Business Continuity Planning Principles and Best Practices Tom Hinkel and Zach Duke

2 Agenda Key components essential to a FFIEC compliant Business Continuity Plan Recovery Time Objectives & Recovery Point Objectives Lessons learned from recent disasters Cost-effective recovery options to protect your critical business processes Testing moving from Compliant to Recoverable

3 FFIEC BCP Update

4 FFIEC Four Phase Process 1 Business Impact Analysis 4 Risk Monitoring and Testing 2 Risk Assessment 3 Risk Management

5 Business Impact Analysis Foundation for all Business Continuity Plans FDIC Are disaster recovery and business continuity plans based upon a business impact analyses (Y/N)? Identification and prioritization of all business functions and processes, including interdependencies Maximum Allowable Downtime (MAD) Recovery Time Objectives (RTO) Recovery Point Objectives (RPO)

6 BIA - Recovery Time Objectives Must factor the cost to NOT recover Reputational Operational Regulatory Strategic as well as the financial cost to recover.

7 BIA Example - Teller - Relative Risk Recovery Time (days) Financial Operational Regulatory Strategic Reputation < = insignificant risk 10 = unacceptably high risk

8 BIA Example - Teller -

9 Business Impact Analysis RTO, MAD & RPO RTO - The amount of downtime that can be tolerated without exceeding risk tolerances. MAD The point at which recovery become impossible or losses unrecoverable. RPO Acceptable data loss. Defined by frequency of data backups.

10 Business Impact Analysis One of the advantages of analyzing allowable downtime and recovery objectives is the potential support it may provide for the funding needs of a specific recovery solution based on the losses identified and the importance of certain business functions and processes.

11 Risk Assessment Impact x Probability = Potential Severity Analyze threats based on the potential impact to the institution The most difficult threats to address are those that have a high impact on the institution but a low probability of occurrence.

12 Risk Assessment Relative Severity Index Disruption Threat

13 High Probability Low Impact High Probability High Impact Fraud, Theft, Blackmail Sabotage Vandalism & Looting Terrorism Fire Probability Flood, Water Damage Air Contaminants Severe Weather Hazardous Spills Comm. w/customers Comm. w/employees Payment Service Providers Affiliates & Vendors Power Failure Low Probability Low Impact -4-5 Impact Low Probability High Impact Equipment & Software Failure Trans. System Disruptions Water System Disruptions Pandemic

14 Interdependencies Analyzing interdependencies represents a critical step in the business continuity process and is an integral part of a business impact analysis. A work flow analysis involves an assessment and prioritization of those business functions and processes that must be recovered. Should assist management in determining the priority of business functions and processes and the overall affect on recovery timelines.

15 Work Flow Example Interdependencies

16 Work Flow Example Interdependencies

17 Criticality Interdependencies

18 Complexity Interdependencies

19 Where to Start? Perform/Re-visit Business Impact Analysis Senior Management / Board of Directors Review Likelihood of Achieving RTO Technology, Facility, and Personnel Review Review High Risks from Risk Assessment Review RTO Estimates Highlight Deficient Areas to Management Gap Analysis

20 Technology s Role Design Addresses BIA / RA Policy and procedure adherence Process / Automation Reduces and RTO Risk reducing Documentation Oversight Trending Testing / Reporting (RTA)

21 Downtime Always on / connected society Acceptable length of downtime continues to shorten Critical applications shift Internet

22 Advantages Centralization of IT Reduced branch / user infrastructure Enhanced administration with one time updates Scalability new locations / institutions Risks All eggs in one basket (DR) Bandwidth All applications may not work

23 What is Virtualization? I keep hearing the term, but what does it mean to me? Common Terms Hypervisor VMWare XENServer Hyper V VDI Virtual Desktop Infrastructure The new Dumb Terminal Citrix Application Virtualization (Centralization) Loans Shared storage (SAN)

24 The Old Server Room

25 Virtualization Virtualization Stable configuration with N+1 design New servers can be provisioned quickly Additional resources easily added for scalability Better utilization of resources Centralized management Disaster recovery replication

26 Basic N+1

27 Shared Storage Server A Virtual N+1 with Shared Storage Server B Server C ESX 1 Server 1 ESX 2 Server 2 SAN Server 3 ESX 3 Server Alpha Server Zulu

28 Server Failure Scenario Server A ESX Physical Server Failure Server B Server C ESX 1 Server 1 ESX 2 Server 2 SAN Server 3 ESX 3 Server Alpha Server Zulu

29 Disaster Recovery Main Office / OPS DR Location Server A Server A Server B Server B Server C ESX 1 SILVER PEAK Server C Server 1 ESX 2 ESX 1 Server 1 Server 2 SAN SAN Server 2 Server 3 ESX 3 Server Alpha SILVER PEAK ESX 2 Server 3 Server Alpha Server Zulu Server Zulu

30 Circuit Redundancy MPLS Communications Automatic Router Failover Internet Connectivity - continues to have increased importance for institutions Key Technology Options Redundant Internet (Main Location & DR location) Redundant Hardware

31 Point to Point Single Point of Failure Location 1 Location 2 Location 3 Location 4

32 Frame Relay Private Virtual Circuits Location 1 Location 2 Location 3 Location 4

33 MPLS Example Everyone is Connected Location 1 Location 2 Location 3 Location 4

34 Data Vaulting Remote offsite backups via Internet connection More frequent backups than tapes (lower RPO) Removes need for tapes and rotations, file recovery NOT location specific Removes people from the process Encrypted from Point to Point Risks Internet connectivity Large restores (i.e. whole server)

35 Server Recovery Server Recovery Allows for snapshot replication of servers Significantly reduced RTO Does not require independent hardware for testing or recovery Centrally managed Designed for smaller data footprints

36 The Cloud

37 Cloud Solutions Service Bureau Core Processing Application Hosting (SaaS) Mortgage, Imaging, New Accounts Hosted Exchange Encryption, SPAM / AV Protection Hosted Servers Web, Intranet, etc. Collocated Solutions Move Servers from the Main Office / Ops Center Reduces Disaster Recovery Risk

38 Additional Considerations Facilities Personnel Cross training Plan for DR without key personnel Unplanned Recovery Events Documentation of DR events that occur during normal processes

39 Why Test? Validation of the Business Continuity Plan Auditors/regulations require BCP testing Practical application to support regulatory and audit issues Potentially identify operational exposures Maintain awareness and education

40 Testable DR/BCP - Recoverable Do test recovery objectives follow BIA? Do test scenarios follow RA? Process recovery NOT system recovery Tested Increasingly complex scenarios Gap analysis between RTO and actual recovery capability

41

42 Questions/Feedback Tom Hinkel, CISA, CRISC, CCSA Director of Compliance Zach Duke EVP, Business Development #SafeConf