Disaster Recovery Journal Spring World 2014

Transcription

1 Disaster Recovery Journal Spring World 2014 What works: Services and service supply chain business continuity risk management Don Hall, CBCP, Cisco Services Business Continuity Analyst Cisco Systems, Inc Kit Creek Rd. Research Triangle Park, NC Slide 2 (untitled) In the 4 th Annual Business Continuity Institute (BCI) Supply Chain Resilience Survey 73% of survey respondents experienced at least one supply chain related disruption with an average of five. This high level is consistent with the trend line over the past four years. (Business Continuity Institute, 2012) Note: The 4 th Annual BCI Supply Chain Resilience Survey can be found at: Slide 3 Agenda The Challenge The Solution o Scope o Methodology o Metrics and Reporting Supporting Contracts Lessons Learned Questions & Answers Slide 4 The Challenge Slide 5 (untitled) If I always appear prepared, it is because before entering an undertaking, I have meditated long and have foreseen what might occur. It is not genius where reveals to me suddenly and secretly what I should do in circumstances unexpected by others; it is thought and preparation. (Napoleon Bonaparte) Note: My favorite quote from Napoleon Bonaparte on the need for contingency planning What works: Services and service supply chain business continuity risk management Page 1

2 Slide 6 (untitled) If you wish to be a success in the world, promise everything, deliver nothing. (Napoleon Bonaparte) but, I don t recommend Napoleon as a Service Supply Chain partner! Slide 7 About Cisco Services Cisco Services includes a skilled, inclusive, diverse professional bench from technical support professionals and network architects, to application specialists and business consultants. Partners 280,000+ partner employees provide channel leverage Customer Interactive Network (CIN) CIN offers a worldwide, seamless one company contact center managing customer inquiries of any nature. Multi-channel support (phone, , chat and communities) 7x24x365 global support using the Follow-the-Sun model 17 languages supported ~600,000 contacts handled per quarter Technical Assistance Center (TAC) TAC Support Capabilities: 180+ countries, 24 TAC facilities, 17+ languages Engineers average 5+ years industry experience 250,000 service requests quarterly Services Supply Chain Fulfillment depots: ,000 parts delivered annually 140,000 service requests quarterly (hardware replacement) 500,000 parts repaired annually Slide 8 Cisco Services Business Continuity Team This presentation represents the Supplier Business Continuity Risk methodologies employed by Cisco Services and does not include manufacturing or other business functions within Cisco Systems, Inc. Small Business Continuity Management (BCM) team to provide: o Business Continuity Program Management o Business Continuity Training o Subject matter expertise for Services Business Continuity teams o and Supplier Business Continuity Risk Management. Business Continuity Plans are managed using in-house templates and document management controls. What works: Services and service supply chain business continuity risk management Page 2

3 Slide 9 Cisco Services Suppliers Cisco Services employs Suppliers in virtually every business function. There are hundreds of contracts in place with Suppliers ranging from contingency workers to fully outsourced services with annual contract values ranging from a few thousand to multiple millions of dollars. Many of these Suppliers support critical business processes. Note: For the purposes of this presentation Suppliers and Vendors are synonymous. For clarity I have referred to the third-party entity as a Supplier, and the Cisco manager responsible for the Supplier as the Vendor Manager. Slide 10 Supporting Programs Cisco Services has a large global service supply chain network to ensure resiliency of operations and this program augments those capabilities. Other programs within Cisco perform periodic on-site visits, such as o Business teams (Cisco Vendor Managers) o Trade-compliance and other auditing teams These teams can be helpful for o Raising additional areas of concern from site visits o Validating key business continuity elements during site visits o Raising awareness of the Supplier Business Continuity Risk Assessment o Identifying Supplier service outages that may indicate gaps in plans Note: Supplier Business Continuity Risk Management is a defined program to identify and mitigate the risk of a Supplier service outage, but it does not operate in a silo and is integral to the Business Continuity Management (BCM) practices in place at Cisco Services. Take away: Like all aspects of BCM, Supplier Risk Management needs to be engrained in daily operations to be most effective. Slide 11 (untitled) Need: Identify and mitigate the risk for a Supplier Business Continuity event-related service outage with an efficient investment of time and funding. Slide 12 The Solution What works: Services and service supply chain business continuity risk management Page 3

4 Slide 13 Scope Goal: Ensure the continuity of critical business processes One Assessment per Supplier (not coupled to contracts) Excluded temporary worker contracts Scope Guidance: o Minimum Annual Contract Value o Minimum Time Remaining on Contract o Representative sample of locations o Exceptions are made when deemed essential to operations o Cisco Vendor Managers must approve Suppliers removed from scope Tip: Start with a goal statement to assist in defining the scope and addressing questions for exceptional cases within the Supplier BC risk program. Lesson Learned: Be careful in applying the scope to your Supplier List. When consolidating the list of Suppliers you want to be careful not to eliminate critical locations and identify all Vendor Managers necessary to represent the scope of operations. Slide 14 Assessment Methodology Collect Supplier BC information o Key locations o Questionnaire o Evidence files Use tool-generated assessment report to focus efforts o Validate evidence files o Engage Suppliers and Cisco Vendor Managers in a dialog when there are perceived risk Validated assessment score and status Work remediation efforts to plan when appropriate Slide 15 Questionnaire All questions are based on best practices for Business Continuity 5 Corporate governance questions o Does the Supplier have a comprehensive Business Continuity program? 14 Site-specific questions (identical for each site) o Has the Supplier completed a Business Impact Analysis (BIA)? o Does the Supplier have a current Business Continuity Plan (BCP)? o Has the Supplier conducted a BCP exercise within the past year? Questions are not specific to any standard, but are aligned with Disaster Recovery Institute International s (DRII) professional practices, and generally support Business Continuity Institute s (BCI) Good Practices, BS25999, and ISO See Disaster Recovery Institute International (DRII) Professional Practices for more information on Business Continuity Management at: What works: Services and service supply chain business continuity risk management Page 4

5 Corporate Governance Questions: 1. Confirm that the organization has a formal Business Continuity Management (BCM) policy that is (a) approved by senior management and (b) applicable to all sites necessary to support the organization's essential business functions. Only answer Yes if the BCM policy mandates resilience strategies, recovery objectives, business continuity, operational risk management considerations and crisis management plans. 2. Confirm that the organization has established and maintains a Business Continuity Management (BCM) Awareness and Training program. 3. Confirm that the organization has implemented an ongoing program for the exercise/testing of business continuity and disaster recovery plans. 4. Confirm that the organization has implemented an ongoing program for reviewing and maintaining plans on a regular basis. 5. Confirm that the organization has implemented an ongoing program for conducting audits (internal or external) to validate BCM program compliance with your organization's Business Continuity standards. Site Specific Questions: 1. Confirm that a Risk Assessment has been performed and approved by senior management to determine the risks that can adversely affect the organization s ability to conduct business. 2. Confirm that a Risk Assessment has been reviewed and approved by senior management within the past 24 months. 3. Confirm that a Business Impact Analysis (BIA) has been completed that identifies the impacts resulting from a loss of critical business functions/services. Only answer Yes if the BIA identifies business-critical functions, their recovery priorities, their dependencies, and establishes their Recovery Time Objectives (RTO - how long a business function can be down) and Recovery Point Objectives (RPO - how old the data may be when restored). 4. Confirm that a BIA has been reviewed and approved by senior management within the past 24 months. 5. Confirm that the organization has developed and implemented Emergency Response plans and procedures for initial response and stabilization of emergency situations at this site until authorities having jurisdiction arrive on-site. Only answer Yes if the Emergency Response plans provide for the protection of personnel, protection of assets, incident assessment, and incident containment. What works: Services and service supply chain business continuity risk management Page 5

6 6. Confirm the organization has written Incident Management/Crisis Management plans and processes to establish an Emergency Operations Center (EOC)/Command Center with an identified Crisis/Incident Management Team (CMT/IMT). Only answer Yes if the CMT/IMT has command and decision authority to determine the appropriate continuity action(s), and to coordinate and communicate with employees, emergency services, and external agencies as required during an incident. 7. Confirm that this site has written crisis communication procedures that comply with regulatory, statutory and contractual requirements for notification of emergency services, external agencies, and customers. Only answer Yes if the written procedures address communication vehicles (e.g., , radio, messengers, cellular telephones, etc.) to be used during an emergency when normal site communications fail. 8. Confirm that this site has access control processes in place to ensure access is only granted to authorized employees and visitors during business and non-business operating hours. 9. Confirm that the site maintains a backup power source in addition to an Uninterruptable Power Supply (UPS), such as backup power generators, that can sustain IT systems, Security systems, and other essential operations at this site in the event of a power outage. 10. Confirm that this site has identified all Information Technology (IT) necessary for business operations, and implemented an electronic data backup and off-site storage process to sustain the Recovery Point Objective (RPO) for each IT system. 11. Confirm that the organization has developed a Business Continuity Strategy for all critical site functions. Only answer Yes if the strategy supports the Recovery Time Objectives (RTO) and the Recovery Point Objectives (RPO) for the function, and is compliant with relevant regulatory, statutory, and contractual requirements with Cisco. 12. Confirm that the organization has documented, and implemented Business Continuity and Disaster Recovery Plans that provide continuity, recovery, and/or transfer of site business functions to support the organization s requirements including contractual obligations with Cisco. Only answer Yes if the plans provide for plan activation, required resources, vital records, communications, recovery Infrastructure (IT and operational), and include the methods/procedures to recover and/or transfer functions. 13. Confirm that all business continuity/recovery plans necessary to support all essential business functions at this site, including those necessary to meet contractual obligations with Cisco, have been exercised/tested within the past 12 months. Only answer Yes if your exercise included post-exercise activities to remediate issues/gaps identified during the exercise. 14. Confirm that all business continuity/recovery plans necessary to support all essential business functions at this site, including those necessary to meet contractual obligations with Cisco, have been reviewed and updated to reflect any material operating change. Only answer Yes if the plans have been updated within the past 12 months to ensure the plans remain current, accurate, and complete. What works: Services and service supply chain business continuity risk management Page 6

7 Slide 16 Evidence Requirements Evidence is required to support responses and prioritize risk dialogue. Corporate governance questions o The BCM policy Site-specific questions (identical for each site included) o BIA (within 2 years) o BCP (within 12 months) o Exercised (within 12 months) Slide 17 Delivery & Reporting Tools Third-party web-portal based solution to gather data o Cisco Services defined Question Set o Cisco Services defined Evidence Requirements o Tool-generated initial assessment report o Provides a number of reports, that can be exported to Excel Excel spreadsheet o Tracking Cisco Vendor Manager contact information o Track history of interactions with Supplier o Historical tracking of assessment scores and status o In-house reporting Note: Cisco Systems, Inc. is not advocating for or against the use of a specific tool. Our solution reflects the tools selected, and you may need to revise the processes based on your tool-set. Slide 18 Assessment Tool Scoring and Status The tool provides a risk status for each Supplier. The combination of site risk levels, corporate governance risk, and supporting evidence determines the overall Supplier s risk status. Cisco Services uses the risk status to focus Supplier risk discussions. Slide 19 Evidence Validation General Balance the need to verify a Business Continuity program and plans against the needs of an organization to protect sensitive and proprietary information. Generally we accept a minimum level of evidence that a document exist, is current, and meets the standards. Note: Cisco Services does not request evidence outside of the portal, but when Suppliers are unwilling to provide evidence in this matter we are open to other methods. What works: Services and service supply chain business continuity risk management Page 7

8 Slide 20 (untitled) Any negative response or gap in evidence is a reason to open a dialog Slide 21 Risk Dialog with Suppliers Goals of dialog o Collaborate with the Supplier to validate responses and level of risk o Identify gaps in Business Continuity practices or plans o Have Supplier provide timeline to close identified gaps o Track and monitor progress to plan This risk dialog is always conducted by an experienced Business Continuity practitioner as the responses to questions may raise other concerns that require follow-up. Tip: Active listening and open-ended questions that require more than a yes or no response are key to revealing additional risk concerns. For example: Ask where do you store your backup media, instead of do you store backup media off-site? Slide 22 Assessment Reporting and Metrics Monthly and Quarterly risk status reporting to management team. Metrics tracked include: o Assessments completed by Supplier o Evidence validated o Evidence gaps identified and closed o Risk status improvements Slide 23 Assessment Reporting and Metrics Sample report What works: Services and service supply chain business continuity risk management Page 8

9 Slide 24 Supporting Contract Language Vendor shall implement and maintain a business continuity program designed to ensure the continued availability of essential business functions during any event that would otherwise materially affect Vendor s ability to deliver services. Vendor shall implement and maintain a Business Continuity Plan ( BCP ). The BCP shall be documented in writing and shall include, without limitation, evidence of a Business Impact Analysis (BIA) that identifies essential business functions and establishes their Recovery Time Objectives (RTO), Crisis Management Plans (CMP) to coordinate and communicate appropriate continuity actions, and Disaster Recovery Plans (DRP) for all essential business functions necessary to meet contractual obligations with Cisco under this Agreement and all SOWs. The BCP shall be reviewed, revised and tested/exercised by Vendor at least once every twelve (12) months. Vendor shall provide evidence of the BCP within ten (10) business days of the Effective Date. Vendor agrees to complete a Vendor BCP assessment using the tools and processes prescribed by Cisco within thirty (30) days of a written request, not to exceed two (2) assessments within a twelve (12) month period. Slide 25 Lessons Learned [Title Slide] Slide 26 Lessons Learned A significant effort will be required in the first quarter. We had to assist 80-90% of the Suppliers complete the assessment. Validating evidence files for self-reporting low risk Suppliers is as important as validating higher risk categories. Evidence gaps in 70-80% of assessments over the past 2 years. Adjust risk status when evidence files do not support a response. Supplier risk metrics will drop as you validate evidence prepare your management team for the changes in status. Note: Assisting Suppliers, includes follow-up communications in addition to tool generated messages, meetings to clarify questions, providing additional accounts for the Suppliers, and other assistance as requested. Tip: A Supplier may postpone providing an evidence file indefinitely, but when you change the status to High Risk and notify the Supplier of the change the need to provide supporting evidence tends to become a priority and they work with a greater degree of urgency. Slide 27 Questions and Answers What works: Services and service supply chain business continuity risk management Page 9

10 Slide 28 Summary Gather Supplier data to obtain an initial risk status Use the risk status to focus efforts on highest risk Suppliers first Validate all critical Suppliers including low risk Have an open risk dialog with the Supplier Keep the Vendor Management team informed and engaged Slide 29 Results Achieved Sample reports to compare FY12 to FY13 Cisco Services achieved 64% Supplier Business Continuity Risk Improvements in FY13 Slide 30 Thank you. What works: Services and service supply chain business continuity risk management Page 10