How to measure your business resiliency

Transcription

1 How to measure your business resiliency Define the KPI s/kri s and scorecards to control your security and business continuity capabilities Krzysztof Pulkiewicz BCMLogic krzysztof.pulkiewicz@bcmlogic.com

2 Abstract Business Continuity Management is the process, not just a one-time project activity. In order to control the alignment between the BC plans and business as usual as well as synchronize the changes, it is required to setup the scorecard based measurement process. The set of KPI's and KRI's is aimed to visualize the maturity of BCM, risk vs. lost metrics and level of protection mechanisms against the business requirement. I will present the business resiliency scorecard framework with special focus on the methods of data gathering and integration with IT infrastructure landscape.

3 Agenda Why to measure? What to measure? How to gather data? How to present results? How to do it in practical way? Key takeaways

4 KPI primer KPI/KRI fundamentals Key performance indicator(kpi) is a measure of performance, commonly used to help an organization define and evaluate how successful it is, typically in terms of making progress towards its long-term organizational goals. Key Risk Indicators Measures are used to indicate how risky an activity is. KRI give us an early warning to identify potential event that may harm continuity of the process From row data to metrics Details raw information Metrics are refined data KPIs are metrics with businesscontext Business context makes security relevant A Key Performance Indicator Must be something that can be measured and continued to be measured Must be precise, meaningful and understandable Must be relevant to the business May be required by legislation and/or Regulations Must have a measurement index that has meaning Should be tied to the organization s vision and strategy

5 Why to measure? You can control only the things you can really measure To understand the overall readiness level of your company To plan and assess the risk based on the statistic and past experience Executives love metrics and dashboards. Always time-constrained, they ask for metrics that can be reviewed at a glance Money talks - especially when you speak with your CFO C-level managers are used to percept from KPI s- give it to them Justify your security investments based on the measurable objectives The KPIs can be used to help comply with legislative or regulatory requirements

6 What to measure? Recoverability Planning Compliance Technology BC project Can our organization be recovered within our tolerance for downtime? Status and results of planning activities Regulatory and audit point of view What are the IT risks and Disaster Recovery capabilitets? Project management based reporting Incidents statistic How did we react? Testing execution times vs. RTO requirements BIA overview Risk assessment results Processes covered by BCP BCM Maturity Compliancy level Risk assessment results IT infrastructure IT services SLA Service Desk Business RTO/RPO vs. technology capabilities Milestones passed Financial spent FTE effort Work (time/mdto complete Overdues Minimum operational teams

7 BIA overview Tactical view on the BIA requirements # of Department- Business Impact Analyses vs. Total Expected # of Department- Table Top Exercises Completed vs. Total Expected # of Supplier Business Continuity Assessments Completed vs. Total Number of Critical Suppliers Ten top processes (based on criticality score) Most critical assets RTO/RPO distribution Estimated financial loss overt time Example BIA dashboard (BCMLOGIC.COM software suite)

8 BIA per business unit Tactical view on the BIA requirements defined at the level of each business unit Financial impact over time Time wise One day stop Reputation impact Formal and Legal impact Number of scenarios affecting the business unit Required assets (MAC) Minimum operational team vs. total unit headcount Critical processes vs. all processes handled by unit Radar chart allows to visualize benchmarking Business Unit X Criticality Business Process BIA Updated Plan Updated Tested 1 Call Center customer support Yes Yes Yes 2 Accounts Payable Yes No No 1 Liquidity managment No No No Criticality Business Process Business Unit X Recovery Objective Recovery Capability (based on most recent test) 1 Call Center customer support 4 hours 8 hours 4 hours 2 Accounts Payable 2 hours 1 hour 1 hour 1 Liquidity managment 2 days 2 days 0 Gap

9 BIA per business process Criticality level defined for each process Prioritize the BC process list based on: Business impact when interrupted Vulnerability of underlying assets Risk level Rank processes criticality among different business units/ entities

10 Impact High Risk assessment/ Risk appetite Defined scenarios are depicted based on the probability and impact Example scenarios Main office building unavaliabilkyt Data Center outage Example scenarios Data Center AC failure WAN link down PLAN ACCEPT AVOID/ ELIMINATE MONITOR Example scenarios Short power outage Modarate staff absence Example scenarios IT infrastructure production load pick performance downgrade Low Probability High

11 Readiness level BCM management objective Proper crisis situation management (incident management, start-up and implementation of the recovery tasks) Eliminating the potential effects of process interruption Providing processes resume after the crisis situation Continuous development and improvement of BCM Example KPI Number of reported incidents The ratio of the risk response plans for scenarios of potential threats Number of incidents that were not closed before crossing the processes RTO that are related to. Incidents can be divided into: - incidents that concerned the processes associated with financial impact - incidents that concerned the processes associated with reputation impact - incidents that concerned the processes associated with law impact The ratio of recovery tasks completed successfully for all recovery tasks Number of performed BCM tests The ratio of the number of BCM plan tests completed successfully for all BCM plan tests at this time Number of risk which probability or potential impact was reduced after implementation BCM corrective tasks To report the progress of BCP project: How many process have contingency measures How many scenarios are planned How many solutions tested

12 IT services management Monitor and visualize critical service conditions Example BIA dashboard (CONTINUITYSOFTWARE.COM)

13 Example KPIs Typical examples of BC KPIs that can be included in a BC Policy Document: Level of disruption response/recovery time Time to detect disruptions Time to trigger action to disruption events Time to complete recovery action Time to declare `business as usual' Level of business continuity testing/exercising/audit Level of service delivery and quality acknowledgement by clients? Level of knowledge of business continuity awareness/acceptance/culture Level of availability and/or knowledge of alternative fall-back to critical resources (human/ technical/ location) Level of effectiveness of Service level agreements

14 Effectiveness of Investment KPIs can be used to measure the Effectiveness of Investment (EOI) A Return on Investment (ROI) for business continuity is difficult to measure since risk, and especially risk reduction, is challenging to quantify in terms of money The Effectiveness of Investment (EOI) could be the comparison of the effectiveness of the resiliency measures with the value of the investment Proper KPI/KRI reporting may also provide a financial institution the ability to reduce the percentage of reserve required to offset operational risk defined by the Basel II, Solvency High Cost Equilibrium Loss Risk Low Protection High

15 How to gather data? Methods Retrieve information from IT systems Data base interface (direct or ETL) API Gather information from people Automated forms workflow (reporting) Sources Service Desk system (incidents, time to resolve) IT infrastructure monitoring (alerts, up/downtime, service level) BPM (process effectiveness) PMO (project reporting) Call Center People (line managers)

16 How to visualize? The reporting mechanism must support three purposes Highlight or alert whether business expected targets are being not met Provide trending and an overview of performance indicators Provide details that pinpoint which areas within each performance indicator require actions

17 KPI reporting audience Each audience may require different information and different presentation C-level Managers Business units IT Value at Risk Complikancy level BC scope and cost Business continuity events IT service availability E2E process SLA Customer service level Customer service downtime IT infrastructure failures MTTR RTO /RPO DR testing Critical services incidents The level of aggregation and or abstraction required may vary considerably You may not want to talk about number of backup site workstations to the Chairman of the Board! Don t assume that the higher the level, the simpler the presentation

18 How to do it in practical way? Define (smart) KPI Identify the KPI stakeholders (RACI) Understand where information resist Leverage the available data to link the KPIs to other key operational metrics that include both technology and process metrics Integrate with other systems and applications Use existing reporting capabilities to establish periodical reporting You can use MS Excel or one of the specialized tools Share the information across the organization Make the KPIs actionable

19 KPI reporting mistakes Lack of management commitment Measuring too much, too soon Measuring too little, too late Measuring the wrong things Imprecise KPI definitions Using KPI data to evaluate individuals Using KPI to motivate, rather than to understand Collecting data that is not used Lack of communication and training

20 Key takeaways 1. Define measurable objectives of the BCM process 2. Utilize existing data sources 3. Learn from the past 4. Align the results presentation to the audience 5. KPIs can be used to help comply with legislative or regulatory requirements

21 Questions