Operational Risk Indicators By Martin Davies November, 2006 Ωausal Ωapital

Transcription

1 Ω Operational Risk Indicators By Martin Davies November, 2006 ausal apital

2 What we do in the Operational Risk Team Brief overview of our services Quantifying Exposure Financial Management All Operational Risk solutions and requirements addressed Bestpoke development or implementation of interactive and holistic operational risk software Operational Risk CC-Impellent is a full suite of operational risk tools that allows the bank to reach AMA compliance. It is an entirely web based integrated solution that will operate on any client and is able create a parametric view of risk that is statistically justified through internal related assumptions. CC-Impellent also allows companies outside the financial sector reach additional benefits past standard SOX regulation and is one of the leading SOX tools. Managing Risk Operational Risk Auditing Systems, Policy Design, Consulting, Governance Reporting, Regulatory Management & end-to-end risk service Correction Action Management & Control Improvement The provision of transparent risk capital systems Operational Efficiency Improvement Programs to assist banks improve their operating positions AML & Fraud Enterprise Solutions for Fraud Prevention and Regulatory Compliance with Anti Money Laundering Consulting & Advisory Enterprise Risk and Operational Risk Advisory Training & Workshops & Conferences & Risk Workflow Design Risk Appetite Policy Design Governance Reporting Business Process Redesign & Change Management Capital Arbitrage Projects Business growth and facility restructure Core Banking and process alignment Diverse Projects We have some very interesting & diverse projects in South East Asia Areas of Banking Retail Banking Insurance Investment Banking Retail Brokerage Merchant Facilities Private Banking Capital Markets Tier 1, 2 & 3 banks

3 INTRODUCTION» overview of indicators from a risk perspective

4 PART I Definition of Key Risk Indicator What is a Key Risk Indictor Operational risk indicators are measures that enable the bank to identify potential losses and losses before they happen. Why is it important for operational risk From an academic perspective operational risk differs from other risk categories in two key ways: Firstly it is exogenous in nature and secondly it lacks Subadditivity. In practice this can be translated in the following way; An operational event may have many causes which are internal and external to the organization. The actual impact from one of these events each time it is captured can be highly volatile. This creates a problem for risk analysts who are trying to refine a good measure of exposure while battling the causes, catalysts, contributing factors and pathogens for failure. The indicator is a likely solution to this problem Firstly indicators can be used as a measurement and management technique, driving change while monitoring its outcome. Indicators reduce the measurement proximity between causes and impacts allowing corrective action to be selected before failure. Indicators can be used to estimate both frequency of events and their magnitude. Indicators are contextual in nature and are easy to align to business unit activities. They are also transparent measures that can be used in all parts of the bank. Indicators are palatable to humans because the exist in so many other circumstances, from thermometers to tachometers.

5 PART I Regulator Stance on Indicators AMA and Basel II To qualify to use the AMA approach to calculate operational risk capital under Basel II, the Basel committee has specified detailed criteria for the use of forward looking measures. Paragraph 676 specifically relates to these scenarios & indicators. Tested Over time, the process and the outcomes need to be validated through comparison to actual internal loss data, indictors form part of the capital testing process and assist highlighting model errors. Paragraph 676 Pillar I Basel II accord The choice of each factor needs to be justified as a meaningful driver of risk and whenever possible, the factors should be translatable into quantitative measures that lend themselves to verification. Qualified The supporting rationale for any adjustments to empirical estimates must be documented and subject to independent review within the bank and by supervisors. Justified Driver The choice of each factor needs to be justified as a meaningful driver of risk, based on experience and involving the expert judgment of the affected business areas. Indicators are attached to causes to fit this goal. Sensitivity The sensitivity of a bank s risk estimates to changes in the factors and the relative weighting of the various factors need to be well reasoned. Indicators can be correlated to attest this type of sensitivity. Factoring Change The framework must also capture potential increases in risk due to a greater complexity of activities or increased business volume. Indicators furnish this information as they are forward measures.

6 Framework Definition Ω PART I The Indicator in the Measurement Framework Components working Together A true operational risk framework comprises of several diverse components that are interconnected to furnish senior risk management with an accurate picture of the exposures presented at a business unit level. The Key Risk Indicator View Definitional Forward Looking Measures Historical Measures Risk Registration Risk Control Assessment & Scorecards Governance Reporting All components within the framework are aligned by a set of definitions. These definitions form the basis for the creation of the taxonomical risk structures of which all other items are aligned. Indicators are connected to business objects through a process that draws on mapping. Mapping Process Control Product Network External Loss Data Internal Loss Data Scenario Analysis Indicators Capital Measurement Indicators can be validated statistically for an understanding of their inference value on loss. Key Risk Indicators form part of the supporting qualification process for governance reporting and benchmarking performance over time. They can be used in every part of the bank. Indicators are used to identify which scenarios are in play and which ones have been parked. Maturity No item operates in a vacuum.

7 PART I The Indicator in the Operational Risk Value Chain Value Chain The measurement of operational risk has a specific cycle that is perpetually in play. 1) Risks & boundaries are defined so that measurement accurately represents the cause and location. 2) Risks are identified as per the definition. 3) These exposures are measured. 4) The exposures are qualified. 5) Exposures are reported. 6) Risk is controlled. 6 Control 4 Gov Report 1 Define Objectives 2 Identify Risks As exposures are controlled the 3 Quantify Exposure institution might adjust its definition to either include a larger range of 4 Qualify Exposure exposures or more granular detail of potential events. Indicators play an important part throughout the process. They form part of the definition objective and are used to prove capital results at the end of the quantification process. They are attached to controls and drivers of risk to directly identify whether an event is about to occur and are used to supplement data to assist in the qualification process. They form a major basis of the governance report to allow business unit performance to be measured over time and for cyclical patterns to be understood. Finally if indicators hooked to controls or are used to drive policy and in that sense they form an integral position in the control gap between event and impact.

8 PART I Different Techniques of Exposure Quantification There are three industry accepted standards for quantifying operational risk MANAGEMENT TOOLS For Operational Framework STREAM 1 - Loss Incident Tracking STREAM 1 - Volumes Stats MEASUREMENT APPROACH For Capital Charge LDA STREAM 2 - Foundational Mapping STREAM 3 - Risk indicators & Controls STREAM 4 - Scenario Planning Optional - Balanced Score Carding KEY DRIVING COMPONENTS sbama RDCA If a financial organisation was to select the Advanced Measurement Approach for defining its exposure and calculating its Value at Risk there are also three styles that are accepted industry wide. Loss Data Approach puts emphasis to the computation of capital on historic loss data. Standard statistical techniques such as those that have been used by the insurance industry for years as well as some of the more complex derived functions including extreme value theory are used to compute regulatory capital. Estimation of exposure is usually performed on the frequency and magnitude of event distributions separately within the Basel event classifications, the results are then aggregated for a clear dimension of Value at Risk. Risk Drivers and Controls Approach (formerly known as the scorecard approach) uses a series of weighted questions (some of which can be interpreted as scenarios) whose answers yield a score that can be aggregated to allow the calculation of capital between business units. sbama RDCA LDA The Scenario Based Approach is based on the assessment of forward-looking what-if scenarios. The output of the scenarios are entered into an operational risk model where regressive techniques such as Monte Carlo are used to compute regulatory capital.

9 PART I Indicators in the Capital Model The End Goal The end goal for the capital model is to be able to parametrically create a probability loss distribution function for each risk category, line of business and product of the bank. This probability loss distribution function is representative of total exposure for the bank and is made up of two parts, an expected loss component and unexpected loss component often referred to as the tail. Expected Loss Unexpected Loss Solvent Insolvent Loss ƒ P Loss Indicator Correlation Factor Investigate Reduce Indicator is showing scenario is out of play or reduced PE Y1 The indicator is used to identify data elements that may be anomalies, in error or showing a regime shift Y2 If a scenario is moving away from probability of event, should we be holding capital for it? Causal indicators help identify if the scenario is in play.

10 PART I Types of Indicators Process Centric View There are various types of indicators that the bank will use to assist in quantifying & qualifying its exposure and these can be found in many parts of any process workflow within the organization. (i) Volume / Value Trends Change of policy (i) Driver present Driver 1 Product Process Risk Driver 2 (i) Service Level Agreements Control 1 Control 2 (i) Control failure breach count Environmental Indicators Some indicators will be captured but don t directly related to a risk however they describe the environment so that a dependence factor can be statistically created to allow a capital model that is more closely aligned to the business units. Measures of failure Some indicators are direct measures of fault however even still that may not result in an impact. Triggers for event management Tetra-choric indicators or variables that translate as decision markers might be used to effect policy.

11 INDICATOR LIFECYCLE» creating & deploying indicator programs

12 PART II Indicator Phases in the Capital Model Phase 1 Phase 2 Phase 3 Phase 4 Phase 5 Phase 6 Indicator Definition Indicator Scoping Indicator Generation Indicator Amalgamation Indicator Collection Validation & Reporting Definitional and taxonomical structure created to connect risk classes to model Business unit investigates its causal network to identify indicator points Business unit nominates indicators for internal operations and escalates opposing value chain indicators Business unit & group risk check for duplicates and crossovers. Duplicates are prioritized Indicators are collected in the format that they have been defined in and in the frequency outlined in phase 3. Indicators positions are reported over time and compared internally. Auto regression may be used for predictor values. Ensures transparency Business unit identifies potential transparent points for indicator capture Business unit indicators are listed. Corrective actions may also be listed during this phase. Indicators are consolidated Indicators are transmitted to central repository Improvements are suggested. Risk positions are reported to the business unit for evasive action Factor values sent to the capital model and Improvements are reviewed.

13 Org Mapping Business & Output Layer Fraud Technology Suppliers Compliance Volumes Subjects Contracts SLA Project Layer Ω PART II Phase 1 Indicator Definition Engaging Indicators Indicator definition represents the first step into constructing a solid and transparent measurement network throughout the bank. 1 Build on definitions The org process/product maps and policies are reviewed 2 BU Facilitation All risks reflecting the business profile are listed by management experts 3 BU Inputs Context On the basis of indicator classes, business experts determine individual case Consider the capital model use to define the scope Risk Classes Documentation of the approach is constructed and business units contacted A common set of indicator classes is applied to different parts of the institution A common set of indicator classes is applied to different parts of the institution 2nd Classes NOTES This component is more successful if mapping is present and the business units are given clear guides on potential and group wide indicators sets. Identify top level risk categories at a business unit level TIE Construct a program to assist business units Identify other possible sources of environmental risk categories GOAL Phase 1 Determine approach for all material risks. Organization Part Organization Part Organization Part Organization Part Ensure business units are able to contribute to the Indicator class stage. Ensure the process is transparent across the bank for generic classes

14 Org Mapping Business & Output Layer Fraud Technology Suppliers Compliance Expertise Controls Contractual???? Business 1 Business 2 Project Layer Ω PART II Phase 2 Indicator Scoping Enabling Indicator Definitions Once the indicator classes have been generated, they have to be evaluated in the context of the business unit. Each indicator in turn is reviewed in the context of business unit specific activity. 1 Draw on framework Review the class structure set in phase 1 in business unit context. 2 BU Facilitation Business describes what data is used & how to link indicator to risk causes 3 BU Inputs Context Supporting notes are created with indicator lists and the cause network Business unit reviews processes, controls, products etc for potential indicators Suggested frequency type and destination of capture in context of causal network Changes to classes are logged, suggested indicators are listed NOTES Good definitions and clear material on how to review business unit activity is critical for the success of this phase. Risk Classes 2nd Classes Organization Part New classes logged GOAL Phase 2 List indicators in context of business activity. Organization Part Indicators, SLA s Suggested along With frequency Trap causal reasons + changes to structure

15 Business & Output Layer Org Mapping Project Layer Ω PART II Phase 3 Indicator Generation Generation of Final Indicator List Once the indicators have been listed and attached to the causal components of a business unit, where indicators stretch across business unit boundaries or have drivers from other parts of the value chain, those parts are investigated. 1 Draw on framework Collected and present suggested indicator items & values to business unit. 2 BU Facilitation Business unit describes adjusts dimension values or breaks cases down. 3 BU Inputs Context The quality of the indicator is tested by group operational risk. Review assessment factors, notes & assumptions for open debate. Other business units may be consulted to supply additional information. Rejected indicators are closed & reasons for rejection kept & reviewed later. NOTES The ability of the business units to openly discuss potential value chain issues and SLA results in the success of this phase. Track third parties Organization Part Organization Part Review SLA s Review in the context of the value chain. Business facing indicators GOAL Phase 3 Identify value chain Indicators. Include causal flow in value chain indicators

16 Sets Business & Output Layer Business 1 Business 2 Project Layer Ω PART II Phase 4 Indicator Amalgamation Network Simplification With all the indicators defined, there may be several cross over s or indicators that don t closely relate to an exposure. This phase will qualify the indicators and may draw on data to describe potential correlation proximity to a risk. 1 Draw on framework All previous material is collected and assimilated into the model Specific data points that draw on material might request collection of data to correlate NOTES 2 BU Facilitation Data for indicators in conduits is described and quality assessed 3 BU Inputs Context The model is aligned in the business capital lines and closed Duplicate indicators are flagged for rejection analysis & drop difficult indicators Indicator quality is recorded and the indicator set is commenced. The ability to consolidate data in a homogeneous fashion will result in a pure model. Rejections need to be documented. Confirm quality GOAL Phase 4 Set 1 Set 2 Set 3 Remove 1 of duplicated sets Estimate potential correlation for critical indicators Rate Indicators Ensure pure list of Indicators is created. Final validity check, anomalies are referred to business owner.

17 Business & Output Layer Event Categories Project Layer Ω PART II Phase 5 Indicator Collection Trapping the Key Risk Indicators With all the indicators listed and their frequencies defined, the business units can move into an operable state of collecting the indicators as indicated and for each individual indicator cycle. 1 Draw on framework Indicators are fully defined and are presented to staff 2 Capture Indicator Business unit managers review the indicator question and answer. 3 Improve & Align Report specific indicators that are difficult to capture or suggest better gauges. On cyclical frequencies indicator questions automatically remind staff. Indicator data is sent back to the group and the central risk division of the bank. Business units may suggest new indicators to their central risk functions. NOTES The correct facilitation of the other phases of the indicator project result in a easy indicator program to manage. Single Business Unit Indicator Questions Question 1 Question 2 Question 3 Question 4 Indicator values transmitted to group operational risk GOAL Phase 5 To collect operational risk indicator variables. Question 5 Promote continual improvements To identify potentially new & more accurate indicators

18 Business & Output Layer Project Layer Ω PART II Phase 6 Indicator Validation & Reporting Dashboards and Reports The power of indicators is realized when business units are able to take a specific action because an alarm has raised their awareness that an operational risk is in play. 1 Review Outputs Group operational risk & central risk departments review dashboard 2 Act on Results Central risk departments monitor changes and advise where needed. 3 Engaged Framework Business units review correlation factors for health of indicator Central business units act on specific results raised by the business units Business units look for trends and regime shifts & give advice to central risk Poor indicators are flagged for review at a periodical consultation phase later. Regime Shift Monitor NOTES The business units ability to take action assumes that they have tracked what action to take when an indicator or combination of indicators reach specific thresholds. Different thresholds respond in different actions. Cycle Planning Periodic Benchmarks Alarm Actions List Correlation Factoring Business Unit 1 Business Unit 2 Business Unit 3 Scenarios in play GOAL Phase 6 Ensure business units are aware of their performance Operational risk able to see the health of the bank in a single dashboard.

19 LAUNCHING» details in commencing the program

20 Has reach & life Is numeric Active or Passive Has frequency Risk Category Causal Factored Activity Attached Describable Supports Decision Owned Benchmark-able Simple Ω PART III Phase 1 Indicator Definition What Makes a good indicator Before business units go out and hunt for indicators at a very top level there must exist a definition of what indicators are acceptable. Contextual Qualified Useable 12 State acceptance factor Top Qualities Contextual Indicators need to be able to be attached to a risk category They need to directly reference a breach, loss or driving factor for an event They are often attached or related to an activity The causal relationship between the factor and the event must be describable Top Qualities Useable Good Indicators generally support decisions The risk, exposure and indicator all need owners They can be compared over time and used as benchmarks The relationships are simple and the indicator easy to see Top Qualities Quantified The indicators proximity to event and its life can be defined The indicator has precision and is not subjective & is a number Its relationship with the risk can described as a warning or driver Its frequency is known so that it can be collected in a timely fashion

21 Processes SLA s Technology Third Parties Fraud Technology Suppliers Compliance Ω PART III Phase 2 Indicator Scoping How to find indicators With the definition aligned from facilitation with the business, managers can set about locating quality indicators in their processes. Stakeholder Needs Regulations & Policies Losses & Incidents Strategies Internal / External Clients Procedural Policies Actual Losses Department Plans / Scope Risk Classes Vendors and Suppliers Management Regulation Standards Legal Environment Near Misses or issues External Data Environmental Cycles Department Vision Dependencies With the results of phase 1 available and generic indicator sets and definitions located, the business unit will review its operations in the context of each risk event classification. It will consider dependencies in the context of its operating environment and take in the full scope of the industry; including experiences from peer banks and legal requirements imposed by regulators. Risk Assessments Scenario Analysis Control Assessments Best Practice

22 PART III Phase 3 Indicator Generation Finalising the indicator allocation process Business units nominate their indicators to the group operational risk department but assess quality first. Process Process Process Decision Process Contributing Factor Risk Control Indicator Indicator Indicator Business Unit 1 Business Unit 2 Indicators through the value chain Indicators that are nominated by one business unit on a risk that is owned by another business unit are referred to this alternate location for approval. Indicators are discussed with peers and staff associated with collection to ensure they are useable. Indicators are described and their frequency discussed internally. Indicators that suffer poor description may be listed and placed in an inactive state. The ability for an indicator to assess a magnitude or frequency of event will be discussed. An individual indicators estimated correlation factor may be included in the description. Where more than one indicator exists for a risk, priorities may be given. Supporting reference material including external data points may added to the description. Thresholds upper and lower may be applied. Expected values may be offered as part of the information pack

23 PART III Phase 3 Indicator Generation [Generic Indicators] Information Technology - outages/disruptions (#, duration) - support/helpdesk calls (#, duration) - project milestones missed (#, cost) - business continuity incidents (#) - emergency change releases (#) - IT security investigations (#) - monitoring violations (#) - super user accounts (#) - firewall changes (#) - virus incidents (#) Finance Department - incomplete p+l sign-offs (#, aged) - daily p+l adjustments (#, amt) - off market trades (#, amt) - unsubstantiated balances (gross amt) - interco y mismatches (amt, aged) - mgt-fnl a/c differences (gross amt) - reporting deadlines missed (#) - regulatory capital buffer (amt) - fixed asset write-offs (amt) Human Resources - headcount (#, grade, svc yrs) - leavers/joiners (#, grade, reason) - vacancies/atrs (#, grade, aged) - absence/sickness days (#, grade) - dismissals/disciplinary cases (#) - discrimination lawsuits (#, cost) Compliance & Audit - o/s litigation cases (#, amt) - money laundering reports (#) - compliance investigations (#) - PA dealing violations (#) -customer complaints (#) - No of internal breaches resolved

24 PART III Phase 3 Indicator Generation [Trading & Sales Business Line] Reporting Function Examples of KRI Measures or Issues Front HR IT Ops Risk Audit Finance Admin Svc Compliance Legal New business initiatives, projects, plans Headcount, vacancies, turnover, disciplinary cases Downtime, security breaches, change releases, project delays Volumes, static updates, open items, breaks/amends, investigations Model approvals, policy/limit breaches, late credit reviews, re-ratings Audit coverage, open/late issues, revised targets, repeat comments Capital ratio, p+l corrections, deadlines missed, breaks, tax issues Space usage, BCP invocations, SLA breaches, T+E violations Money laundering cases, PA dealing violations, investigations Litigation cases, new legal developments/exposures

25 PART III Phase 6 Phase 6 Acting on Reporting Quick SPC Tip A body known as Statistical Process Control (SPC) which promotes a structured method of sampling results. Every process is subject to some variation due to common causes outside the control of those managing the process itself. If a process is in statistical control it exhibits results that fluctuate around a mean performance. The SPC attempts to prove that while these fluctuations may not be normally distributed, they may still follow a trend and by monitoring such trends they are able to develop early warning signs SPC managers have some general rules of thumb: The SPC Alarms For processing areas 1) A single outlier beyond three standard deviations, if a process results are normally distributed, such events occur usually in about 370 trials and must be investigated. 2) Two out of three consecutive points beyond two standard deviations in one direction. 3) Four out of five points beyond one standard deviation in one direction. 4) Eight or more points on one side of the mean regardless of how far removed. 5) Six or more points with a common trend (that is five or more consecutive first differences of the same sign). 6) Fourteen or more points that oscillate up and down. This may be related to change of shift and sampling must be done. 7) Eight or more points beyond one standard deviation in either direction. Avoiding the centre of the distribution may indicate a new and previously unrecognised source of volatility. 8) Fifteen or more points within one standard deviation. Signals are not always bad news. An unexpected string of results within one standard deviation may indicate some favourable improvement in the control process that can be isolated and replicated elsewhere.

26 PART III CC-Impellent Workflow Process Identify risk categories to be monitored Identify the risk drivers Transform risk drivers in measures Define measure as KRI Collect & analyse data Follow up Collect KRI values and comments Approve KRI values and comments Analyse KRI values and comments Determine actions Follow up on actions Assessor Approver Risk Controller Management Risk Controller Request for approval Close Reject

27 PART III CC-Impellent Workflow Roles Workflow for Key Risk Indicator collection Reader read Approver / Delegate Assessor / Delegate Coordinator set up KRI Definition generate KRI instance open request for approval fillin fillin request approval rejected reject approve closed informed if alert or critical level has been exceeded Risk Controller / Management identified Risk Driver day to day Management actions

28 About the Author Martin Davies Martin Davies is a managing partner within the business solutions competency at Causal Capital. He specializes in designing operational risk measurement and management systems with a particular focus on Basel II and regulatory capital frameworks. He has more than 10 years experience developing bespoke knowledge / workflow and scorecard solutions for financial institutions in both strategic and processing areas of the business. At the Commonwealth Bank of Australia he contributed to the risk program by running the demarcation project for measuring arbitrage between credit and operational risk and a recent engagement saw him with group risk at Suncorp Metway, where he was pioneering the design of the AMA capital framework. Martin was recently driving subject matter resource at Capco in Singapore but has moved on to assist local banks with their needs directly and importantly to be directly involved in the metamorphosis of the CC-Impellent Operational Risk Engine. He is able to ensure that cost effective solutions are delivered to banks in South East Asia in a timely and quality manner. Currently he specializes in aligning banks to the operational risk solutions from Causal Capital and works with loss data, control assessments, indicator systems and scenario analysis. Banking Experience He has experience across the entire retail banking space from loans to transactional accounts both in the front and back office areas of a division and has worked with initiatives such as delinquency management on revolving lines of credit. He also has exposure to both business lending, restructuring as well as with investment products specifically in re-designing process workflow and business measurement metrics to ensure a selected product and customer portfolio is delivered to a channel in the most efficient and cost effective manner. Academic Involvement Martin is proactively entertaining divergent operational risk theories / techniques, has published several papers and delivered speeches on the use of quantification instruments for risk with a good focus on managing fraud, anti money laundering and business continuity. Recent interest envelopes the ability to dimension type II curves, particularly for use in economic capital models. Additionally he is keenly researching the study of the causality effects of pathogens on complex neural networks and tying those structures to control frameworks or economic capital lines within financial institutions. PRMIA Last month he was published by Basel Alert, a UK risk magazine and Business Continuity in Australia. He recently chaired the Op Risk Congress in Singapore and he is an operational commentary journalist for PRMIA (the global risk organisation) and you can access his journal at the following address:

29 Ω Ω ausal apital THANK YOU» respecting your time