SCHOOL OF FINANCE AND ECONOMICS

Transcription

1 SCHOOL OF FINANCE AND ECONOMICS UTS:BUSINESS WORKING PAPER NO. 141 MAY, 2005 A Test of the Strategic Effect of Basel II Operational Risk Requirements on Banks Carolyn Currie ISSN:

2 A test of the strategic effect of Basel II operational risk requirements on banks Dr Carolyn Currie 1 Abstract Most problematic of the Basel II capital adequacy requirements is the subset of Pillar I, requiring provision for operational risk (OR) as distinct from credit and market risk. Previous tests of the strategic effect of this new regulation from three prior Quality Impact Studies (QIS) conducted in G10 countries under the guidance of the Bank for International Settlements, have concluded that OR requirements poses difficulties of definition, implementation, and strategic planning. Anticipated strategic effects include dramatic changes to product development, investment and asset mix, as well as the necessity to rapidly develop new risk rating models and techniques, together with vastly expanded internal and external audit compliance routines. Unlike QIS1, 2 and 3, QIS4 focuses on operational risk, but still has drawbacks. This paper discusses its approach, in view of the ongoing difficulties that banks are experiencing with operational risk, particularly in the construction of a database. It concludes by listing the unanswered questions that have not even been addressed in four studies of the strategic impact of Basel II s OR requirements. It also suggests that many smaller banks and emerging nations may not be able to use the sophisticated approaches and hence will suffer a competitive disadvantage. Hence in view of drawbacks in the simpler approaches such as lack of correlation of operational risk and revenue, other indicators such as the standard deviation of efficiency measures are suggested. JEL Classification: E42, E44, E58.Key words: operational risk, Basel II. 1 School of Finance and Economics, University of Technology, Sydney, Kuring-gai campus, PO Box 222, Lindfield, NSW 2070 Australia Tel: ; Fax: address: Carolyn.currie@uts.edu.au. Draft paper prepared for the Global Finance Conference, Dublin, Not to be reproduced without author s permission 1

3 1. Background The current Basel II "settings" for credit and operational risk are based on previous Quality Impact Studies (QISs) and some strategic negotiating by the regulators who drafted the document. 2 Several member countries decided to conduct a further national impact study or field test during 2004 or 2005, known as QIS4, which is expected to throw up worthwhile information at the national level about the impact of Basel II on individual countries. These exercises do not represent a joint effort of the Basel Committee on Banking Supervision, and the details vary significantly across countries. Hence the Basel Committee will have a difficult time drawing global comparisons as countries are able to use their own formats, and these will not necessarily be comparable. Nevertheless, the Committee's working group on Overall Capital and Quantitative Impact Studies prepared templates to support these national exercises first, a questionnaire in the form of an Excel workbook and second, corresponding instructions that specify how to complete the questionnaire. In contrast to earlier exercises conducted by the Committee, it is expected that national supervisory agencies intending to carry out an impact study or field test adjust the workbook accordingly to reflect the particularities of the implementation of the revised Framework in their respective jurisdiction. Similarly, the instructions provided only discuss technical issues related to the workbook and would have to be adjusted in order to reflect the changes to the workbook template national supervisors made. They are not intended to interpret the revised Framework. All guidance on issues related to implementation 2 As part of the second quantitative impact survey, the Committee conducted its first survey of operational risk data in May The data collected in that survey and in the 2002 exercise was designed to allow for the further calibration of the Basic Indicator and Standardised Approaches, and to inform the development of the Advanced Measurement Approach (AMA) framework, in particular, resolving issues concerning the qualifying criteria for the AMA. The Committee envisaged that these surveys would be part of an on-going data programme undertaken over the next few years to further refine the calibration of the operational risk charge. 2

4 and interpretation of the revised Framework within a certain jurisdiction which might be necessary to complete the questionnaire will be provided by national supervisory agencies. Although the exercise will be some improvement over the results of QIS3 (which held pretty limited data for operational risk), it is still unlikely to elicit a full blown response from banks in disclosing the type of data sought originally in QIS 2 (2002). This second survey (QIS2, 2002) attempted to get loss data from banks over and above that from QIS1, through a loss data collection exercise. The type of data requested was the collection of granular (eventby-event) operational risk loss data to help the Committee determine the appropriate form and structure of the Advanced Measurement Approach (AMA).. To facilitate the collection of comparable loss data at both the granular and aggregate levels across banks, the Committee again used its detailed framework for classifying losses. In the framework, losses were classified in terms of a matrix comprising eight standard business lines and seven loss event categories. These seven event categories were then further divided into 20 sub-categories, so that the Basel Committee could then attempt to retrieve from banks data on individual loss events classified at this second level of detail. In QIS 2 the Committee also sought information on six "exposure indicators" such as number of employees or total assets. The exposure indicator data served two purposes. First, they were critical to the Committee's effort to aggregate loss data across banking institutions to arrive at an industry loss distribution. Second, the exposure indicators were necessary for banks and supervisors to relate historical loss experience to the current level of business activity. This information also enables banks and supervisors to determine separate frequency and severity distributions for the operational risk loss experience. Although indicators other than gross income were included in this survey, the Committee did not at that stage envision revisiting the use of gross income as the base for the Basic 3

5 Indicator and Standardised Approaches. However as will be seen in the next sections, the Committee changed its mind. QIS2, although a repeat of QIS1, included a number of additional items but also simplified data requests. Specifically banks were no longer asked to provide operational risk loss data by `effect types', nor to provide quarterl y aggregated loss data, nor to provide data on the value of transactions/deals/trades, or the number of transactions/deals/trades. They were however asked to provide data on expected as well as received recoveries, to indicate the internal threshold used for collecting loss data, and to identify those losses arising from a `corporate centre' business. Unfortunately the attempts in QIS2 and QIS3 to gather data on operational risk as an aid to policy formulation, proved difficult, according to several industry sources 3, who said that banks were reluctant to give proprietary data to the regulator about some of the lawsuits they are currently involved with. Consequently, they declined to participate rather than give the regulator "edited" data. In this paper we examine the fourth Quantitative Impact Study Survey (QIS-4) being circulated to participating US based institutions, so that the U.S. federal bank regulatory agencies, Federal Reserve Board, Office of the Comptroller of the Currency, Federal Deposit Insurance Corporation, Office of Thrift Supervision, (Agencies) may gain a better understanding of how the implementation of a more risk-sensitive approach for regulatory capital standards might affect minimum required capital at the industry, institution, and portfolio level. The objective of this paper is to determine how well the survey addresses the main problem areas identified by commentators, practitioners and academics and if not, to evolve one that does, which banks are more likely to answer. The reason for focussing on the US survey is 3 Editor s comment, News and analysis on Basel II and banking supervision, BaselAlert.com, 28 th February,

6 that it is not only one of the first of such studies issued, but it also asks some very specific questions about the measurement and management of operational risk. The results of the US survey will be used ultimately to produce a final revised risk-based capital rule for US qualifying institutions and is expected to be issued in The results are also expected to be used to ensure that minimum capital requirements are appropriately calibrated for both U.S. and international financial institutions. To a large extent, the information and capital treatments requested in the QIS4 survey reflect provisions of the international capital framework proposed in June 2004 (the June framework) by the Basel Committee on Banking Supervision. It also reflects certain adjustments and clarifications needed to tailor the survey for U.S. implementation and to elicit specific policy information considered helpful for the U.S. rulemaking process. The US regulators point out that the capital treatments set forth in QIS-4 are for the informational and analytical needs of the Federal Reserve Agencies only, and should not be construed to represent final decisions regarding implementation of new capital standards or reporting requirements. For example, this survey requests information for a banking organization on a consolidated basis, while future reporting requirements will include information on material subsidiaries and all insured entities using the new Framework. Table 1 summarises the questions specifically aimed at operational risk. The rest of this paper is structured as follows. The next section discusses the requirements for operational risk that exist as at the date of the fourth Quantitative Impact Study (QIS4). Section 3 discusses problems with OR specifications, while Section 4 explores the potential effects on efficiency and stability. Section 5 concludes by listing measurement and management difficulties, and putting forward an alternative to QIS4 s questions on OR, which are more substantive than a state of the art review, which banks are unlikely to answer. 5

7 Table 1: Questions on Operational Risk from QIS What analytical framework was used to quantify operational risk exposure? 2. What was the unit of measurement in the assessment of operational risk exposures (e.g., major business lines, second level business lines, across all loss types, etc.)? 3. Describe how the following elements were individually incorporated into this framework: a. Internal data. How were internal data incorporated into the model? Are there components of the model that rely solely on internal data? If so, how did you assess data sufficiency? b. External data. Were external data a direct input to your model? If so, describe the process for determining when external data were included. If external data were not used as a direct data input, how were they used (e.g. scenario analysis, fit severity distributions, and/or understanding industry experience, etc.)? c. Scenario analysis. Describe how scenario analysis was used in the analytical framework. Were scenarios a direct input into your model? If so, describe the process used to determine when scenarios were included. d. Business environment and internal control factor assessments (and any other qualitative adjustment factors). Were business environment and internal control factor assessments included in your model? What parameters did you incorporate into your model to adjust the operational risk exposure number to reflect these qualitative assessments? 4. What weighting scheme or methodology was used to incorporate each of the four components listed above? Did the weighting vary by business line and/or event type, or for different units of measurement? 5. What specific statistical distributions (e.g., frequency and severity) were used to fit loss data? Did these vary by data type (i.e. internal, external, scenario), business line, or event type? If so, how? 6. Were adjustments made to internal or external data to account for changes in the scale or scope of the business, or factors such as inflation? 7. Describe any correlation and diversification benefit assumptions used as part of the operational risk exposure calculation. Specifically, what model parameters were used as they relate to these assumptions (e.g., an x% correlation in operational losses across different business units)? Describe how you arrived at these assumptions. If there is a diversification benefit, is that amount held at the consolidated entity level or allocated back to the business line? If so, how? 8. Does the operational risk exposure number, reflected in cell G104 represent the sum of expected losses (EL) plus unexpected losses (UL), or UL only? 9. If the operational risk exposure number represents UL only, provide the following information: a. Provide the EL amounts, and describe how EL is derived (e.g. statistically measured, subjective estimation, etc.). b. Describe how EL is accounted for. In particular, describe if operational risk EL is addressed through GAAP-compliant reserves/provisions, pricing or other internal business practices. c. Cells G114 and G115 seek specific information on fraud-related losses. Describe the methodology used to categorize these losses as UL or EL? 10. What loss data thresholds were used to collect the internal data underlying the calculations reported? Please be as specific as possible. If different thresholds were used for different business lines and/or event types, then each threshold should be listed together with a brief rationale for why that threshold value was chosen. Was there a mechanism through which losses under the threshold were reflected in either EL or in the estimate of the operational risk exposure (EL+UL)? 11. Describe the methodology used to take account of the effects of insurance. 4 Extract on Operational Risk from Fourth Quantitative Impact Study Survey (QIS-4) conducted under auspices of the Bank for International Settlements (BIS) - see the announcement National impact studies and field tests in 2004 or 2005, (Basel Committee on Banking Supervision - 6

8 2 The Final Basel II requirements for Operational Risk The best source for current Basel II requirements is the document issued by the Basel Committee of Prudential Supervision in June 2004 entitled A Revised Framework. This document was issued after a long period of consultation starting with the announcement to revise the 1988 Accord on June 2, Since 1999 a number of discussion papers and consultations have recorded various problems with proposed changes. 5 Basel II will for the first time require financial institutions to incorporate an explicit measure of operational risk into their regulatory capital requirements. The requirement applies to Bank Financial Institutions (BFIs) starting in 2004 with Basel I and II systems running parallel until 2006 when Basel I will be phased out. The requirements will be refined to apply to other types of financial institutions such as insurance companies after discussion, with a planned target of application in 2006/7. It is quite clear in all the Basel Committee statements, that in calculating the amount of capital that should be provided for operational risk, that this requirement is to be added to that provided for credit and market risk. The minimum capital requirements are composed of three fundamental elements: a definition of regulatory capital, risk weighted assets and the minimum ratio of capital to risk weighted assets. In calculating the capital ratio, the denominator or total risk weighted assets will be determined by multiplying the capital requirements for market risk and operational risk by 12.5 (i.e. the reciprocal of the minimum 5 See the Third Consultative Document, CP3, The new Basel Capital Accord, (Basel Committee on Banking Supervision, April, 2003). However the most important and informative of the evolution of OR requirements for financial institutions are Sound Practices for the Management and Supervision of Operational Risk, Basel Committee on Banking Supervision (Bank for International Settlements, July 2002); Risk Management Group, The 2002 Loss Data Collection Exercise for Operational Risk: Summary of the Data Collected, Basel Committee on Banking Supervision, March Reviewing the systems needed is the Consultation Paper No. 142, Operational risk systems and controls, Financial Service Authority, July An additional important paper on implementation difficulties is ORIAG, Implementation of the Capital Accord for Operational Risk, (Working Paper, Financial Service Authority, UK, 12 February,

9 capital ratio of 8%) and adding the resulting figures to the sum of risk-weighted assets compiled for credit risk. The ratio will be calculated in relation to the denominator, using regulatory capital as the numerator. The ratio must be no lower than 8% for total capital. Tier 2 capital will continue to be limited to 100% of Tier 1 capital. Minimum floors will be in place for BFIs using advanced models to determine risk levels to ensure that they do not underprovide for capital. BFIs can choose from three main approaches- the basic indicator approach (BIA) where the capital requirement is to be based on a fixed percentage (alpha) currently 15% of gross income; the Standardised Approach (TSA) where the capital charge is still based on gross income but the firm s activities are divided along business lines, each with their own percentage (beta) charge and the Advanced Measurement Approach (AMA), which allows firms to determine their operational risk capital requirement according to an internal model, providing it meets certain requirements. Bank Financial Institutions (BFIs) using the Basic Indicator Approach must hold capital for operational risk equal to the average over the previous three years of a fixed percentage (denoted alpha) of positive annual gross income. Figures for any year in which annual gross income is negative or zero should be excluded from both the numerator and denominator when calculating the average. The charge may be expressed as follows: KBIA = [Ó(GI1 n x á)]/n where, KBIA = the capital charge under the Basic Indicator Approach; GI = annual gross income, where positive, over the previous three years; n = number of the previous three years for which gross income i s positive; á = 15%, which is set by the Committee, relating the industry wide level of required capital to the industry wide level of the indicator. Gross income is defined as net interest income plus net non-interest income. It is intended that this measure should be gross of any provisions (e.g. for unpaid interest);) be gross of 8

10 operating expenses, including fees paid to outsourcing service providers; exclude realised profits/losses from the sale of securities in the banking book; and exclude extraordinary or irregular items as well as income derived from insurance. Simply, if gross income of a BFI is US$1billion, US$150 million will have to be provided over and above the minimum level of capital for other specified risks. Banks are also encouraged to comply with the Basel Committee s guidelines as to Sound Practices for the Management and Supervision of Operational Risk, February In the Standardised Approach, banks activities are divided into eight business lines. The business lines are strictly defined by the Basel Committee. Within each business line, gross income is a broad indicator that serves as a proxy for the scale of business operations and thus the likely scale of operational risk exposure within each of these business lines. The capital charge for each business line is calculated by multiplying gross income by a factor (denoted beta) assigned to that business line. Beta serves as a proxy for the industry-wide relationship between the operational risk loss experience for a given business line and the aggregate level of gross income for that business line. More detail of the defined business lines are given in Table 2 below. It should be noted that in the Standardised Approach gross income is measured for each business line, not the whole institution, i.e. in corporate finance, the indicator is the gross income generated in the corporate finance business line. The total capital charge is calculated as the three-year average of the simple summation of the regulatory capital charges across each of the business lines in each year. In any given year, negative capital charges (resulting from negative gross income) in any business line may offset positive capital charges in other business lines without limit. However, where the aggregate capital charge across all business lines within a given year is negative, then the input to the numerator for that year will be zero. The total capital charge 9

11 may be expressed as: KTSA={Óyears 1-3 max[ó(gi1-8 x â1-8),0]}/3 Where KTSA = the capital charge under the Standardised Approach GI1-8 = annual gross income in a given year, as defined above in the Basic Indicator Appr oach, for each of the ei ght business lines; â1-8 = a fixed percentage, set by the Committee, relating the level of required capital to the level of the gross income for each of the eight business lines. Table 2: Example Mapping of Business Lines 6 Business Unit INVESTMENT BANKING BANKING OTHERS Business Lines Level 1 Level 2 Corporate Municipal/Governme Finance nt Finance Advisory Services Merchant Banking Trading Sales and Sales Market Making Proprietary Positions Treasury Retail Banking Retail Banking Commercial Banking Payment and Settlement Agency Services Asset Management Private Banking Card Services Commercial Banking External Clients Custody Corporate agency Corporate Trust Discretionary Fund Management Activity Groups Mergers and Acquisitions, Underwriting, Privatisations, Securitisation, Research, Debt (government, high yield Equity, Syndications, IPO, Secondary Private Placements Fixed Income, equity, foreign exchanges, commodities, credit, funding, own position securities, lending and repos, brokerage, debt, prime brokerage Retail lending and deposits, banking services, trust and estates Private lending and deposits, banking services, trust and estates, investment advice Merchant/Commercial/Corporate Cards, private labels and retail Project finance, real estate, export finance, trade finance, factoring, leasing, lends, guarantees, bills of exchange Payments and collections, funds transfer, clearing and settlement Escrow, Depository Receipts, Securities lending (Customers) Corporate actions Issuer and paying agents Pooled, segregated, retail, institutional, closed, open, private equity Retail Brokerage Non-Discretionary Fund Management Retail Brokerage Pooled, segregated, retail, institutional, closed, open Execution and full service 6 Basel Committee on Banking Supervision, (2002), Operational Risk Data Collection Exercise 2002, Bank for International Settlements, 4 th June. 10

12 The values of the betas assigned to each business line are detailed in Table 3 below. The effect on the structure of BFIs with divisions that dominate the bank which also have higher assigned betas may lead to unintended effects on dynamic and allocative efficiency. These are discussed later in the paper. Table 3 Business Lines Beta Factors Corporate finance (â1) 18% T rading and sal es (â2) 18% Retail banking (â3) 12% Commercial banking (â4) 15% Payment and settl ement (â5) 18% Agency services (â6) 15% Asset management (â7) 12% Retail brokerage (â8) 12% Under the Advanced Measurement Approach, the regulatory capital requirement will equal the risk measure generated by the bank s internal operational risk measurement system using the quantitative and qualitative criteria for the AMA discussed below. That is if a bank with US$1billion in revenue determines only US$50 million is the value at risk, then capital of only 5% of GI has to be provided. However use of the AMA is subject to supervisory approval. BFIs can use methods partially but if adopting an Advanced Measurement Approach (AMA) they must move a significant portion of business over. Due to major concerns expressed by a number of organisations about practical impediments to the cross-border implementation of an Advanced Measurement Approach (AMA) for operational risk, the Basel Committee issued in January 2004 a further policy statement..7 The policy document suggested a hybrid approach for AMA banks under which a banking group would be permitted, subject to supervisory approval, to use a combination of stand-alone AMA calculations for significantly active banking subsidiaries, and an allocation portion of the group-wide AMA capital requirement for other internationally active banking subsidiaries. Basel II requirements for Operational Risk can be described as a trade-off between efficiency and complexity. For the Advanced Measurement Approach, the internal 7 Basel Committee on Banking Supervision, (2004), Principles for the home-host recognition of AMA operational risk capital, (Bank for International Settlements, January 2004). 11

13 measurement system must estimate unexpected losses based on a combination of internal and external data, scenario analysis, and bank-specific environment and internal controls. The internal measurement system must be capable of supporting allocation of economic capital to business units in a fashion that creates incentive for them to improve their operational risk management. The implications for advanced approaches for operational assessment are that it requires a comprehensive enterprise-wide framework; combines the use of quantitative and qualitative analysis; and tailored solutions are necessary if activities and capabilities across business units are varied. Also implementation plans must be put in place across Groups so that a significant level of effort is required to comply with Basel II operational risk requirements. Overriding this there must be an Operational Risk Policy Framework with procedures covering risk assessment and approval, business risk management, third party risk, business continuity management, fraud risk management, operational loss reporting, non-lending loss ownership and model risk. The above description appears simple. However, there are some obstacles that are perceived as insurmountable by many analysts. These are described in the ensuing sections. 3 Problems with Operational Risk Specifications Operational Risk has been defined by Basel II as the risk of loss resulting from inadequate or failed internal processes, people and systems or from external events, with the overriding requirement that, internationally active banks and banks with significant operational risk exposures are expected to use an approach appropriate for the risk profile and sophistication of the institution. Sources of operational risk are at times hard to segmentalise. Table 4 attempts this below, categorizing Operational Risk into eight main risk categories (Level 1), which can have 21 12

14 types of consequences (Level 2) and require specific controls in order to reduce the inherent probability of loss, and hence produce a lower estimation of value at.risk. Level 3 details some activities that are the result of bad or non existent OR controls. Table 4: Loss Event Type Classification 8 Event-Type Category (Level 1) INTERNAL FRAUD Definition Losses due to acts of a type intended to defraud, misappropriate property or circumvent regulations, the law or company policy, excluding diversity/ discrimination events, which involves at least one internal party. Categories (Level 2) Unauthorised Activity Theft Fraud and Activity Examples (Level 3) Transactions not reported (intentional) Unauthorised transactions (w/monetary loss) Mismarking of position (intentional) Fraud / credit fraud / worthless deposits; Theft / extortion / embezzlement / robbery; Misappropriation of assets; Malicious destruction of assets; Forgery; Check kiting; Smuggling Account take-over / impersonation / etc. Tax non-compliance / evasion (wilful). Bribes / kickbacks Insider trading (not on firm s account) EXTERNAL FRAUD Losses due to acts of a type intended to defraud, misappropriate property or circumvent the law, by a third party Theft Fraud and Theft/Robbery, Forgery, Check kiting Systems Security Hacking damage Theft of information (w/monetary loss) EMPLOYMENT PRACTICES AND WORKPLACE SAFETY Losses arising from acts inconsistent with employment, health or safety laws or agreements, from payment of personal injury claims, or from diversity / discrimination events Employee Relations Compensation, benefit, termination issues Organised labour activity Safe Environment General liability (slip and fall, etc.) Employee health & safety rules events 8 Basel Committee on Banking Supervision, (2002) Operational Risk Data Collection Exercise 2002, Bank for International Settlements, 4 th June. 13

15 Workers compensation CLIENTS PRODUCTS AND BUSINESS PRACTICES Losses arising from an unintentional or negligent failure to meet a professional obligation to specific clients (including fiduciary and suitability requirements), or from the nature or design of a product. Diversity & Discrimination Suitability, Disclosure & Fiduciary All discrimination types Fiduciary breaches / guideline violations Suitability / disclosure issues (KYC, etc.) Retail consumer disclosure violations Breach of privacy Aggressive sales Account churning Misuse of confidential information Lender Liability Improper Business Market Practices or Antitrust Improper trade / market practices Market manipulation Insider trading (on firm s account) Unlicensed activity Money laundering Product Flaws Product defects (unauthorised, etc.) Model errors Selection, Sponsorship & Exposure Failure to investigate client per guidelines Exceeding client exposure limits DAMAGE PHYSICAL ASSETS TO Losses arising from loss or damage to physical assets from natural disaster or other events Advisory Activities Disasters and other events Disputes over performance of advisory activities Natural disaster losses, Human losses from external sources (terrorism, vandalism) BUSINESS DISRUPTION AND SYSTEM FAILURES Losses arising from disruption of business or system failures Systems Hardware, Software, Telecommunications Utility outage / disruptions EXECUTION, DELIVERY & PROCESS MANAGEMENT Losses from failed transaction processing or process management, from relations with trade counterparties and vendors Transaction Capture, Execution & Maintenance Miscommunication Data entry, maintenance or loading error Missed deadline or responsibility Model / system misoperation Accounting error / entity attribution error Other task misperformance Delivery failure Collateral management failure Reference Data Maintenance Monitoring and Reporting Failed mandatory reporting obligation Inaccurate external report (loss incurred) 14

16 Customer Intake and Documentation Customer / Client Account Management Trade Counterparties Client permissions / disclaimers missing Legal documents missing / incomplete Unapproved access given to accounts Incorrect client records (loss incurred) Negligent loss or damage of client assets Non-client counterparty misperformance Misc. non-client counterparty disputes Vendors Suppliers & Outsourcing, Vendor disputes At this point, it is helpful to consider the original management literature that first analysed operational risk in a manufacturing context, which suggested various measurement techniques. 9 This literature was based on refuting two assumptions - that factors which cannot be measured cannot be controlled and that quality cannot be measured so it cannot be controlled. The second statement was soundly refuted by the total quality management movement that started in Japan in the middle of the twentieth century and then spread to the US manufacturing sector starting in the late 1970s. The problem is that there is no single measure of quality. Rather, it is reflected in consistent performance on a variety of eclectic measures, which were developed in a body of knowledge known as Statistical Process Control (SPC). Unfortunately the SPC literature ignores that operational risk in banks is an amalgamation of many disparate risks. 10 While there have been many attempts to define it positively, its primary definition remains a negative one losses that are not related to either credit or 9 This is best exemplified by statistical process control (SPC) as pioneered by Walter Stewart and described in his 1931 book, entitled Economic Control of Quality of Manufactured Product. 10 Holmes, M., (2003) Measuring operational risk: a reality check, Risk, September 2003 Vol 16 / No 9. 15

17 market events. Such events include fraud, settlement errors, accounting, and modelling mistakes, lawsuits, natural disasters, IT breakdowns, and many other types of loss. The heterogeneous nature of operational risk is a key difficulty underlying many of the issues we describe further in this article. In credit and market risk, there is some commonality among the risks in question they form a natural grouping. For example, credit risk is typically extended via a consistent process; the issues of default likelihood, exposure measurement, and loss-given default are similar; and the resulting exposures are subject to common risks, such as the risk of an economic downturn. Likewise, market risks deriving from price fluctuations of financial assets have common properties so that they can normally be managed in a consistent way, and modelled with a common process. Operational risk appears to be different. It is useful to categorise operational risk into two groups - low-frequency large-loss events ( major ), for example, rogue trading, major lawsuits and natural disasters and high-frequency small-loss events ( minor ), for example, settlement errors and credit card fraud. The primary challenge for a capital model is addressing the major events. These events can threaten the capital or even the solvency of the firm, as was seen in the Barings case. Minor events are a secondary challenge. Reducing these events may create efficiency savings but is unlikely to affect the risk of the bank materially. The causes of major events can be complex. They often include human failure, organisational failure, and adverse external environmental factors, all acting in combination. It is easy to see that a modeller who tries to capture the risk from major events has a very difficult, even questionable task. Mathematical models are used in market and credit risk management for decision-making purposes because they provide the user with information on the potential losses that can be 16

18 incurred for a given portfolio of positions. There is a clear link between the generators of risk interest rate, equity price sensitivities and money lent and the potential financial impact on the firm. The links can subsequently be tested and proved to work. The model should capture the essential features of the situation in a plausible manner; have predictive qualities that can be used for decision making; which can be validated. At a minimum, a good risk model should enable an observor to judge whether bank A is riskier than bank B, and whether bank A s risk is increasing or decreasing over time. Market and credit risk models generally satisfy these requirements, even though there remains lively debate about the best approaches, implementation specifics and other features. Operational risk models currently proposed do not appear to satisfy these requirements at present. Current models are typically descriptive and backward looking, with limited intuition about how key features could create a risk event. Holmes (2003) claims there is no model that has a convincing capability to rank interbank risk or bank risk over time, nor, most critically, is there any model that has been validated for the major events that are crucial for risk capital. Typical operational risk models start with either a self-assessment scorecard approach or a loss-data approach. The scorecard approach is inherently qualitative. It raises the question of whether scorecards are really models, or whether they are simply a formalisation of the discussions that already exist in banks about risk prioritisation. Holmes (2003) is sceptical that this approach would give reliable information about bank risk over time or rank the relative risk of two banks. There appears to be no conclusive evidence that these models work in practice and have predictive properties. The loss-data approach (LDA) appears to be a more serious attempt at modelling this type of risk, and has many scientific elements. These models typically collect losses down to a low dollar threshold then apply an off-the-shelf distribution to fit the loss data. Patterns in 17

19 the low-loss frequent observation area are by virtue of the distribution believed to affect the likelihood of a high-impact event. In effect, the data and the distribution are the model. The model develops simply because of the addition of new loss events or a revision to the supposed distribution. There is no attempt to determine whether the risk or size of the portfolio has changed. This is analogous to trying to model credit risk using only past default losses, with no account taken of the size and riskiness of the current credit portfolio. Fundamental challenges in measuring operational risk follow from flawed definitions. Many groups in industry, academia and the regulatory community are trying to produce OR models for the finance industry, approaching operational risk measurement in a similar way to market risk and credit risk, using loss-data style models as their primary tool. The success of this approach will rest on whether operational risk has similar properties to market and credit risk. One characteristic of operational risk that illustrates the weakness of the analogy is that while market and credit risk are independent of the bank taking the risk, operational risk is inherent in and an attribute of the bank itself. For example, consider two banks with identical trading positions and loan portfolios with exactly the same customers. Their market and credit risk will be the same but their operational risks could be significantly different. This poses deep issues for the use of industry-pooled data. Both credit and market risk exposures are typically explicit, and normally accepted because of a discrete trading decision. Indeed, often the risk-taking decision depends on the ability to measure the risk of a transaction relative to its expected profitability. Market and credit exposures are also subject to well-understood concepts of quantifiable size. Credit risk exposures can be measured as money lent, mark-to-market exposure, or potential exposure on a derivative. The risk of the positions can be estimated using credit ratings, market-based 18

20 models and other tools. Market risk positions can be treated as principal amounts or decomposed into risk sensitivities and exposures. The risk of these positions can be quantified with scenarios, value-at-risk models, and so on. In both market and credit risk there is a direct link to the driver of risk, the size of the position and the level of risk exposure. These risk models allow the user to predict the potential impact on the firm for different risk positions in various market environments. In contrast, operational risk is normally an implicit event. It is accepted as part of being in business, rather than as part of any particular transaction. There is also no inherent operational risk size in any transaction, system, or process that is easy to measure. A related issue is the issue of completeness of the portfolio of operational risk exposures. For both market risk and credit risk, modelling starts with a known portfolio of risks. Indeed, it is a fundamental test of a bank s risk management systems and processes to ensure that there is complete risk capture. However, in operational risk modelling, the portfolio of risks is not available with any reasonable degree of certainty by any direct means. Even if a bank knows its processes and could ascertain the size of the risk in those processes, it is difficult to identify unknown risks or non-process type risks (for example, fraud risk or a new type of IT breakdown). As mentioned above, many major events are of this type they are simply outside the bank s normal set of unde rstood risks (for example, the September 11 impact on trade processing capability in New York City). The issue of completeness explains the weakness in proposed approaches to measuring operational risk that rely mainly on operational risk loss experience to infer a loss distribution. In essence, these quantification approaches effectively try to imply the portfolio of possible operational risk loss events from historic loss events. Imagine taking this approach to credit risk modelling, that is, deducing the loan portfolio from historic defaults (experienced both at the bank in question and in the rest of the industry) instead of obtaining it from the firm s 19

21 books and records this would certainly not be regarded as an acceptable modelling approach for effective risk management. It is important to realise that this lack of knowledge about the portfolio of possible operational risk loss events is not a technical modelling challenge; rather, it is an inherent characteristic of operational risk. The third important issue that affects the ability to effectively measure operational risk is context dependency. This describes whether the size or likelihood of an incident varies in different situations. It is important in modelling because it determines how relevant your data is to the current problem. For example, an analysis of transportation accidents over the past century would clearly contain data that had lost relevance due to different modes of transport, changing infrastructure, better communications, etc. For example, consider the following questions: are your businesses, people or processing systems similar to 10 years ago (for example, many banks have merged and/or materially changed their systems and processes); are the threats to those systems similar to 10 years ago (for example, did firms worry about internet virus attacks in 1993)? The chances are that you answered no to both questions, illustrating the high context dependency of operational risk. Context dependency is driven by how quickly the underlying system or process changes. Many market risks appear to have a moderate level of context dependency, as stock market prices tend to exhibit statistical properties that appear to be somewhat stable across time (for example, New York Stock Exchange behaviour in 1925 would be recognisable to a modern trader). Likewise, credit ratings and loss statistics have been measured for many decades and show some reliable properties. The level of context dependency has a fundamental impact on the ability to model and validate a system; in general, the higher the context dependency, the less the past will be a good predictor for the future. 20

22 For those risk types that exhibit low context dependency and have high data frequency, it is usually possible to identify risk patterns and test whether these properties hold true over time. That is, it is possible to use statistical methods to quantify the risk and to predict future outcomes. Conversely, for risk types that show high context dependency and low data frequency, it is inherently difficult to make predictions of their future size. Sufficient frequency of relevant data is critical for all risk modelling. To summarise, operational risk has been divided into major and minor type events. It is arguable that adequate data exists to generate a distribution for minor events so that they can be treated with statistical methods, but these events are less important for risk. The primary challenge is addressing the major events that can adversely affect the capital of the firm, severely harm its reputation, or in extreme situations put it out of business. In this case, the high level of context dependency and the low level of relevant outcome data suggest that attempting to effectively quantify operational risk based on loss experience will be difficult because of the lack of data around major events. Validation of operational risk models remains a major challenge. The causes of major events are often complex and due largely to human factors. The ability to predict future major events based on previous major events is difficult and questionable. The ability to validate a model used to measure a given type of risk is also related to the frequency of outcome data from that risk. For market risk, model validation is relatively easy, by comparing daily VAR versus observed profit and loss (back testing). For credit risk, validation is possible but a longer time horizon a number of years is required, though other tools can also help close the gap. In contrast, information about major operational risk loss data is infrequent compared with market and credit risks. A fundamental challenge for any operational risk model is that the system changes in character (context dependency) before adequate data is accumulated to validate the model. 21

23 Application to financial services SPC has been shaped largely in the context of product manufacturing. As such, its practices need to be adapted to the somewhat different circumstances of the financial services industry. In some ways, however, its application may well be easier in finance. For example, the daily number of failed trades or unmatched confirms is already a sample of a significant number of individual transactions. As such, these are likely to be normally distributed. Some experts in the field of SPC advise financial executives should look to their peers in manufacturing for important lessons in the analysis and control of operational risk 11. However, there are unique problems in the application of SPC to finance, which will be discussed in Section 5. Before turning to the finer problems is it worth considering the relationship between operational risk minimisation and the regulatory goals that have been defined as the optimum for any government, central banker, or prudential supervisor. These goals are maintaining and improving systemic efficiency, stability, safety and confidence The Strategic Effects of Basel II OR Requirements on Banks and the Financial System. If the requirement to provide for operational risk significantly affects the cost of funds to a financial institution, banks may raise pricing levels which could result in a restriction of credit. This would affect the operational efficiency of the system. Alternatively certain products and business units may be perceived to carry more operational risk, requiring more capital and this could constrain and or distort allocative and dynamic efficiency, leading to a 11 Refer to related articles on - Breaking down the model; Asset manager technology hinders op risk management; Geithner to replace McDonough at New York Fed ; Algo to release flagship Basel II-compliant system in January; 'A good deal for regulators and banks' ; Black Thursday; China's regulator publishes new draft derivatives guidelines; Weasel parade; Geopolitical futures: The politics of betting ; FSA warns of treasury management flaws 12 Sinkey Jr, J.F., Commercial Bank Financial Management. Maxwell MacMillan 22

24 restriction in credit and the reduced provision of products and services perceived to have high operational risk levels. The strategic importance of this possible chain of events is best illustrative by considering the effect on loan pricing, as pricing decisions directly impact on lenders revenue and hence the future accumulation of capital. On average in a bank financial institution (BFI) loans represent approximately 70% of earning risk assets of which between 40-50% are commercial loans. The profitability of loan portfolios is affected by a variety of interacting factors: volatility, globalization, competition, customer sophistication, macroeconomic indicators. However regulation of the markets is probably the most dominant component. With each customer type, BFIs use some form of customer profitability analysis (CPA) as a guideline to loan pricing. CPA is designed to evaluate all relevant expenses and revenues associated with a customers total banking relationship to the banks target rate of return to shareholders. CPA avoids the cross subsidisation and subjectivity most frequently seen in the less sophisticated systems and becomes of greater importance as customers have multibank relationships. CPA can be viewed as defensive for existing business or aggressive pricing in an attempt to acquire new business. The major cost input into this is cost of funds of which capital is the most expensive source. Hence changes in capital adequacy resulting from including operational risk in the regulatory requirements will affect not only pricing but may also reduce the RAROC of customers and products/services so that banks restrict their supply. This brief and simplistic overview of pricing principles above illustrates the potential effect of changes in capital adequacy requirements on the cost to the end user, and hence the efficiency of the banking system, and on a macro level the productivity frontier for the entire economy. Also to be considered is whether operational risk is a major cause of bank crises. Causes can range from lack of investor and depositor confidence precipitated by perception 23