Measuring operational risk in financial institutions

Transcription

1 Measuring operationa risk in financia institutions Operationa risk is now seen as a major risk for financia institutions. This paper considers the various methods avaiabe to measure operationa risk, and identifies a new framework for anaysis. Amandha Ganegoda is a Master of Phiosophy student at the University of New South Waes. Emai: amandha_ ganegoda@yahoo.com John Evans F Fin is Associate Professor in Actuaria Studies at the University of New South Waes. Emai: john.evans@ unsw.edu.au A framework to measure operationa risk The framework we propose to measure Operationa Risk (OpRisk) is inspired by a statement given by the United States former Secretary of Defense, Donad Rumsfed. Thus we have caed the method the Rumsfed Approach. We broady categorise a OpRisk into three categories: Known-Known risks that we know exist and know how to mode; Known-Unknown risks that we know exist but do not know how to mode (or are difficut to mode, e.g. ega risk); and Unknown-Unknown risks that we are unaware of. The motivation behind the Rumsfed approach is to provide a consistent framework that woud take into account a three categories of OpRisk to determine the risk capita for a financia institution. The subsequent sections discuss the suitabe methods avaiabe to measure the OpRisk for each category. Modeing Known-Known OpRisk OpRisks that we know exist and can be modeed are categorised as Known-Known OpRisk. There are many approaches that have been used by the financia industry as we as other industries to mode such risk. Most of these approaches can be broady subdivided into two categories: top-down approaches which attempt to mode aggregate operationa osses without giving attention to underying operationa process and bottom-up approaches which attempt to mode osses by mapping the operationa process to predetermined risk components. jassa the finsia journa of appied finance issue

2 Scaars Scaars are simpistic top-down methods to measure operationa risk. The methodoogy assumes operationa risk to be a predetermined percentage of a business scaar such as gross income, operationa costs, assets, funds under management etc. (Lawrence 2000). Two we-known exampes of this approach are the basic indicator approach and the standardised approach specified for banks under Base II. The main advantage of using scaars is that it s a ow-cost method of measuring OpRisk. However, the main disadvantage is that the cost of capita is not directy inked to the oss data, operationa process or the contro process. Thus the accuracy of the mode is questionabe. Regression and trend anaysis The modes based on regression and trend anaysis attempt to identify the key risk indicators (KRIs, e.g. audit ratings, empoyee turnover and transaction voume) that drive the operationa risk and then use these KRIs to monitor and measure OpRisk. Since KRIs are directy inked to the operationa process, the mode gives the ine managers a behavioura incentive to keep the KRIs at a desired eve in order to manage the OpRisk. Financia statement modes Financia statement modes assume operationa risk has an infuence on financia resuts such as stock returns, earnings, expenses and profitabiity. The first step in the modeing process is to identify a target variabe which is highy infuenced by the operationa risk. Then the target variabe is modeed using externa risk factors which drive the target variabe. Operationa risk is measured as the variance in the target variabe that is unexpained by the externa risk factors. CAPM-based modes (Hoffman 2002; Ceske et a. 2000) are exampes of this approach. Due to the highy skewed nature of operationa oss data, conventiona frequency and severity modes used in LDA are unabe to provide adequate resuts in describing the oss data; especiay in the extreme percenties. A further shortcoming of the LDA is that it does not take into account risk diversification when cacuating the capita charge. LDA is not suitabe to be used when significant changes have occurred in the business or contro environment. CAPM-based modes assume OpRisk is the differentia between risk measured by CAPM and the risk measured separatey by the credit and market risk modes. The main advantage of financia statement modes is that they are easy to impement and inputs are readiy avaiabe. A shortcoming of this approach is that not a OpRisks are sensitive to externa risk factors and it is possibe that some important OpRisk such as fraud can be overooked in such instances. Expected oss modes Expected oss modes attempt to project future expected operationa osses by using the institution s interna oss data as a key input to measuring operationa risk. Thus, in contrast to scaars, the capita charge is directy inked to the institution s oss distribution. These modes assume unexpected osses (the tai of the oss distribution) can be extrapoated using expected osses (the mean of the oss distribution). A we-known exampe of this approach is the interna measurement approach (IMA), one of the methods prescribed under the Base II advanced measurement approach (AMA) 1. The main advantage of the expected oss mode approach is that it directy inks the firm s oss experience into OpRisk capita cacuations as opposed to the top-down modes discussed earier. However, the main drawback of this method is that it does not attempt to assess the tai of the oss distribution directy. Loss distribution approach The oss distribution approach (LDA) is a technique that has been used to mode insurance osses for many years. Simiar to IMA, LDA categorises the risk events in a matrix by business ine and event type. But, rather than compute the expected osses, LDA estimates two separate distributions for frequency of osses and severity of osses for each risk ce using interna data. In theory, LDA is abe to provide superior resuts to the expected oss modes since it makes fu use of the interna data to directy measure the unexpected osses. However, research incuding Moscadei (2004) and (Evans et a. 2007) has demonstrated that due to the highy skewed nature of operationa oss data, conventiona frequency and severity modes used in LDA are unabe to provide adequate resuts in describing the oss data, especiay in the extreme percenties. A further shortcoming of the LDA is that it does not take into account risk diversification when cacuating the capita charge. LDA is not suitabe to be used when significant changes have occurred in the business or contro environment. Many improvements for the LDA approach have been proposed by various authors to address the issues. One method that has been put forward to mode the tai of the distribution is to use the extreme vaue theory (EVT). Moscadei (2004) used the peak-overthreshod method (POT) to mode the tai of the oss distribution to provide significanty better fit to the operationa oss data in the extreme percenties. Simiar resuts have been reported by Evans et a. (2007). 10 jassa the finsia journa of appied finance issue

3 Risk drivers and contro approach (RDCA) The RDCA approach, which is aso known as scorecards, is one of the aternative methods specified by Base II under the AMA regime (Base 2001). This approach heaviy reies on contro sef-assessment (CSA) techniques to identify the principa drivers and contros of OpRisk. The RDCA approach has the advantage of invoving the ine managers in the modeing process, and the method is more transparent to managers as risk exposures are measured using statistics they are famiiar with. It is possibe to compare the performance of managers by comparing the scores of the business units they are responsibe for. The RDCA approach not ony provides the means to quantify OpRisk but aso to monitor and manage OpRisk. Deta-EVT Deta-EVT is a method deveoped by King (2001) to quantify OpRisk. The method measures operationa risk as the uncertainty of earnings due to two types of operationa osses: first, the high frequency-ow severity osses that can be mapped to causa factors; and second, the ow-frequency high-severity osses that cannot be mapped to causa factors (e.g. contro breakdowns). King (2001) proposes modeing the ow severity osses using the Deta method and extreme osses using EVT. The Deta method attempts to quantify the aggregate uncertainty in profits propagated from uncertainties in each OpRisk factors. For exampe, consider the simpe case where iabiity of a superannuation fund with m investment options cacuated using the formua: The Deta-EVT method has many advantages. The Deta method empoyed to quantify highfrequency events is based on the cassic error propagation aw which is an ISO standard for measuring uncertainty (ISO 1993) and the Deta method is fairy forwardooking as the key inputs are very sensitive to the operationa exposure and the changes in the contro environment. Then assuming unit pricing and accounting methods used to vaue each investment method are simiar, in other words assuming perfect correation among iabiity vauation errors for each investment option, tota error in vauation due to OpRisk is: According to this formua an operationa oss due to mispricing of iabiity can happen due to two reasons: a unit pricing error or an accounting error in the tota number of units. Thus taking unit pricing errors and accounting errors as the risk factors, we can write the formua for voatiity of iabiity due to OpRisk in ith investment using the deta method as foows: The Deta method ony measures the operationa risk that can be reated to causa factors. Most of the catastrophic osses and contro breakdowns are not reated to causa factors. Thus King (2001) suggests the use of EVT to quantify such risk due to rare events. He proposes obtaining the maximum operating oss due to causa factors using the Deta method and setting it as a threshod to fiter the arge osses, and then using EVT to mode the excess osses. The Deta-EVT method has many advantages. The Deta method empoyed to quantify highfrequency events is based on the cassic error propagation aw which is an ISO standard for measuring uncertainty (ISO 1993), and the Deta method is fairy forwardooking as the key inputs are very sensitive to the operationa exposure and the changes in the contro environment. Furthermore, even when historic data is not avaiabe, the Deta method can be empoyed using expert judgment to estimate standard deviations. Couped with EVT to measure high percentie osses, a Deta-EVT method provides an eegant soution to quantify OpRisk. prefix represents the error term. jassa the finsia journa of appied finance issue

4 Causa modes Bayesian beief networks (BBN) Bayesian beief networks are a type of causa mode which empoy Bayesian probabiity theory to mode cause and effect in a system. In contrast to a Deta-EVT method, BBNs aow us to mode events where casua events exist but deterministic reationships cannot be obtained (Mast et a. 1999). Appications of BBNs can be found in the nucear industry (Santoso et a. 1999), medica diagnosis (Nikovski 2000), data mining (Heckerman 1997), inteigent troube shooting systems (Heckerman & Breese 1996), and aviation faiure diagnosis (Mast et a. 1999). A BBN is a set of variabes caed nodes which are connected by arrows (known as directed edges, or arcs) representing the dependencies among the nodes such that there are no directed cyces. Once the BBN has been buit it can be used to measure, monitor and manage OpRisk. An advantage of using a BBN to mode OpRisk is that it aows management to dynamicay observe the changes to the oss distribution with respect to changes in the business and contro environment. Other modes using a process approach There are many other modes based on the principes of process approach. Some of them incude reiabiity modes, connectivity modes, system dynamics, and neura networks. Reiabiity modes have been in use for many years in engineering to measure and mitigate OpRisk in power pants, nucear reactors etc. They mode the time between OpRisk events rather than their frequency (Saunders et a. 2004). Thus these modes may become usefu in measuring particuar operationa risk (e.g. IT faiure) at a business unit eve. A system dynamics approach is another causa mode that has been proposed to measure OpRisk. A discussion contributed by Jerry Miccois and Samir Shah of the use of a system dynamics approach at Tiinghast/Tower Perrin can be found in Hoffman (2002). Another promising method in the deveopment stage is neura networks. Perera (2000) caims that the neura networks deveoped at NASA to anayse reiabiity of micro-eectromechanica systems (MEMS) can be adapted to measure operationa risk for financia institutions. So which method to choose? As discussed above, there are many methods avaiabe to mode Known-Known OpRisk. The method chosen wi depend on factors such as data avaiabiity, time and budget constraints. Top-down methods such as scaars, financia statement modes and trend anaysis are suitabe for sma institutions that do not have the capabiity to carry out sophisticated modeing. On the other hand, arge institutions may find bottom-up methods such as LDA and causa modeing more attractive as they wi aow the firm to gain a better understanding of its OpRisk. Among the bottom-up methods avaiabe, causa modes provide superior resuts in terms of accuracy, forward-ooking capita estimates, behavioura impact and eary warning signas. Use of KRI in the RDCA approach makes RDCA somewhat forward ooking. If propery set up, RDCA can provide vauabe information to OpRisk managers on how to manage risk and even estabish eary warning signas. Causa modes coud be quite usefu to measure particuary high risk OpRisk at business unit eve. For exampe, given that most of the arge-scae osses in recent years, such as Sumitomo, Barings, Aied Irish, Lehman, and IAG, were infrequent and, due to different causes, the time and cost invoved in buiding a causa mode may not be unreasonabe to measure and monitor such OpRisk. Assessing Known-Unknown OpRisk In order to mode risk that we know exists but cannot mode (e.g. ega risk), we propose a methodoogy based on the Sovency II oss-given-defaut approach. Under Sovency II, European insurers need to stress test the survivabiity of the business assuming arge catastrophic industry osses. This method is caed the oss-givendefaut approach. The method simpy ooks at whether an insurer has enough capita to cover their exposures under a given set of catastrophic industry osses (e.g. European windstorm causing a $4 biion industry oss). The method makes no attempt to quantify the frequency of osses. The Back Swan approach outined beow is a sighty modified version of the oss-given-defaut approach such that it woud be suitabe to measure OpRisk. The basic steps of the Back Swan approach are as foows: Identify the types of OpRisk casses that we cannot mode using conventiona modes. Obtain industry oss data for each risk cass and make corrections for inherent bias. Use the adjusted data to find the maximum operationa oss experienced by simiar organisations in the industry under each OpRisk cass. Survivabiity of the business is stress tested against the maximum oss. If the business cannot survive, then the necessary capita or risk management practice is put forward for adoption. Correcting for inherent bias The most important step in the Back Swan approach is to correct for inherent bias in the data since data that has not been corrected for bias can yied perverse capita estimates. According to APRA (2007) there are mainy three types of bias which affect externa data: Reporting bias occurs when a different threshod has been used by institutions to report osses. Contro bias occurs when data is coected from institutions with different contro systems. Scae bias occurs when data is coected from institutions with different sizes. The main advantage of the Back Swan approach is that it is simpe to impement and the inputs are readiy avaiabe. Unike most of the methods discussed earier, the Back Swan approach focuses on risks that pose the greatest threat to the sovency of the business. 12 jassa the finsia journa of appied finance issue

5 Among the bottom-up methods avaiabe, causa modes provide superior resuts in terms of accuracy, forward-ooking capita estimates, behavioura impact and eary warning signas. Risk margin for Unknown-Unknown OpRisk Unknown-Unknown OpRisk is the OpRisk that the firm is exposed to but is unaware of. The ony sensibe way to account for these types of risk is to add a risk margin above the capita charge cacuated under Known-Known and Known-Unknown OpRisk. Benchmarking is one of the easiest ways to determine the appropriate risk margin. This method simpy ooks at the OpRisk capita hed by simiar firms in the industry to manage their unknownunknown risk and benchmark against it. This is a proxy measure rather than a rea quantification, so it is necessary to use expert judgment when benchmarking against each other so that the nature of the firm s business environment and contro process is taken into account when deciding the risk margin. Concusion In this paper we have introduced a consistent framework to measure operationa risk such that a three categories of risk: known-known, known-unknown and unknown-unknown risks are taken into account when determining the capita charge. There are many methods avaiabe to measure known-known risks. Choice of the method wi depend on the factors such as data avaiabiity, and time and budget constraints. Most of the top-down modes avaiabe to measure Known-Known risks are easy to impement, ow cost and suitabe when in need of quick answers. However, the accuracy of these modes is questionabe. On the other hand, bottom-up approaches are abe to provide more accurate capita estimates as they make fu use of the interna oss data and, in some cases, even externa data and expert opinion. Causa modes such as Bayesian networks seem to most ikey provide best resuts in terms of accuracy but, due to the time and costs invoved in setting up a causa mode, most businesses might find these modes ess feasibe to impement. The RDCA method seems to provide a good trade-off between performance and time/cost. If propery set up, an RDCA mode can provide fairy forward-ooking estimates of OpRisk capita charge, behavioura incentives and eary warning signs to managers. In order to assess Known-Unknown OpRisk we propose stress testing using the Back Swan approach, which is a modified version of the oss-given-defaut approach. Unknown-Unknown risk is taken into account by adding a risk margin above the capita estimates obtained under known-known risk and known-unknown risk. Note 1 Methods recommended under Base II advanced measurement approach incude: Interna Measurement Approach (IMA), Scenario Based Advanced Measurement Approach (sbama), Risk Drivers and Contros Approach (RDCA) and Loss Distribution Approach (LDA). References Austraian Prudentia Reguation Authority 2007, A review of correction techniques for inherent biases in externa operationa risk oss data, Working Paper Series. Base Committee on Banking Supervision 2001, Working paper on the reguatory treatment of operationa risk. Ceske, R., Ernández, J. V. H. and Sánchez, L. M. 2000, Quantifying event risk: the next convergence, The Journa of Risk Finance, vo, 1, iss. 3, pp Evans, J., Womersey, R. and Wong, D. 2007, Operationa risks in banks: an anaysis of empirica data from an Austraian bank, in Biennia Convention 23 26, Institute of Actuaries of Austraia, Christchurch, New Zeaand. Heckerman, D. 1997, Bayesian networks for data mining, Data Mining and Knowedge Discovery, vo. 1, no. 1, pp Heckerman, D. and Breese, J. S. 1996, Causa independence for probabiity assessment and inference using Bayesian networks, Systems, Man and Cybernetics, Part A, IEEE Transactions on Knowedge and Data Engineering, vo. 26, no. 6, pp Hoffman, D. 2002, Managing operationa risk: 20 firmwide best practice strategies, Wiey, New York. Internationa Organization for Standardization 1993, Guide to the expression of uncertainty in measurement, Internationa Organization for Standardization, Geneva. King, J. L. 2001, Operationa risk: measurement and modeing, Wiey, New York. Lawrence, M. 2000, Marking the cards at ANZ, artice originay appeared in the November 2000 issue of Risk magazine: Operationa Risk Specia Report. Mast, T. A., Reed, A. T., Yurkovich, S., Ashby, M. and Adibhata, S. 1999, Bayesian beief networks for faut identification in aircraft gas turbine engines, IEEE Internationa Conference on Contro Appication, vo. 1, pp , Kohaa Coast, HI, USA. Moscadei, M. 2004, The modeing of operationa risk: experience with the anaysis of the data coected by the Base Committee, in Temi di discussione (Economic working papers), Bank of Itay, Economic Research Department. Nikovski, D. 2000, Constructing Bayesian networks for medica diagnosis from incompete and partiay correct statistics, IEEE Transactions on Knowedge and Data Engineering, vo. 12, no. 4, pp Perera, J. 2000, Neura networks do a the work, artice originay appeared in the November 2000 issue of Operationa Risk suppement to Risk magazine. Santoso, N. I., Darken, C., Povh, G. and Erdmann, J. 1999, Nucear pant faut diagnosis using probabiistic reasoning, IEEE, vo. 712, no. 2, pp Saunders, A., Boudoukh, J. and Aen, L. 2004, Understanding market, credit, and operationa risk: the vaue at risk approach, Backwe, Maden, Mass. jassa the finsia journa of appied finance issue