Forecastng and Stress Testng Credt Card Default usng Dynamc Models Tony Bellott and Jonathan Crook Credt Research Centre Unversty of Ednburgh Busness School Verson 4.5 Abstract Typcally models of credt card default are bult on statc data, often collected at tme of applcaton. We consder alternatve models that also nclude behavoural data about credt card holders and macroeconomc condtons across the credt card lfetme, usng a dscrete survval analyss framework. We fnd that dynamc models that nclude these behavoural and macroeconomc varables gve statstcally sgnfcant mprovements n model ft whch translates nto better forecasts of default at both account and portfolo level when appled to an out-of-sample data set. Addtonally, by smulatng extreme economc condtons, we show how these models can be used to stress test credt card portfolos. JEL: G21, D14. Keywords: dscrete survval models, stress testng, loss dstrbutons, cholesk decomposton, credt rsk. Address for correspondence: Credt Research Centre, Unversty of Ednburgh Busness School, 29 Buccleuch Place, Ednburgh EH10 4SW, UK. {j.crook,t.bellott}@ed.ac.uk Acknowledgement: Ths work was partally funded by the EPSRC grant EP/D505380/1. Copyrght Unversty of Ednburgh 1 of 36
1. Introducton Applcaton consumer credt scorng models use detals about oblgors or potental customers that are statc. Such models are used to determne whether an applcant should be granted credt based on data collected at tme of applcaton that then reman fxed. Typcally, ths s nformaton taken from a completed applcaton form and a credt score for the ndvdual provded by a credt bureau. But such models are restrctve and models whch answer more specfc questons can be estmated from credt portfolos because the latter provde panel data (Crook and Bellott: 2009) for a sample of oblgor accounts, each wth ts own credt hstory over tme. As such a dynamc model would be more approprate to determne credtworthness wthn a portfolo. In ths way, recent tme-varyng behavoural factors such as credt usage and payments can be ncluded to supplement the basc applcaton data n order to yeld more accurate estmates of credtworthness. Addtonally, a dynamc model can nclude other tme-varyng components. In partcular we may expect common economc rsk factors to affect all oblgors n a portfolo generally n the same way. For example, we would expect that a large ncrease n nterest rates would cause, ceters parbus, a general ncrease n probablty of default (PD). Further, statc models typcally only have value n assessng the rskness of applcants and oblgors. However, f we want a complete pcture we should be lookng at return alongsde rsk and ths requres the use of dynamc rather than statc models ( Thomas et al :2001, Ma et al :2009). In ths paper, we present dynamc models of default whch nclude behavoural varables (BV) and macroeconomc varables (MV) n addton to applcaton varables (AV). 2 of 36
The ncluson of MVs also enables us to perform stress tests snce extreme economc condtons can be smulated and ncluded n the model to generate a measure of stressed loss. Accurate stress tests are becomng ncreasngly mportant n evaluatng the rsk to banks as s evdent by the recent evaluaton of US banks (FRS 2009) and the recognton by the FSA (2008) that stress testng s a key tool n helpng fnancal nsttutons make busness strategy, rsk management and captal plannng decsons. However, very few stress test results for retal loan portfolos have been publshed. Breedan and Ingram (2009) dscuss ssues nvolved n generatng scenaros usng a model where the default rate of a portfolo over tme s explaned n terms of a functon of duraton tme, a functon of calendar tme and a functon of vntage. But they do not present the results of a stress test for a portfolo and t s debatable whether smulatng a parametersed functon of calendar tme s the same as smulatng macroeconomc varables that are related to the probablty of default at account level. Rösch and Scheule (2004) assume a Merton one factor model and estmate loss dstrbutons for credt cards, mortgages and other consumer loans n the US. But they use aggregate default rate data and omt varables specfc to the oblgor. It s also unclear how they preserved the correlaton structure between the MVs n ther model. In ths paper we use Monte Carlo smulaton to generate loss dstrbutons of estmated default rates as the bass of a stress test of our credt card data. Boss (2002) uses a smlar dynamc model structure for smulaton-based stress tests but on corporate loans. Several modellng technques have been proposed to develop a dynamc model of credt (see Crook and Bellott: 2009 for a revew). Thomas et al (2001) descrbe how to use a Markov chan stochastc process as a dynamc model of delnquency. 3 of 36
However, the approach they descrbe does not allow for model covarates, although models can be bult on separate segments allowng modellng of dfferent rsk groups. They also descrbe survval analyss as a means to buld dynamc models snce ths readly allows the ncluson of BVs and MVs as tme-varyng covarates (TVCs). (2008) follow ths path usng the Cox proportonal hazard survval model to model tme to default for a large database of credt cards. They nclude MVs, but not BVs, as TVCs and fnd a modest mprovement n predctve performance n comparson to a statc logstc regresson. Here, we take a smlar approach usng a survval model. However, n contrast wth these papers whch use contnuous tme models, we use dscrete survval analyss. Dscrete survval analyss can also be understood as a logstc regresson on a panel data set wth the data arranged so that default s condtonal on no pror default havng already occurred on that account. Snce credt data s usually n the form of panel data, and n partcular account records are dscrete (eg monthly records), ths s a more natural choce than contnuous tme survval analyss. It also has the advantage of beng more computatonally effcent snce probablty forecasts nvolve smple summatons over tme perods, rather than an ntegraton whch may be complex when TVCs are ncluded n the model. Dscrete survval models have been appled successfully n the analyss of personal bankruptcy and delnquency n the USA (Gross and Souleles 2002), mortgage termnatons (Calhoun and Deng 2002) and competng rsks of foreclosure and sales n the US subprme market (Gerard et al 2008). Gross and Souleles (2002) use several BVs and MVs. In partcular they ncluded outstandng account balance and repayments and found the former had a postve affect on bankruptces and the latter 4 of 36
had a negatve effect. They also found that local unemployment rate had a statstcally sgnfcant postve effect on bankruptcy, whch s what we would expect snce an ncrease n unemployment s lkely to affect some oblgors adversely. Calhoun and Deng (2002) derve dynamc varables measurng probablty of negatve equty and mortgage premum. Both change over tme and have a postve affect on default. They also nclude the rato of 10-year to 1-year Constant Maturty Treasury yeld and fnd t statstcally sgnfcant for models of early repayment. For fxed-rate mortgages, the coeffcent ncreases for hgher ratos; the ratonale s that mortgagors are movng to adjustable-rate mortgages to take advantage of the short-term relatvely low nterest rates. Gerard et al (2008) found that nterest rates (6-month lbor rate) and unemployment rate are statstcally sgnfcant explanatory varables for both mortgage default and sales wth a postve affect on default, as we would expect, and a negatve affect on sales. All these studes show that both BVs and MVs are useful explanatory covarates for consumer credt rsk. However none of them report usng these dynamc models for forecasts or stress testng. Ultmately, fnancal nsttutons and regulators are nterested n consumer credt rsk models for estmaton of future losses at both account and portfolo levels, ether n normal (expected) crcumstances or consderng adverse condtons. For ths reason, we focus prmarly on usng the models for forecastng PD and default rate. For a large database of UK credt cards we establsh the followng new results. Frst, the ncluson of BVs mproves model ft and also mproves forecasts. The best results are acheved wth the most recent behavoural data. Second, several MVs are found to be statstcally sgnfcant explanatory varables of default, but ths does not translate nto mproved forecasts at the account- 5 of 36
level. Thrd, we show that ncludng MVs does mprove estmaton of loss (default rate) at a portfolo or aggregate level. Fourthly, we show how models wth MVs can be used for stress testng and report a loss dstrbuton based on Monte Carlo smulaton of economc condtons. In secton 2 we outlne the methods we use, descrbng the dscrete survval model, test procedures and stress testng methodology. In secton 3 we descrbe our data and present results n secton 4. Fnally, we gve concluson and dscussons n secton 5. 2. Methods 2.1 Dscrete survval model for dynamc credt scorng. We treat tme as beng dscrete and adopt the followng notaton. We denote calendar tme as c and a s the date that account was opened. Let t be the number of months snce an account was opened (duraton tme). The term d t ndcates whether account defaults at tme t after account openng (0=non-default, 1=default). The term w s a vector of statc AVs collected at tme of account applcaton and x t s a vector of BVs collected across the lfetme of the account. The term z t s a vector of MVs whch s the same for each account on the same date; that s, for any two accounts, j havng records for duraton tmes t and s respectvely, f a + t = a j + s then z t = z js. We model the probablty of default (PD) for each account at tme t as P t = Pr = F ( d = 1 d = 0 for all s < t w x z k l) t s t a + t T ( α + φ β 4 ) T T T () t β1 + w β 2 + x( t k ) β3 + z ( a + t l ) ;,,,, (1) 6 of 36
where k and l are fxed lags on BVs and MVs respectvely; φ s a vector transformaton functon of duraton that s used to buld a parametrc survval 2 ( ) 2 model and n partcular, we use the transformaton ( t ) = t, t,logt,( logt) φ α s an ntercept and β 1, β 2, β 3, β 4 are vectors of coeffcents to be estmated. F s a gven cumulatve dstrbuton functon. We use the logt functon x ( e ) F( x) = 1 1+. We ensure that the underlyng panel data s constraned by the condton n (1): that s, no observatons are recorded after the frst default on any account. Gven ths condton, the model s a proportonal odds dscrete survval model wth the falure event defned as default. It can be estmated usng standard maxmum lkelhood estmaton for logstc regresson (Allson 1995). Coeffcent estmates on duraton φ ( t) gve a baselne hazard. If t ncluded dummy varables for each dscrete tme then the coeffcents would form a non-parametrc baselne hazard and model (1), overall, would be a sem-parametrc model parallelng the commonly used Cox proportonal hazard model. Publshed studes suggest there s a common shape to the dstrbuton over duraton tme of default hazard rates: they rse sharply wthn the frst few months before they begn to fall steadly over the remanng duraton of the account (Gross and Souleles 2002, Fgure 1, and Andreeva 2006, Fgure 1). We use a parametrc form for φ snce ths allows us to capture ths structure of hazard over tme. Log terms are ncluded to allow ths structure to take a skewed shape. The estmated survval probablty of an ndvdual at some tme t s gven as the product of the probablty of not falng at each tme perod condtonal on not havng faled prevously. That s 7 of 36
t () t = ( 1 ) Sˆ. (2) P s s= 1 The falure probablty 1 ˆ () t then gves PD wthn tme t whch s a typcal S measure of PD and can be used n further analyss, at the account or portfolo level, for credt scorng and computng captal requrements. To compare performance of dfferent model components such as BVs and MVs we consder the followng specal cases of model (1): 1. Duraton only: fx β 2, β 3, β 4 to zero. 2. AV only: fx β 3, β 4 to zero. 3. AV and BV only: fx β 4 to zero. 4. AV, BV and MV: all coeffcents are estmated. The lag k on the BVs restrcts the range of forecasts that can be made by the model, snce a perod k after our observaton date, there wll no longer be any behavoural data avalable to make estmates. For example, f the lag s 6 months then we can only forecast usng the BV model up to 6 months ahead. Clearly the longer perod we can forecast forward, the better. However, we would expect that f longer lags were used, forecast performance would deterorate. So we have a trade-off. We expect forecasts of 6-12 months ahead to be useful and so we consder lags of 12, 9 and 6 months. We also consder a 3 month lag model, even though ths s not such a useful forecast perod, for comparatve purposes over short lag perods. It s also possble that some BVs are endogenous varables. For example, there may be a common underlyng factor whch causes both an ncrease n account balance and default. Then hgh balance s not a cause of default, although t may be found to be an mportant 8 of 36
drver of default n the model. The shorter the lag perod, the more lkely ths connecton, whch s a further reason why longer lags are preferable, and so we report the BV lag 12 explanatory model. Nevertheless, we note that although endogenety affects the dentfcaton of cause, t does not affect forecasts whch are the man concern of ths paper. The mplcatons of the lag term l on the MVs are dfferent. The MVs can be estmated usng standard autoregressve methods (Hamlton 1994) or may be used wth smulated values durng stress testng. For ths reason we can use MV values at tme of default. In partcular, snce we defne default as 3 consecutvely mssed payments, we use 3 months lag on MVs to correspond wth the begnnng of mssed payments leadng to default. 2.2 Forecastng procedure Credt rsk models can be used to explore causal hypotheses of consumer credt behavour; for example Calhoun and Deng (2002) explore the dynamcs and causes of mortgage termnatons. However, for fnancal nsttutons and regulators these models typcally have value for estmaton of the rsk to ndvdual accounts or losses on credt portfolos. In ths way, banks can assess possble future losses and calculate captal requrements as buffers aganst adverse loss (FRS 2009). It s n ths forecast capacty that we assess these models. Followng Granger and Huang (1997) we dvde the panel credt data set nto an n-sample tranng data set T and a post, outof-sample test data set S. Models of default are bult on T and estmates of default are measured on S. In detal, accounts are randomly sampled so that the rato r of number of accounts n T to S s fxed. Then gven a calendar date at the tme of observaton, 9 of 36
Ω, accounts n T are rght censored so they were opened pror to the observaton date and only those records pror to the observaton date are ncluded (e t + a Ω ). Set S also ncludes accounts opened pror to the observaton date (snce n a real-world stuaton accounts opened after would be unknown) but only the post-observaton date records are n the test set (e t + a > Ω ). That s they are left censored. Ths procedure unfortunately results n a large number of records beng removed, but the random samplng ensures that no bas s ntroduced when generatng the out-ofsample test set, whlst the censorng ensures all predctons are forecasts. As a practcal matter, fnancal nsttutons could use post, n-sample data sets for forecasts and these may well gve more accurate results. However, for ths exercse, to avod over-fttng and the ntroducton of bas, forecasts are restrcted to an out-of-sample data set (Granger and Huang 1997). 2.3 Performance measures We use the log-lkelhood rato (LLR) to measure model ft for each model separately and also to test goodness-of-ft for nested models. But LLR only measures model ft on the tranng sample and not accuracy of forecasts. Snce we are usng survval models whch model tme to default, the usual predctve performance measures for classfcaton algorthms, such as error rates and the Gn coeffcent, do not naturally apply, nor do the standard resduals for regresson such as mean square error. Survval analyss has ts own resduals related to how well the estmated survval probablty matches the observed (true) tme of default. In partcular, these resduals take account of censored data. One such useful measure s the devance resdual gven by r D ( r )[ 2{ r + log( r )}] 1/ 2 = sgn δ δ (3) M M M 10 of 36
where r M * = δ r s the martngale resdual and r = log Sˆ ( t ) s the Cox-Snell C C resdual where t * s the last observaton avalable for account and δ = d t ndcates whether t faled. The martngale resduals takes account of whether or not an ndvdual fals but unfortunately they are not symmetrcally dstrbuted about zero nor are they addtve terms. The devance resduals have the advantage that they are approxmately symmetrcally dstrbuted and the sum of ther squares forms the * statstc l og ˆ R = r = 2 2 D ( L L ) C l og ˆ f (4) where Lˆ C and Lˆ f are the maxmum partal lkelhood under the current and the full model respectvely. The full model mples a model wth perfect ft to the data, therefore R gves a measure of log-lkelhood devance of the estmated model from the best case. Therefore models yeldng smaller values of R gve better ft (Collett 1994). Many of the propertes of martngale devance resduals are proved under the assumptons that, frstly, the survval model s a Cox proportonal hazards model and, secondly, that no TVCs are ncluded n the model (Therneau et al 1990). Unfortunately nether assumpton s true n our analyss. Nevertheless, the devance resdual s a typcal resdual for survval analyss and so, supposng robustness, we report devance calculated over the test set as one of our performance measures. However, we also derve an alternatve resdual based drectly on the log-lkelhood of survval for each test case gven a model. The resdual for each account s the negatve of the log of the probablty of the seres of events for the account, gven the model. The lower ths s, the better the model s predcton matches outcome. The 11 of 36
contrbuton of each ndvdual to the log-lkelhood functon for dscrete survval analyss usng the logstc functon s L = t * s= 1 = δ log P d s log P * + s + ( 1 d ) log( 1 P ) s t 1 * ( 1 δ ) log( 1 P ) + log( 1 Ps ) s * s= 1 (5) * where P = P * denotes the hazard probablty of the last observaton, rememberng t that only the last observaton can fal wthn the survval analyss framework. From (2) and (5) t then follows that the log-lkelhood resdual s * * ( P /( 1 P ) L = r δ log (6) C whch, nterestngly, s smlar to the Martngale resdual, except for a change of sgn * * and the account-specfc scalng term on defaults log( /( 1 P ) P. The advantage of (6), however, s that t s meanngfully addtve snce ts sum over all ndvduals s the negatve of the log-lkelhood statstc. The devance and log-lkelhood resduals above are desgned for assessng forecasts at the account level. However, our models can also be used to forecast at an aggregate level: eg across accounts wthn a sngle portfolo. The observed default rate for an aggregate of N accounts at a partcular calendar date c s gven by D c = 1 N N = 1 d ( c a ) (7) whch, assumng ndependence between default events, mples that the estmated default rate forecast gven by a partcular model s E 1 N ( Dc ) = P ( c a ) N = 1. (8) 12 of 36
The dfference between expected and observed default rate then gves a measure of performance for aggregate forecasts. 2.4 Stress testng We consder a smulaton-based stress test of default rate on an aggregate of accounts usng Monte Carlo smulaton (see eg Marrson 2002). The procedure s as follows. 1. Buld a dynamc model wth MVs from a tranng data set. 2. Generate a smulaton of economc condtons usng values of MVs based on hstorc macroeconomc data. 3. Smulate default events on test data by substtutng the smulated MV values nto the model. 4. Repeat steps 2 and 3, m tmes to buld a loss dstrbuton of estmated DR over dfferent economc scenaros. 5. Use the loss dstrbuton to compute estmated DR for extreme economc crcumstances. Stress tests should consder unexpected but plausble events. When m s large, suffcent extreme events can be smulated to meet the frst crtera; basng the smulatons on hstorcal data ensures the second. Value at Rsk (VaR) s defned as the maxmum expected loss, wthn a certan tme perod, for a gven percentle, q. Sometmes VaR s used to compute stressed values n step 5. However, VaR captures worst loss n normal crcumstances, whereas stress tests should consder losses durng unusual crcumstances. Therefore VaR may not be the approprate measure of loss durng adverse condtons (BIS 2005). For ths reason we also consder expected shortfall as a measure of loss. Ths s defned as the expected (mean) loss n the upper q percentle of the loss dstrbuton, for a gven q. 13 of 36
Usng the latent varable model of logstc regresson (Verbeek 2004, secton 7.1.3), we can smulate default rates for some calendar tme perod c, gven a model, a vector of macroeconomc condtons z, and a vector of N ndependent resdual terms,, e = e L e, each cumulatvely dstrbuted as F, as ( ) () 1 ( N ) ˆ D c, 1 N N I ˆ ˆ ˆ ˆ ˆ ( 1 > 0) T T T T ( z ε) = + φ() t β + w β 2 + x( t k ) β 3 + z ( a + t l ) β 4 + e() = 1 α (9) where I () s the ndcator functon. Monte Carlo smulaton can then be used to approxmate expected shortfall default rate wth S 1 qm q Dˆ c qm j= 1 (, e ) z (10) where j=1 to m, each z j s generated by macroeconomc smulaton and e j are N generated randomly from F and both are ndexed so that the smulated default rates are n descendng order; e for all h j, D ˆ ( z, e ) Dˆ ( z, e ). The number of teratons m s chosen so that (10) converges to a stable value. Ths smulaton takes nto consderaton the error n the model represented by the resdual terms, e j, along wth changes n macroeconomc condtons. Ths s natural, snce otherwse the pont predctons of equaton (8) are wrongly assumed to be exactly correct. c h h j j c j j Smulated values for MVs could be drawn, navely, drectly from hstorc values. However, ths would not preserve the structure of dependences between the MVs and so wll yeld mplausble scenaros and lead to msleadng results. To preserve the covarance structure between MVs we use Cholesky decomposton (Marrson 2002). If V s a matrx of covarances for hstorc macroeconomc data then t s decomposed by a lower trangular matrx L such that T V = LL. Then, f u j s a sequence of 14 of 36
ndependently generated values from the standard normal dstrbuton, z = Lu j j wll follow the covarance structure of V and so can be used as plausble economc smulatons. Cholesky decomposton assumes the varables are normally dstrbuted. However, ths s not usually the case for MVs and so we apply a transformaton to MVs f ths s requred, pror to smulaton. A Box-Cox transformaton s used snce ths often produces an approxmately normal dstrbuton (Box and Cox 1964). Alternatvely, we use an emprcal probt transformaton to mpose a normal dstrbuton on the hstorcal data. 3. Data 3.1 Credt card data We have three large data sets of UK credt card data coverng a perod from 1999 to md-2006 comprsng over 750,000 accounts. All data sets nclude AVs taken at tme of applcaton, along wth monthly account behavoural records. Most data are collected n the same way and have the same objectve meanng between credt card products, although dstrbutons vary snce dfferent products wll have dfferent demographc and rsk profles. Varables that may be defned dfferently for each product have not been used. A lst of varables used s gven n Table 2. Categorcal varables for employment and payment status are ncluded as a seres of ndcator varables. Age s dvded nto a seres of age category ndcator varables snce age has a non-lnear relatonshp to PD. All monetary values such as ncome and balance are gven as log values n order to normalze ther dstrbutons. There s a small proporton of mssng values for monthly payment amount so an ndcator varable s also ncluded for these varables. Also, there are a large proporton of zero values for 15 of 36
some BVs, payment amount, sales amount and APR, so ndcator varables are ncluded for those cases too. In these experments, we defne an account as n default when t goes three consecutve months delnquent on payments. Ths s a common defnton n the ndustry and follows the Basel II conventon of 90 days delnquency for consumer credt (BCSC 2006). The data we use for our analyss s commercally senstve and therefore we cannot provde further detals, data descrpton statstcs or report the observed default rates. To assess forecasts, an observaton date of 1 January 2005 s set. Snce the data runs to md-2006, ths provdes up to 18 months of test data, whch s a good perod for forecasts, whlst allowng for a long run of tranng data. We set the tranng/test data set splt rato r=2/1. After censorng, usng the procedure descrbed n secton 2.2, ths gves over 400,000 and 150,000 accounts n the tranng and test sets respectvely, provdng suffcent observatons for tranng whlst leavng a good number of accounts out-of-sample for forecasts. 3.2 Hstorc UK macroeconomc data We consder several UK MVs for whch we had a pror expectaton of ther havng an effect on PD. These are lsted n Table 1. In a prevous study on a dfferent data set (2008) found that bank nterest rates, earnngs, producton ndex and house prce were statstcally sgnfcant explanatory varables of UK default so we nclude these. Producton ndex s used nstead of GDP snce t s avalable monthly, whereas GDP fgures are only provded quarterly. Gerard et al (2008) also found unemployment rate was sgnfcant for US defaults, and Breedon and Thomas (2008) found varables for consumer sales and prces were correlated to default and bankruptcy n a study of a number of stressed credt markets worldwde, usng a 16 of 36
dynamc model. Therefore, we also nclude MVs for these rsk factors. Addtonally, FTSE ndex and a consumer confdence ndex are also ncluded snce they may be good ndcators of confdence n the economy. TABLE 1 HERE Many of the MVs have a tme trend. For example, earnngs and housng prce have an obvous upward trend, whereas nterest rates have an overall downward trend from 1999-2006. Ths could be a problem snce default rate also has a tme trend and so model ft could smply be due to fttng ths trend, rather than the macroeconomc condton tself. Therefore, to reduce ths possblty, all MVs are ncluded n the model as dfference varables over 12 months. Addtonally, log values are taken for those MVs wth clear exponental growth: earnngs, FTSE and house prces. For stress testng, hstorcal values of MVs are taken from 1986 to 2004; e only MV data pror to the observaton date s ncluded. We expermented usng nteracton terms between MVs and BVs and AVs, snce dfferent rsk groups may be more susceptble to economc changes than others. However, as wth (2008), we dd not fnd ther ncluson mproved model ft or forecasts, and ndeed made them worse. For ths reason we do not report results usng nteracton terms. 4. Results We present results n fve subsectons. Frstly, we present the underlyng hazard rate for default. Secondly, we dscuss coeffcent estmates from the model buld. 17 of 36
Thrdly, we present model ft and forecastng results at the account level. Fourthly, we gve forecast results at the aggregate level. And lastly we present results for stress testng. 4.1 Hazard rate for default The duraton only model provdes ntal baselne hazards. Fgure 1 shows the shape of hazard probablty over tme. It has the typcal survval profle for consumer credt: PD peaks early at 8 months then slowly declnes over tme as those hghly lkely to default drop out. Ths structure has been reported by others; see eg Gross and Souleles (2002) and Andreeva (2006). Fgure 1 also shows a small second rse n hazard, peakng around 36 months. Ths s because for all credt card products, accounts wth no recent usage are removed from the portfolos after two years. Snce these tend to be low rsk accounts, ther removal leads to a small overall ncrease n default rsk 1. FIGURE 1 HERE 4.2 Model and coeffcent estmates Many AVs and BVs and several MVs were statstcally sgnfcant explanatory varables. We focus attenton on the model for BV lag 12 months, snce ths s the most practcally valuable model n terms of forecast range. Table 2 shows coeffcent estmates for ths model. We fnd the followng key outcomes. Frst, the sgns on current balance (log) and ts square are opposte but the postve sgn on the square 1 It s therefore mportant to realze that the hazard rate s not just an ndcaton of oblgor s propensty to default but wll also be nfluenced by perodc operatonal decsons by portfolo managers. 18 of 36
term domnates. Therefore, balance outstandng on the account has an ncreasng postve effect on default hazard. Ths s unsurprsng snce a larger balance wll be more dffcult to clear. Second, An ncrease n credt lmt reduces the hazard. Intally ths may be surprsng snce we mght argue that a hgh credt lmt encourages hgher balance and therefore greater rsk. However, frstly, at least n the short term, a hgh credt lmt enables the oblgor to have a buffer to buld up debts before reachng default. Secondly, the bank sets the credt lmt based on ther own assessment of the oblgor s behavour, so credt lmt s actng partally as a proxy for a behavoural score. Thrd, the amount pad back each month, ndcated by payment status and payment amount, has a negatve effect on default. Ths s expected snce a greater ablty to repay mples that default s less lkely. Fourth, number of transactons has a postve effect on default. Ths s expected snce t ndcates greater card use and hence a rsng balance. However, nterestngly, the effect of transacton sales amount s negatve. A possble explanaton s that sales amount s actng as an ndcator of wealth when taken together wth number of transactons. That s, people who make a few bg purchases are more lkely to be wealther and therefore more able to repay than those who make many small purchases. Ffth, when behavoural data s mssng, PD decreases consderably. However, snce all duraton tmes up to 12 months wll not have BVs (because of the lag) ths s manly a jont effect wth duraton. Sxth, ndcator varables have been added for vntage, whch ndcates year of account openng. These are sgnfcant and therefore mply that cohorts explan some of the effect over tme for default rates wthn the data set. Ths s natural snce lenders wll allow greater or less rsky new accounts onto ther books at dfferent tmes, dependng on ther changng atttude to rsk at dfferent tmes n the busness cycle. 19 of 36
Concernng macroeconomc varables, nterest rate has a postve effect on default. Ths s expected snce rsng nterest rates mply greater demand for repayment on outstandng loans and mortgages whch wll adversely affect those people who are more hghly ndebted. Unemployment rate also has a postve effect on default. Unemployment rate s an ndcator of drect economc stress on ndvduals. In partcular, oblgors who become or reman unemployed wll fnd t more dffcult to repay debt. Conversely, f unemployment decreases, then we would generally expect unemployed oblgors to fnd jobs, therefore makng t easer for them to repay. Therefore, the effect of ths MV on default s as expected. The frst three fndngs corroborate the results of Gross and Souleles (2002) who bult dynamc models of default for US credt card data. They found rsk of default rses wth balance and falls wth repayments. They used utlzaton - outstandng balance dvded by credt lmt - nstead of the raw value of balance, whch s sensble gven the relatonshp dscussed n pont 2. Also they had the same outcome for nterest rates and unemployment descrbed n ponts 7 and 8. (2008) found smlar results for nterest rates on a dfferent UK credt card data set, although ths study dscovered earnngs to be a more mportant MV than unemployment. The macroeconomc effects also corroborate the study by Breedon and Thomas (2008) across several world-wde data sets, although they also found GDP to be sgnfcant n many cases. They also ncluded vntage effect n ther models. These results are for the model wth lag 12 month BVs. We found smlar results for models wth shorter lag perods and the comments made above also hold n these 20 of 36
cases, except that the effects and statstcal sgnfcance tends to be stronger for models wth shorter lags. 4.3 Model ft and forecasts of tme to default Model ft s shown n Fgure 2 for several alternatve models. Ths shows a general mprovement n model ft as we move from the smple duraton only model to the AV only model to the AV and BV model. Addtonally, we also observe that model ft mproves wth shorter lag on BVs wth a relatvely large mprovement at 3 month lag. However, as we have dscussed, ths mprovement comes at the prce of a much shorter range of forecasts. We see n Fgure 2 that, although some of the MVs are statstcally sgnfcant, ther contrbuton to model ft s weak. Nested model ft s also assessed wth results shown n Table 3. Ths shows that addng BVs to the model gves a statstcally sgnfcant mprovement n ft and also addng MVs to the model gves a statstcally sgnfcant mprovement, even though ths s small. FIGURE 2 HERE TABLE 3 HERE Fgure 2 also shows results of forecasts. These follow the model ft results very closely. They show a marked mprovement n ft for the BV models, mprovng wth shorter lags. However, there s no notceable change n forecast accuracy when MVs are ncluded. Also, both the conventonal devance resdual and the log-lkelhood resdual follow each other closely, mplyng that ether measure s suffcent for ths problem doman. 21 of 36
4.4 Estmaton of Default Rates Fgure 3 shows estmated default rates for dfferent models along wth the observed (or true) default rates for each month of the test data set. The monthly observed default rates have hgh varance but there s a general trend of hgh values begnnng n 2005, fallng durng 2005, then rsng agan n 2006. The AV model s able to model the general fall n default rates. However BVs are requred to forecast the overall trend ncludng the rse n 2006. However, the best forecasts ndcatng hgh DR n early 2005 and md-2006, whlst also forecastng the dp n default rates at the end of 2005 are only made when MVs are ncluded n the model. The BV model, lag 3 months, also performs well, but ths s not surprsng gven the short forecast perod, usng behavoural data just one month before accounts begn mssng payments. Overall the BV lag 12 month model wth MVs performs best at forecastng aggregate DR, achevng better results than even BV models wth shorter lags as demonstrated n Table 4. FIGURE 3 HERE TABLE 4 HERE 4.5 Stress test results We ran Monte Carlo smulatons usng the MV model gven n Table 2. Estmated DR was smulated on the test data set, 12 months followng the observaton date; e for December 2005. A stable loss dstrbuton was generated after m=25,000 smulatons and s shown n Fgure 4. The rght-hand tal shows rsk for more adverse condtons. In partcular we have ncluded the fgure for expected shortfall at the 99% percentle. Ths shows that for the worst 1% of economc scenaros we consder, the expected DR s 1.73 tmes greater than normal condtons (e medan estmated DR). 22 of 36
VaR s also shown for comparson. We see that ths gves a lower estmate of loss (1.59) whch may not reflect extreme crcumstances suffcently. These fgures are slghtly hgher than those suggested as part of the US stress testng exercse by FRS (2009). In partcular the FRS study estmates a more modest rse between 20% and 55% n DR when contrastng a normal baselne fgure to more adverse condtons 2. But our results appear lower than those of Rösch (2004) who found a VaR for US credt cards to be 2.31 tmes the mean, although he used aggregate data, not account level data as we do. FIGURE 4 HERE 5. Concluson Dynamc models are a flexble approach to model and forecast consumer credt rsk. They have a number of well known advantages over statc models ncludng modellng the condtonal probablty of default n a specfc tme perod rather than n a tme wndow and enablng the predcton of the proftablty of specfc loans ( 2009). We have used dscrete-tme survval analyss to model credt card rsk. Ths has two man advantages. Frstly, t s a prncpled means to buld dynamc models of default and, secondly, modellng and forecastng s computatonally effcent when compared to commonly used contnuous-tme survval models. Ths s mportant when model bulders use large databases of credt accounts. 2 FRS (2009) gves baselne two-year loss rates as 12-17% and more adverse as 18-20%. Takng the lower and upper bounds on each range and convertng to an average monthly DR gves 20-55% expected ncrease n loss. Takng a mean value for baselne and more adverse (14.5% and 19% respectvely) gves a mean ncrease of 34%. 23 of 36
We have used a large data set of UK credt card accounts to test the effectveness of dynamc survval models wth BVs and MVs as models of default. Unlke prevous lterature we explore these models as tools for rsk measurement, forecastng and stress testng. We conclude that many BVs are statstcally sgnfcant explanatory varables of default and ncludng them gves mproved model ft. Important BVs are account balance, repayments, number of transactons wthn each month and credt lmt. We fnd model ft translates nto mproved forecasts of tme to default. Performance mproves wth shorter lags on BVs. Ths s expected snce shorter lags mply that the model s usng more recent nformaton about the oblgors. However, we also note that shorter lags mply shorter ranges of forecasts and greater endogenety between BVs and the default event. For ths reason we focus on lag 12 month BVs. Ths gves mproved performance, relatve to the AV only model, and also allows for useful forecasts up to 12 months ahead. Second bank nterest rates and unemployment rate sgnfcantly affect the hazard. Whlst ther ncluson gave only a modest mprovement n model ft and no notceable mprovement n forecasts of tme to default at the account level, ther ncluson mproves forecasts of default rate at the aggregate level. Ths s understandable snce MVs affect all predcted PDs, rather than at the ndvdual account level. Hence ther affect wll only become notceable at the aggregate level where accounts are taken together. Where comparable our results corroborate results gven by others (Gross and Souleles 2002, Calhoun and Deng 2002, Gerard et al 2008). 24 of 36
Thrd, The ncluson of MVs enables stress tests whch generate credble results ndcatng that adverse condtons may rase DR by around 79%. We used a smulaton-based approach for our experments but scenaros could also be desgned and used wth these models. Acknowledgements We are grateful for fundng through an EPSRC grant EP/D505380/1, workng as part of the Quanttatve Fnancal Rsk Management Centre. 25 of 36
References Allson PD (1995). Survval analyss usng SAS. SAS Press. Andreeva G (2006). European generc scorng models usng survval analyss. Journal of the Operatonal Research Socety 57 pp 1180-1187. Basel Commttee on Bankng Supervson BCBS (2005). Basel II: Internatonal Convergence of Captal Measurement and Captal Standards at www.bs.org/publ/bcbsca.htm Bank for Internatonal Settlements BIS (2005). Stress testng at major fnancal nsttutons: survey results and practce. Workng report from Commttee on the Global Fnancal System. Bellott T and Crook J (2009). Credt scorng wth macroeconomc varables usng survval analyss. The Journal of the Operatonal Research Socety, vol 60, no 12, December, pp1699-1707. Boss M (2002). A macroeconomc credt rsk model for stress testng the Austran credt portfolo. Fnancal Stablty Revew 4, Oesterrechsche Natonalbank. Box GEP and Cox DR (1964). An analyss of transformatons. Journal of the Royal Statstcal Socety, Seres B 26: pp211 246 Breedon J and Thomas L (2008). The relatonshp between default and economc cycles for retal portfolos across countres. The Journal of Rsk Model Valdaton 2(3), 11-44. Calhoun CA and Deng Y (2002): A dynamc analyss of fxed- and adjustable-rate mortgage termnatons. Journal of Real Estate Fnance and Economcs 24:1/2 pp 9-33. Collett D (1994). Modellng survval data n medcal research. Chapman & Hall: London. Crook J and Bellott T (2009) Asset Correlatons for Credt Card Defaults. Workng paper, Credt Research Centre, Unversty of Ednburgh Busness School (March 2009). Fnancal Servces Authorty FSA (2008). Stress and scenaro testng. Consultaton paper 08/24 FSA: UK. Board of Governors of the Federal Reserve System FRS (2009). The supervsory assessment program: overvew of results. FRS: USA. Crook J and Bellott T (2009). Tme Varyng and Dynamc Models of Consumer default. Journal of the Royal Statstcal Socety, Seres A (forthcomng). 26 of 36
Gross DB and Souleles NS (2002). An emprcal analyss of personal bankruptcy and delnquency. The Revew of Fnancal Studes Vol 15, no 1, pp319-347. Gerard K, Shapro AH, Wllen PS (2008). Subprme outcomes: rsky mortgages, homeownershp experences, and foreclosures. Workng paper 07-15 Federal Reserve Bank of Boston. Granger CWJ and Huang LL (1997). Evaluaton of panel data models: some suggestons from tme seres. Dscusson paper 97-10, Department of Economcs, Unversty of Calaforna, San Dego. Hamlton JD (1994). Tme Seres Analyss. Prnceton Unversty Press. Ma P, Crook J and Ansell J (2009) Modellng take-up and Proftablty. Journal of the Operatonal Research Socety (1 Aprl 2009) do:10.1057/jors.2009.33 Specal Feature. Marrson C (2002). Fundamentals of Rsk Measurement (McGraw-Hll NY). Rösch D and Scheule,T. (2004). Forecastng Retal Portfolo Credt Rsk. Journal of Rsk Fnance, Wnter/Sprng, pp16-32. Rösch D (2003). Correlatons and busness cycles of credt rsk: evdence from bankruptces n Germany. Swss Socety for Fnancal Market Research vol 17(3) pp 309-331. Therneau TM, Grambsch PM and Flemng TR (1990). Martngale based resduals for survval models. Bometrka 77 pp 147-160. Thomas LC, Ho J and Scherer WT (2001). Tme wll tell: behavoural scorng and the dynamcs of consumer credt assessment. IMA Journal of Management Mathematcs (2001) 12 pp 89-103. Verbeek (2004). A Gude to Modern Econometrcs (2 nd ed, Wley). 27 of 36
Tables Table 1. Descrptve Statstcs for Macroeconomc Varables (MVs) MV Descrpton Source Descrptve statstcs (for dfference n value over 12 months) Mn Mean SD Max IR UK bank nterest rates ONS -4.5-0.43 1.90 6.5 Unemp UK unemployment rate (n ONS -535-94 238 575 000s) SA Prod UK producton ndex (all) ONS -5.2 1.10 2.30 6 RS Retal sales value ONS 0.3 3.92 1.49 8.5 FTSE FTSE 100 all share ndex FTSE -822 81 286 682 HP Halfax House Prce ndex LBG -6.5% +7.9% 7.6% +26% RPI Earnngs CC Retal prce ndex (all tems) Earnngs (log) all ncludng bonus Consumer confdence ndex ONS 1.2 4.96 2.36 12.8 ONS 0.008 0.019 0.006 0.038 EC -20.3 0.7 24.2 186.8 The data s from1986 to 2004. Sources: UK Offce of Natonal Statstcs (ONS), Lloyds Bankng Group (LBG) and the European Commsson (EC). Data s monthly and may be seasonally adjusted (SA). 28 of 36
Table 2. Coeffcent estmates for model wth all AVs, BVs lag 12 months and MVs. Intercept Covarate Estmate n/a** Duraton 1.35** (squared) -0.00698** (log) 16.4** (log squared) -6.42** Selected applcaton varables (AV) Tme customer wth bank (years) -0.00250** Tme wth bank unknown + -0.342** Income (log) -0.146** Income unknown + -1.46** Number of cards -0.0610** Tme at current address -0.00129 Employment + : Self-employed 0.303** Homemaker 0.072 Retred 0.111 Student -0.035 Unemployed 0.231 Part tme -0.365** Other -0.037 Excluded category: Employed Age + : 18 to 24 0.074 25 to 29-0.058 30 to 33 0.010 34 to 37 0.100** 38 to 41 0.046 48 to 55-0.108** 56 and over -0.243** unknown -2.74** Excluded category: 42 to 47 Credt bureau score -0.00322** Product + : A 0.535** B 0.371** Excluded category: C Vntage (+): 1999-2003 n/a ** Behavoural varables (BV) lag 12 months Payment status + : Fully pad -0.390** Greater than mnmum pad -0.090** Mnmum pad 0.149** Less than mnmum pad 0.714** Unknown -0.148* Excluded category: No payment 29 of 36
Current balance (log) -1.58** (log squared) 0.517** s zero + -1.05** s negatve + -0.802** Credt lmt (log) -1.22** Payment amount (log) -0.154** s zero + -0.133 s unknown + -0.452** Number of months past due 0.134* Past due amount (log) 0.0795 s zero + -0.623** Number of transactons 0.00663** Transacton sales amount (log) -0.350** s zero + -0.567** APR on purchases -0.00487 s zero + -0.482** Behavoural data s mssng + -3.73** Macroeconomc varables (MV) lag 3 months Bank nterest rate 0.113** Unemployment rate 0.000672** Producton ndex -0.0101 FTSE all 100 (log) 0.0591 Earnngs (log) 1.57 Retal sales 0.00929 House prce (log) -0.218 Consumer confdence -0.00217 Retal prce ndex (RPI) -0.0298 Indcator varables are denoted by a plus sgn (+). Statstcal sgnfcance levels are denoted by astersks: ** s less than 0.001 and * s less than 0.01 level. Coeffcent estmates on the ntercept and vntages are not shown for reasons of commercal confdentalty. Only selected applcaton varables are ncluded for the same reason. 30 of 36
Table 3. Model ft Lne Nested model Compared to base model Dfference n 2 LLR Number of added covarables P-value 1. AV only Duraton only 22531 34 <0.0001 2. AV & BV lag 12 AV only 9237 22 <0.0001 3. AV, BV lag 12 & MV lag 3 AV & BV lag 12 47 9 <0.0001 4. AV & BV lag 9 AV only 7946 22 <0.0001 5. AV & BV lag 6 AV only 12458 22 <0.0001 6. AV & BV lag 3 AV only 26559 22 <0.0001 Results are for nested models usng dfference n LLR and a ch-square sgnfcance test. 31 of 36
Table 4. Mean absolute dfference between estmated and observed default rates across the test set Model Mean absolute dfference between estmated and observed DR AV only 0.087 BV lag 12 0.058 BV lag 12 & MV lag 3 0.049 BV lag 9 0.062 BV lag 6 0.070 BV lag 3 0.068 Results relate to models based on the 18 months of test results shown n Fgure 3. 32 of 36
Fgure 1. Hazard rate functon for parametrc duraton only model of default. Hazard probablty 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 Duraton (age of account n months) The hazard probablty scale s not shown for reasons of commercal confdentalty. 33 of 36
Fgure 2. Model ft and forecast performance for dfferent models. Model ft 170000 160000 150000 140000 130000 60000 55000 50000 45000 40000 Forecast 120000 35000 AV & BV lag 3 AV & BV lag 6 AV & BV lag 9 AV, BV lag 12 & MV AV & BV lag 12 AV only Duraton only Model Model ft: - log lkelhood rato Forecast: Devance resdual Forecast: - log-lkelhood rato 34 of 36
Fgure 3. Comparson of estmated and observed default rates for each month of the test data set. Jan-05 Feb-05 Mar-05 Apr-05 May-05 Jun-05 Jul-05 Aug-05 Sep-05 Oct-05 Nov-05 Dec-05 Jan-06 Feb-06 Mar-06 Apr-06 May-06 Jun-06 Observed or estmated Default Rate (DR) Observed DR AV only AV & BV lag 12 AV, BV lag 12 & MV AV & BV lag 3 The scale on the default rate axs s not shown for reasons of commercal confdentalty. 35 of 36
Fgure 4. Dstrbuton of estmated default rates Medan Observed DR VaR (99% level) Expected shortfall (99% level) 0.5 0.75 1 1.25 1.5 1.75 2 2.4 Estmated default rate (as rato of medan value) Regon of expected shortfall calculaton (99% level) The dstrbuton s based on smulaton of economc scenaros for credt card accounts durng December 2005, based on a model wth MVs traned on data pror to January 2005, shown as a hstogram. The observed DR for the test data set s shown along wth Value at Rsk (VaR) and expected shortfall at 99% probablty. All values are expressed as a rato of the medan estmated DR. 36 of 36