Credit Scoring Methods A Comparative Analysis

Credit Scoring Methods A Comparative Analysis North American Power Credit Organization Scottsdale: January 2013, Inc. 813 East Ballard Colbert, WA, 99005 grossmann@rmgfinancial.com Phone: (509) 468-2956 Fax: (509) 468-3217 Cell: (509) 990-0894 www.rmgfinancial.com aaaaaaa 1

Usefulness of Company Scoring Credit scoring allows for consistency in counterparty evaluations Provides indicative ratings for non-rated counterparties Adds supporting data inputs for: Estimation of counterparty creditworthiness Credit limit determination Estimation of credit reserves Capital usage calculations Capital adequacy calculations Other portfolio metrics aaaaaaa 2

Scoring Wholesale vs. Retail Credit scoring for retail accounts (individuals and small businesses) tends to have fundamental differences from scoring wholesale accounts: Wholesale energy transactions tend to be of high dollar value and spread across a small group of counterparties. Sample sizes are much smaller and do not often lend themselves well to statistical analysis. Counterparties are fragmented into several distinct company types. Each industry sector tends to have specific financial profiles. Financial data and ratings tend to clump together in narrow ranges. aaaaaaa 3

Common Methods of Credit Scoring Fundamental analysis Linear Regression Linear regression refresher (least squares) Linear regression as descriptive analysis Bankruptcy predictive regression Asset volatility (EDF) Multivariate weighted average scoring Peer group comparison aaaaaaa 4

Part I Fundamental Analysis aaaaaaa 5

Fundamental Analysis Banks and rating agencies employ fundamental analysis when analyzing financial strength. Both banks and rating agencies tend to have open access to company management and specific financial information and forecasts that may not be disclosed in public filings. Bottom up approach reviewing the individual financial status of each company in detail as a stand alone entity Top down approach putting each company in the context of its industry and the overall economy Banks and rating agencies use both quantitative and qualitative elements in their process of evaluating company creditworthiness There is no true substitute for thorough due diligence of one s trading counterparties and the more you know the better able you are to make informed credit decisions. aaaaaaa 6

Fundamental Analysis Importance of financial analysis: We all must now be financial analysts whether we like it or not. We need to be able to source and evaluate many types of market and financial information in determining the creditworthiness of our trading counterparties. 10-K s and 10-Q s are the most detailed source of information generally available on a company s financial standing, and we all need to be able to read, understand and adjust this information as necessary in performing adequate due diligence. Company management has incentives to smooth financial data to meet earnings expectations and debt covenants. Accounting rules and reported financial statement data does not necessarily reflect a company s true economic condition. aaaaaaa 7

Relying on Debt Ratings is Not Enough Ratings tend to lag the market: Rating agencies are hesitant to change company ratings due to the possible effects on market participants. A recent study determined that on average, downward rating changes lagged the market by approximately six months and upward rating changes lagged by approximately nine months. CDS Company Name Moody S&P Fitch Spread 1 Progress Energy, Inc. Baa2 BBB BBB 18.1 2 Duke Energy Corp Baa2 BBB BBB+ 30.0 3 Dominion Resources Inc Baa2 A- BBB+ 39.3 4 American Electric Power Company, Inc. Baa2 BBB- BBB 46.3 5 Northeast Utilities Baa2 A- BBB+ 46.6 6 TECO Energy Inc. Baa2 BBB BBB 55.0 7 Exelon Corp. Baa2 BBB BBB+ 74.8 8 Pepco Holdings Inc. Baa3 BBB BBB 80.4 9 NiSource Inc. na BBB- BBB- 82.2 10 Southern California Edison Co. A3 BBB+ A 88.5 11 DTE Energy Co. Baa2 na BBB 91.3 12 CenterPoint Energy, Inc. Baa3 BBB BBB 91.8 13 SCANA Corp Baa3 BBB BBB+ 101.8 14 CMS Energy Corp Ba1 BB+ BB+ 115.6 15 Entergy Corp. aaaaaaa Baa3 BBB- na 174.2 8

Part II Linear Regression Basic Least Squares Regression Model Regression as Descriptive Statistical Analysis Regression as Predictive Statistical Analysis aaaaaaa 9

Basic Linear Least Squares Model Form: Y = a + b 1 X 1 + b 2 X 2 + b 3 X 3 + b n X n + E Assumptions: Causality is pre-existing in functional form Form of relationship is indeed linear Independent variables are not cross correlated Sample used is non-biased Errors are independent and have mean value of zero Causes for error / bias: Errors of measurement / missing data Omitted Variables Incorrect functional form Multicollinearity (non-independence of variables) Autocorrelation (non-independence of errors) Heteroskadasticity (non-constant variance of errors) aaaaaaa 10

Basic Linear Least Squares Model A few of the key resulting statistics: R 2 - Measures goodness of fit or amount explained by the regression R 2 = 1 - Explained Sum of Squares Total Sum of Squares F Statistic Used to test the hypothesis that all coefficients in the regression taken together are not statistically significant at a predetermined level of probability. Standard Errors Used to test the hypothesis that individual coefficients are not statistically significant at a predetermined level of probability (.there are plenty of others) aaaaaaa 11

Basic Linear Least Squares Model R 2 & Goodness of Fit aaaaaaa 12

Basic Linear Least Squares Model Effects of outlier data: 10.00 9.00 8.00 N= 39 R 2 = 0.54 7.00 6.00 5.00 4.00 3.00 95% Confidence Interval 2.00 1.00 0.00 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 aaaaaaa 13

Basic Linear Least Squares Model Effects of outlier data: 10.00 9.00 8.00 7.00 N= 39 R 2 = 0.31 95% Confidence Interval 6.00 5.00 4.00 3.00 2.00 1.00 0.00 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 aaaaaaa 14

Linear Regression as Descriptive Analysis So, let s look at a simple regression for Investor Owned Utilities: Rating = a +b1(ptroa) + b2(ebit/int) + b3(td/tc) +E Stats Return on Assets Interest Cov'g Total Debt/Cap Intercept (x1) (x2) (x3) b mn..., b 0.238 0.298-0.038 13.267 sen, se b 0.114 0.144 0.029 2.195 r 2, se y 0.362 2.901 #N/A #N/A F, df 15.3 81 #N/A #N/A ss reg, ss resid 387 682 #N/A #N/A Pearson 0.539 0.438-0.544 aaaaaaa 15

Linear Regression as Descriptive Analysis Very good results? Is this model descriptive? Is this model predictive? aaaaaaa 16

Bankruptcy Predictive Regression Attempts to use regression analysis on historical financial data to determine a predictive model for company failure. Altman (1968): Multivariate analysis Z i = a 1 x i1 + a 2 X i2 + + a n x in where: x i = financial ratios successful company: failure: Z i => z Z i <= z The Altman Z-score was initially based on a sample of 66 manufacturing companies, 33 of which had filed for bankruptcy during the period 1946 through 1965. Altman s original model correctly identified 79% of the sample one year prior to failure. aaaaaaa 17

Bankruptcy Predictive Regression aaaaaaa 18

Bankruptcy Predictive Regression Zone of Ignorance 1.81 2.67 aaaaaaa 19

Altman s Model Z = 0.012 X 1 + 0.014X 2 + 0.033X 3 + 0.006X 4 + 0.999X 5 Where: X 1 = working capital / total assets X 2 = retained earnings / total assets X 3 = EBIT / total assets X 4 = market value of equity / book value of liabilities X 5 = sales / total assets Z-scores of greater than 2.99 clearly represent non-failure Z-scores of less than 1.81 clearly represent failure Altman subsequently developed a revised Z-score model which dropped variables X 4 and X 5 and replaced them with a new variable, X 4 = net worth (book value / total liabilities). Sales / total assets was dropped to minimize potential industry effects relating to asset turnover. aaaaaaa 20

Altman (continued) In 1977 Altman developed a private company, ZETA Services, Inc. that sold a new model touted as being, far more accurate in bankruptcy classification While the coefficients were not specified, the model was based on the following factors (see: Altman, E. I. (2000): Predicting Financial Distress of Companies: Revisiting the Z-Score and ZETA Models. Stern School of Business, New York University.): Return on assets Stability of earnings Debt service Cumulative profitability Liquidity / current ratio Capitalization (five year average of market value) Size (total tangible assets) aaaaaaa 21

Other Bankruptcy Prediction Models Gambler s ruin model - Wilcox (1976) Assumes that net assets follow a random walk process with some fixed probability of a negative cash flow in each period. For long enough periods there is a probability for a clustering of net negative cash flows that will exceed net assets and the ability to borrow. Ohlson s O-Score (1980) Ohlson employed logit regression on a much larger sample size of 105 bankrupt and 2,058 non-bankrupt companies. Support Vector Machine Model (several authors) Employs statistical learning theory and artificial neural networks approach to finding specific solutions while complex the approach works well with small sample sizes aaaaaaa 22

Linear Regression and Bankruptcy Prediction Models The usefulness of regression analysis in determining default, while interesting, is questionable. The usefulness of fail / non-fail prediction models as described by Ohlson: real world problems concern themselves with choices which have a richer set of possible outcomes. No decision problem I can think of has a payoff space which is partitioned naturally into the binary status of bankruptcy versus non-bankruptcy Most of the analysis should simply be viewed as descriptive statistics which may, to some extent, include estimated prediction error-rates, and no theories of bankruptcy or usefulness of financial ratios are tested. aaaaaaa 23

Part III Asset Volatility Models aaaaaaa 24

Asset Volatility Models Asset volatility models are thought to be more accurate than the older probability of bankruptcy (PB) model s, such as Altman and Ohlson since: While PB estimates are statements of the likelihood of future events they rely on accounting information that is designed to measure past performance Financial statement info is formulated based on the accounting principle of ongoing-concern which assumes a company won t go bankrupt, thus the PB model is limited by design Accounting principles incorporate conservatism and can cause assets to be underestimated, injecting bias into the structure of the PB model PB models do not incorporate asset volatility which captures the likelihood that the value of a firm s assets may decline (two companies with the same Debt/Capital ratio may have significantly different likelihoods of default) aaaaaaa 25

Asset Volatility Models Asset volatility models use a Black/Scholes/Merton (BSM) option pricing theory to determine a debt/equity-based estimate of default probability as of a point in time. Asset volatility assumes that a company becomes more likely to default as the market value of the company converges to the value of the company s debt - much like an option is assumed more likely to be exercised as the market price converges to the strike price of the option. Asset volatility models employ market data (share price volatility) and financial data (debt levels) to calculate estimated default frequencies (EDF). Sources for EDF s include: - Moody s KMV - Bloomberg - CreditGrades. aaaaaaa 26

Asset Volatility Models Asset values are unobservable, but using the equity value today (which is observable) and the Black-Scholes model, the asset value (and asset volatility) today can be estimated. Standard Black-Scholes-Merton: V E = N(d 1 ) e -rt XN(d 2 ), where: d 1 = (ln (V A /X) + (r + ½ σ A2 /2)T) / σ A T -½, and d 2 = d 1 - σ A T -½ DD = (ln (V A /X) + (μ + ½ σ A2 /2)T) / σ A T -½ P def = N(-DD) V E = equity value X = book value of debt r = risk free rate V A = asset value σ A = asset volatility DD = Distance to default T = time to maturity of longest debt N = normal distribution P def = Probability of default aaaaaaa 27

EDF Example aaaaaaa 28

Relying on Market Indicators is Not Enough Market indicators, such as estimated default frequencies (EDF s), credit swap rates and bond spreads provide good leading indicators of deteriorating or improving credit quality, but market indicators don t tell you WHY a company s credit quality is improving or declining. Market indicators such as EDF s are subject to share price volatility and market perceptions and tend to overstate downturns and understate upturns in credit quality through a business cycle. Bond spreads and share price volatility may reflect more factors than simply credit premiums / credit risk. Market indicators assume market and price transparency some bonds or equities may have limited transactions: not all markets are fully transparent. aaaaaaa 29

Part IV Multivariate Weighted Average Models aaaaaaa 30

Multivariate Weighted Average Models Multivariate weighted average (MWA) scoring models use financial and non-financial indicators to produce company rankings. Weighting factors and scoring tables act to compare companies to their peers within industry groups. Employs a straightforward methodology which is easily understood. Scoring results can easily be mapped to common financial scales such as debt ratings. A well designed and well tested MWA model can produce consistent and accurate results. aaaaaaa 31

Basic MVWA Model Form: Assumptions: Causality is pre-existing in functional form Choice of ratios are significant (appropriate) Weightings are appropriate Causes for error / bias: Errors of measurement / missing data Omitted or incorrect ratios Incorrect weightings (Ranked AAA to D) Ratio (1) Scoring Vector (1) Weighting (1) Ratio (2) Scoring Vector (2) Weighting (2) Ratio (3) Scoring Vector (3) Weighting (3) Score =. X. X....... Ratio (n) Scoring Vector (n) Weighting (n) aaaaaaa 32

MVWA Model Issues Choice of weighting factors: Which factors should be used? How should they be weighted? How to make an informed decision? Development of scoring tables: How to determine discrete scoring vectors? Ranges of sample data may be narrow Samples sizes are often small Data may not be statistically significant When initial results are not statistically significant, the art of financial analysis becomes as useful as the science of statistical analysis. aaaaaaa 33

MVWA Model Issues Choice of financial ratios: Pearson correlation results of Debt Ratings to various financial ratios Sample Size (Over 5 year-ends and 1 Q3) 20 69 126 125 Ratio Merchants IOU Publics Corp 1 Pre Tax Return On Equity 2 Pre Tax Return on Assets 3 Operating Income / Sales 4 TNW / Total Assets 5 Operating Cash Flow / Total Debt 6 Funds from Operations / Total Debt 7 EBIT / Interest Expense 8 EBITD / Total Debt 9 Total Debt / Total Assets 10 Total Debt / Total Cap 0.11 0.23 0.01 0.02 0.40 0.43 0.03 0.44 0.30 0.29 0.04 0.29 0.36 0.41 0.09 0.09 0.38 0.30 0.05 0.12 0.40 0.43 0.03 0.20 0.09 0.18 0.01 0.43 0.35 0.34 0.03 0.21-0.70-0.52-0.13-0.14-0.53-0.57-0.16-0.40 aaaaaaa 34

MVWA Model Issues Limited range of data sets: Investor Owned Utilities aaaaaaa 35

MVWA Model Issues Limited range of data sets: Public / Municipal Utilities aaaaaaa 36

MWA Model Issues Limited scope and range of data sets: Power Cooperatives aaaaaaa 37

Part V Peer Group Analysis aaaaaaa 38

Basic Peer Group Model Form: (Ranked by quartile) Ratio (1) Quartile Averages (1) Weighting (1) Ratio (2) Quartile Averages (2) Weighting (2) Ratio (3) Quartile Averages (3) Weighting (3) Score =. X. X....... Ratio (n) Quartile Averages (n) Weighting (n) Assumptions: Causality is pre-existing in functional form Choice of ratios are significant (appropriate) Weightings are appropriate Causes for error / bias: Errors of measurement / missing data Omitted or incorrect ratios Incorrect weightings aaaaaaa 39

Peer Group Analysis Peer group analysis is a simple statistical approach to multivariate weighted average scoring. For a quick and easy way to build scoring tables why not just compare a company s financial ratio results to other companies in its SIC code? Method: - Define the SIC codes for each company type (peer groups) - Access financial data for all companies within each SIC code - Choose a few financial ratios and run them for all companies within each SIC code - Stratify the resulting peer group data around mean values And you have an easy way to build discrete scoring vectors to score companies against, right? aaaaaaa 40

Peer Group Analysis - Limitations Structure of peer group data Data may fall within a limited range. Compressed strata may bias results. Choice of peer groups SIC code groupings should be screened. Some companies within an SIC Code Classification may not belong. Peer group size Peer group must be at least large enough to adequately calculate scoring vector. Market cyclicality and peer group bias Peer groups may trend together over time and through a business cycle. Omission bias Missing data will imply bias in calculating scoring vectors and applying rankings. aaaaaaa 41

Peer Group Analysis - Summary Peer group comparison is only useful if one has: - Well thought out methods - Well defined peer groups - Large sample sizes - Broad ranges of sample data - A complete data set for each peer group - Data that doesn t track together over time So peer group analysis is only useful if you have thoroughly analyzed the financial profile of each company type and the data being used and back tested your model results. aaaaaaa 42

Other Topics aaaaaaa 43

Unadjusted Financial Data is Not Enough Company management has incentives to smooth financial data to: Meet earnings expectations Satisfy debt covenants Qualify for bonuses Just because they can Accounting rules do not necessarily reflect a company s true economic condition an issue of Accounting vs. Economic Value. Accounting value follows GAAP accounting rules that allow for timing adjustments, deferrals of expenses and the matching of asset value to revenue recognition. Economic value looks to the underlying market value of a company s assets, liabilities and earnings independent of GAAP. aaaaaaa 44

Model Validation Any scoring model should be well documented and well tested. All assumptions should be clearly defined. Data sources should be tested and verified. A scoring model should be thoroughly tested for accuracy and completeness, and this process of validation should be re-run and updated on a regular basis. How can you tell if your model s scoring results are accurate and meaningful? What baseline do you compare your model s results against? How do you best quantify the accuracy (or lack thereof) of your model s results? aaaaaaa 45

Model Validation - Back Testing Sample Size = 97 Standard Error = 1.02 Adjusted Std Error = 0.65 aaaaaaa 46

Model Validation - Back Testing Sample Size = 15 Standard Error = 0.68 Adjusted Std Error = 0.30 aaaaaaa 47

Model Validation - Back Testing Sample Size = 128 Standard Error = 0.98 Adjusted Std Error = 0.63 aaaaaaa 48

Model Validation - Back Testing Sample Size = 19 Standard Error = 2.50 Adjusted Std Error = 1.54 aaaaaaa 49

Summary Credit scoring models can provide useful and straightforward methods for ranking wholesale counterparties; however, your model must be: - Well thought out and documented - Based on prior financial research - Able to score different company types - Thoroughly tested and documented - Not over-relied upon - Not used as a substitute for thorough due diligence aaaaaaa 50

Bibliography: E. Altman and H. Rijken: (2006) A Point in Time Perspective on Through-the-Cycle Ratings,, Financial Analysts Journal, Volume 62 No.1, January/February L. Revsine, D. Collins and W. Johnson: (1998): Financial Reporting and Analysis, Prentice Hall A. Sondhi and D. Fried: (1994) The Analysis and Use of Financial Statements, G. White, John Wiley & Sons, Inc. Moody s Investor Service: (July 2005) Moody s Approach to Global Standard Adjustments in the Analysis of Financial Statements for Non-Financial Corporations Part 1, Maddala, G.S.(1983): Limited dependent and qualitative variables in econometrics, Cambridge University Press, Cambridge Maddala, G.S.(1987): Limited Dependent Variable Models Using Panel Data, The Journal of Human Resources..Vol.3,p.307-337 aaaaaaa 51

Bibliography: Hillegeist, Cram, Keating and Lundstedt (2003): Assessing the Probability of Bankruptcy, Review of Accounting Studies. Wlcox, J. W., (1971): A Gambler s Ruin Prediction of Business Failure Using Accounting Data, Sloan Management Review, Vol. 12. Wilcox, J. W., (1973): A Prediction of Business Failure Using Accounting Data, Journal of Accounting Research. Begley, J., Ming, J., and Watts, S., (1997): Bankruptcy Classification Errors in the 1980s: An Empirical Analysis of Altman s and Ohlson s Models, Review of Accounting Studies. Scott, J., (1981): The Probability of Bankruptcy: A Comparison of Empirical Predictions and Theoretical models, Journal of Banking and Finance. Chung, H. M., and K. Y. Tam (1992): A Comparative Analysis of Inductive-Learning Algorithms, Intelligent Systems in Accounting, Finance and Management. Johnson, C. G., (1970): Ratio Analysis and the Prediction of Firm Failure, Journal of Finance. Kelejian, H., and Oates, W. (1974): Introduction to Econometrics, Principles and Applications, Harper and Row aaaaaaa 52

Credit Scoring Methods A Comparative Analysis North American Power Credit Organization Scottsdale: January 2013, Inc. 813 East Ballard Colbert, WA, 99005 grossmann@rmgfinancial.com Phone: (509) 468-2956 Fax: (509) 468-3217 Cell: (509) 990-0894 www.rmgfinancial.com aaaaaaa 53