CART/MARS Risk Assessment in Automobile Loans and Leases. John Trimble Wells Fargo

Transcription

1 CART/MARS Risk Assessment in Automobile Loans and Leases John Trimble Wells Fargo

2 Appeal of Data Mining Quality More accurate prediction Better analysis from same skill set Quantity Improved efficiency Same modeling job takes less time Increasing total accomplishment in given period

3 Topic of This Presentation Two Case Studies Where Efficiency Really Mattered Quick replacement of legacy scorecard (model) Eliminated yearly licensing fee Where Efficiency Frees Time for Higher Quality Pursuit in Accounting for loan prepayments Leading to better credit risk assessment

4 About Wells Fargo Auto Finance

5 Auto Loan Originations Automated Approval Car Dealer Legacy Model Wells Fargo Lender #2 Lender #3

6 Wells Fargo Portfolio Approximately $13 billion in loans and leases (retail and wholesale) 840,000 Customers Approve new loans with credit scoring models to large degree Model assesses risk of approx. 1.8 million individuals per year (800,000 applications)

7 First Case Study Using MARS to Replace a Legacy Model

8 Replacing A Legacy Model Eliminate Annual Licensing Fee Legacy Model Performing well Historical inputs and performance available New Model Only legacy input variables available New data platform under construction

9 Legacy Model Inputs and Model Inputs Performance Credit Bureau Variables in Month t Application Variables in Month t Model Performance Measure Delinquency 60 dpd or worse In 18 subsequent months (i.e. t+1,,t+18)

10 Limited # Input Variables LEGACY MODEL' S VARI ABLES No. S o u r c e La b e l 1 Application Vehicle Age 2 C. Bu r e a u CBR Te n u r e 3 C. Bureau Del i nquenci es 4 Application % Down Payment 5 C. Bureau Credit Utilization 6 Application Gross Mont hl y I ncome 7 C. Bur eau # I nqui r i es 8 C. Bur eau Maj or Der og Rat i ngs 9 C. Bur eau Revol vi ng Bur den 10 C. Bureau Recent Del i nquenci es 11 C. Bureau # Account s

11 Variables Selected by MARS MARS TRANS FORMATI ONS OF LEGACY MODEL' S VARI ABLES # Del i nquenci es NO Delinquencies Long CBR Tenur e Shor t CBR Tenur e Hi gh Revol vi ng Bur den Low Revolving Burden Recent Del i nquency NO Recent Del i nquency Hi gh # Recent Inqui ri es Low # Recent Inquiries Do wn Pa y me n t ( %) NO Do wn Pa y me n t Vehi cl e Age

12 MARS Generated An Excellent Replacement Legacy and New Model Lorenz Curves Legacy New Random % of Bads % of Obs

13 Benefits and Costs of the Replacement Benefit: Eliminated substantial annual licensing fee Costs: Data preparation (1 week) Model development (1 day) Model implementation (2 weeks)

14 2nd Case Study Using CART and MARS to Understand Who Prepays Auto Loans

15 When Loans Prepay Expected Return < Realized Return Approx. 2-3% of loans voluntarily pay off each month, 33% prepay over 18 mos Prepays Impose additional administrative burden Expose the business to reinvestment risk If Can Predict Prepayment Tendencies Can price in prepayment risks Develop other mitigation strategies

16 Prepayment Behavior Underlying Causes Likely Complex Natural Causes Auto accidents Theft Voluntary Causes Advantageous refinancing opportunities Vehicle replacement Interest rate levels Manufacturer incentives Retention Sometimes is lack of attractive alternatives

17 Predicting Prepayments Available Attributes Nearly 70 Credit Bureau and Application Attributes Available Examples include: Credit Inquiries Loan Maturity Recent Delinquencies Credit Utilization # Accounts Debt Burden Delinquency History

18 Search for Salient Attributes Search for Common Attributes of Prepayers Regardless of when prepay CART Trees Developed for Each Month of Performance Window Variable Importance Scores (VIS) Averaged for each attribute Attributes Ranked by Average VIS

19 VIS Attributes of Prepayers Top 10 by Average VIS Loan Maturity Attribute Utilization of Credit Union Accounts Number of Credit Union Accounts Extent of Credit Seeking Activity Credit Utilization Number of Accounts Extent of Credit Union Utilization Volume of Recent Credit Seeking Activity Average Number of Months Trades Open

20 Prepays CART Tree 47 Nodes

21 Prepays CART Tree 1 Obs Per Applicant Gini Splitting Rule Standard CART settings Top 7 Node Splits (47 Nodes Total) Node 1 Loan Maturity (<=37 Mos) (Left Node Terminal) Node 2 # Credit Union Accounts in Good Standing (<= 1) Node 3 Credit Utilization. (<= 21.6%) Node 4 Extent of Credit Seeking Activity (<= 44%) Node 5 # New Accounts. (<= 3) Node 6 Extent of Credit Seeking Activity (<= 51%) Node 7 Credit Utilization. (<= 28.3%)

22 Salient Revelations from CART Loan maturity seems to be a key attribute Shorter term loans prepay at greater rate But term also enters in more complex ways Access to credit union financing is important, but relationship not simple Active credit union customers prepay at 1.6 times rate as non-credit-union customers Credit utilization and credit worthiness are also important

23 Logistic Regression Lorenz CART Nodes Learn and Test Sample Lorenz Curves % of Attrites Learn Test Random % of Population

24 CART Logistic Model Lift is relatively modest Prepayment motivated by many underlying factors Natural causes not measured Model still offers useful predictions Could use in present form to adjust pricing To compensate for prepayment risk Application data not in current study potentially may add to predictive accuracy

25 Logistic Regression Lorenz Top 5 CART Nodes+ MARS Vars Learn and Test Sample Lorenz Curves % of Attrites Learn Test Random % of Population

26 CART vs. MARS MARS: CART Dummies Plus Vars Model: CART Dummy variables Let MARS pick other variables MARS model adds modest lift over strictly CART model MARS Logistic Model Provides: Smoothes lift Continuous probability predictions

27 Logistic Regression Lorenz Top 5 CART Nodes Interacted with MARS Vars Learn and Test Sample Lorenz Curves 100 % of Attrites % of Population Learn Test Random

28 MARS vs. MARS CART Interacted with MARS Vars Model: CART dummies interacted with variables picked by MARS Each segment modeled separately then combined into single Logistic model Commonly used approach to model building Believed to improve prediction But adds imperceptible lift over previous MARS model with CART dummies and MARS variables Does add new statistically significant variables

29 What Have We Learned? About Who Prepays

30 Prepayer Profiles Greatest Propensity to Prepay Preference for shorter maturity (< 42 mos) Shorter the maturity more likely to prepay Access to lower-cost financing Credit Union member (1+ accts open) Home equity line (tax-advantaged rate) Low credit utilization (< 58%) Increases with decreases in credit utilization Longevity with smaller # accounts (>2yrs) Positive balances on only small # such accts More accounts opened recently (>62%)

31 Prepayer Profiles Least Propensity to Prepay Preference for longer maturity (> 42 mos) Longer the maturity less likely to prepay Lower # accts opened recently (< 62%) Increases with increases in % opened recently High credit utilization (> 58%) Larger # accts with balances (8+) Balances in good standing May be Credit Union member, but access to lowcost financing stymied by debt burden Never used credit union much in past (0-2 times)

32 What About The Timing of Prepayment?

33 Linear Regression Lorenz MARS Timing-of of-prepayment Predictors Learn and Test Sample Lorenz % of Attrites Each Month % of Population Learn Test Random

34 Timing-of of-prepayment Lorenz Lift appears to be quite small But is continuous target variable Target is Month of Prepayment Prediction in each month good on average But variance quite large Small R 2 (1.2%) Still get useful indicators of timing of prepayment

35 Prepayer Profiles Greatest Propensity to Prepay Early Prefer longer maturity (> 24 mos) Propensity declines with maturity Very low credit utilization (<= 15%) Increases with decreases in credit utilization High % accts recently opened (>=167%) Increases with # such accounts Not many recent accounts used (< 24%) Increases with decreases in % such accounts High # Credit Union accounts (2+) Increases with # of such accounts Good credit rating FICO Score > 780

36 Prepayer Profiles Least Propensity to Prepay Early Prefer shorter maturity (<= 24 mos) Propensity declines with maturity Carries balances on several accts (1+) Balances in good standing Lower % accts opened recently (< 167%) Propensity to prepay decreases with % of accts recently opened