Using the Power of Predictive Analytics for Case Outcomes ERICSA 52 nd Annual Training Conference & Exposition April 26 30 Hershey Lodge Hershey, Pennsylvania
Participants Presenter Edward V. Lehman, Jr. Director, Case Processing & Data Management Philadelphia Family Court Domestic Relations Division Presenter Steven J. Golightly, Ph.D. Los Angeles County Child Support Moderator Joyce Match Business Analyst Manager Pennsylvania Bureau of Child Support Enforcement
Predictive Analytics in Action Edward V. Lehmann, Jr. ERICSA 52 nd Annual Training Conference & Exposition April 26 30 Hershey Lodge Hershey, Pennsylvania
What is the requirement? Access to the right information is extremely important Work the Right Cases Reduce Information Overload Increase Visible Results Determine Next Appropriate Action Proactively Effective Case Worker Management
Understanding Past, Measuring Present, Predicting the Future
Predictive Analytics in Child Support Improved Workforce Effectiveness Reduce user guesswork and cherry picking Prioritize workload based on high impact Incorporate historical experience to drive future activities Eliminate a one size fits all approach Efficiency of customer service interactions Over time reduce future workload Increased Resource Allocation Efficiency Shift from reactive enforcement to early intervention Learning component for new case management approaches Assign high risk cases to case workers sooner Different approaches for different regions and different case types Automate certain research processes Improved Outcomes Better estimates of delinquencies, child support collections Fewer child support cases in arrears Model results can be used to quantify historical process inefficiencies Significant marketing value We Know Our Citizens More reliable payments for children
Predictive Modeling for Child Support o Child support enforcement has traditionally been a reactive process. o What if we had a tool that could help us predict which NCPs are most likely to become in-arrears in the near future? o We could use such a tool to: o Prevent arrears o Decrease custodial parent complaints o Take the right action on a case at the right time o Assign the right case workers to the right cases o Gather information for potential policy changes Use analytics to make smarter decisions and do more with less.
How do Models Work? Statistical models just refine what we do naturally all the time. For example: which of these NCP s is most/least likely to pay next month? Gene: 35 years old $2000 in arrears No other children on case No Salary attachment History of family violence Case is a IV-A assistance Tom: 28 years old $100 in arrears 1 other child on case Salary attachment No history of violence Case is not an assistance case Rick: 39 years old $8000 in arrears No other children on case No salary attachment No history of violence NCP has other child support cases
How do Models Work?
Pennsylvania s Solution Pennsylvania s Predictive Analytics solution included updates to the existing Performance Improvement Module (PIM) solution and a new application to calculate a predictive analytics score. Allows a worker to calculate a score for a case based on 20 variables Score is the likelihood of the defendant to pay 80% towards current support obligation in the next three months Information required to generate the score is easily accessed from the system at the time the support order is created or modified Provides a list of upcoming child support establishment conferences with the cases assigned to the worker to allow the score creation prior to the conference.
PIM/Predictive Analytics o o o o o Targeted case lists Suggested case actions Predictive score and reasons Performance metrics Projects for targeted, custom outreach including text messages
Payment Score Calculator Objectives Establish a good payment pattern: Increased quantity and frequency of collections More effective meetings with defendants Make available new methods of reaching out Identifying opportunities for proactive enforcement activities Maximize performance metrics Minimize costs related to enforcement Improve cost/effectiveness ratios Effective case assignment based on scores Establishing consistent payment patterns based on successful business process actions for various scores Provide the right services at the right time to encourage compliance with the order Let the defendant know the case is being monitored
Variable Selection Final model contains 20 variables 36 variables considered for modeling after Exploratory Data Analysis [EDA] Phase 64 candidate variables created during data scrubbing / brainstorming phase Over 400 data elements collected and considered in variable creation phase The case for which a score is being calculated is compared against the historical behavior of thousands of cases with similar characteristics
Example Predictive Variables Number Predictive Variables 1 Collection Indicator 2 High Number of Enforcement Activities Indicator 3 Number of Cases 4 Number of Enforcement Activities 5 Balance of Arrears 6 Defendant Net Income 7 Active Income Attachment Indicator 8 Number of Defendant Member Addresses on MADD 9 Number of Defendant Employers 10 Distance between the Defendant and Plaintiff These are just a few examples of predictive variables used in Pennsylvania. Variables used in each State are different.
Predictive Modeling Scores Probability of NCP paying 80 percent on current support in the three months after support order issued/modified. 0 30% 31 50% 51 79% 80% +
Confidence in the Score Payment Score Average Percent of Current Support 1 51% 2 66% 3 77% 4 87% To validate the accuracy of the payment score, a study of 5,000 PACSES cases was completed For each payment score, the average FYTD percent of current support collected was calculated. A direct relationship between payment score and percent of current paid was found (i.e., the higher the payment score the higher the percent of current paid.
PSC Enhancements Planned enhancements to the existing functionality of the Payment Score Calculator Development of a mechanism for calculating a Payment Score for all open cases Quarterly refresh the Payment Score Calculations on all open cases Allow worker to complete a guideline calculation if the PSC has been updated within the last three months Modification of PIM to add several pre-defined filters which use the Payment Score as one of the factors for consideration Implementation planned for June, 2015
PIM/Predictive Analytics Effect in Pennsylvania After PIM was implemented, Pennsylvania became the first State to achieve above 80% in both percent of current and percent of arrears PA has been ranked #1 in the United States for both percent of current and percent of arrears, both of which have increased significantly since the implementation of the Performance Improvement Module
Predictive Analytics Impact in Child Support Revenue Impact: Exceeding 80% of collections targets Accuracy: Likely payers 4x more likely to be above 50% paid than very unlikely payers Timeliness: Allows taking action at time of support order, before payments are missed Actionability: Caseworker gets clear guidance on recommended steps Policy Insights: Ability to better segment NCPs to better understand policy impacts Workforce effectiveness: Shift from reactive enforcement to pro-active education Ability to prioritize workload based on high impact Efficiency: Increased effectiveness of retention outreach campaign, audits Time to production: Scored 17,000 cases within first three months Transparency: Unified view across all regions and business processes Marketing value: Helps build positive relationships with our clients. Everyone wins when payments are made
Questions? Need More Information? Feel free to contact us. Edward V. Lehmann, Jr. Director, Case Processing and Data Management Philadelphia Family Court Domestic Relations Section EdwardLehmann@PACSES.com
ERICSA 52 nd Annual Training Conference & Exposition April 26 30 Hershey Lodge Hershey, Pennsylvania
WHERE DO WE START? Starting with the Why Simon Sinek s Golden Circle 2
WHY 3
TAILORING SERVICES USING PREDICTIVE ANALYTICS Apply a Predictive Analytics lens to Case Management 4
PREDICTIVE ANALYTICS CLASSIFICATION MODEL Forecast (predict) an outcome using multiple predictors TARGET: Predict which cases we can expect a payment of at least 70% of Current Support Amount in subsequent month Built a Classification Model started with 100+ programmatic, demographic, and economic variables narrowed down to 20 statistically significant variables Model accurately predicts target variable for 90% of cases. 5
CASE SCORING Payment Status (Prior Month) Prediction (Next Month) Confidence Score Score (Propensity Score) 1 Non-Payer Non-Payer.91 1 2 Non-Payer Non-Payer.73 3 3 Payer Payer.91 9 4 Payer Payer.73 7 5 Payer Non-Payer.55 5 6 Non-Payer Payer.55 5 50% 40% 30% 20% 10% 0% 0 1 2 3 4 5 6 7 8 9 6
PREDICTIVE ANALYTICS - SEGMENTATION Group cases using one or more input fields TARGET: Create cluster of cases based on case variables to identify natural grouping of cases Built a Segmentation Model Started with same variables as in classification model Clustering Analysis identified natural groupings based on payment history 7
EARLY SUCCESSES Arrears Only Project - January 2014 9,486 cases with an Arrears balance, no minor children, no current support, and no payment within last federal fiscal year Strategy Focused Caseload, Locate Experts Results $2 million in 12 months! FPM 4 55.99% 47.96% Department Arrears Only 8
PREDICTIVE ANALYTICS FINDINGS Predictive Analytic Model identifies cases that perform similarly PA Model predicts, with a high level of confidence, how a case will perform PA Model identifies characteristics and motivations of different customer types One-Size Fits One Arrears Only Project Success Personalized the client s experience Case Worker a subject matter expert for case type 9
HOW CREATE NEW BUSINESS MODEL Case Ownership Case Segmentation Other Arrears Only EST Estab. Paying IGR Zero Order Occasion al Zero Order Non- Paying Arrears Only Paying Non- Paying Occasional 10
SEGMENTATION MODEL Establishment Arrears Only IGR Intake Customer Response ENFORCEMENT Div. 1 Div. 3 Div. 4 Div. 5 Div. 6 11
ENFORCEMENT CASELOAD ZERO ORDER QUADRANT Definition Occasional Zero Order Paying Non-Paying Case has an order for $0 OR Case has a reserved or pending supplemental order OR Case has a Medical Only order OR Case does not have an active order, but case has an arrears balance and child is a minor 12
ENFORCEMENT CASELOAD PAYING QUADRANT Definition Occasional Zero Order Paying Non-Paying Case has an order AND Current Support Amount is a dollar amount greater than $0 AND NCP pays regularly AND NCP pays over 70% of CS amount 13
ENFORCEMENT CASELOAD OCCASIONAL PAY QUADRANT Occasional Zero Order Paying Non-Paying Definition Case has an order AND Current Support Amount is a dollar amount greater than $0 AND NCP pays irregularly OR NCP pays less than 70% of CS amount 14
ENFORCEMENT CASELOAD NON-PAY QUADRANT Definition Occasional Zero Order Paying Non-Paying Case has an order AND Current Support Amount is a dollar amount greater than $0 AND NCP has not made a payment during the current Federal Fiscal Year 15
STRATEGIES BY QUADRANT Occasional Paying Send letters of Introduction to new case manager UIB/DIB/ Mod review (CMT Sort) Review for SLMS suspension Manual submission for NCP s with valid license Incarcerated Special Mod/closing Project CMT High TMSO paying less than 90% sort Review EFO case function cases for order for subsequent child Work LC005 task for IWOs Review CMT for SSI benefits Send letters of Introduction to new case manager Low Order with reported Earnings Mod Project Admin IWO Project Incarcerated Special Mod Project CMT High TMSO paying less than 90% sort UIB/DIB Mod review (CMT Sort) Review EFO case function cases for order for subsequent child New Order clerical contact of NCP (EI) Work LC005 task for IWOs Zero Order Non-Paying Send letters of Introduction to new case manager Zero Mod Project Review EFO case function cases for order for subsequent child Review Paid in Full cases to confirm emancipated DP s updated for auto closures. Send letters of Introduction to new case manager Review for SLMS suspension Manual submission for NCP s with valid license Incarcerated Special Mod Project Review EFO case function cases for order for subsequent child Work LC005 task for IWOs Review CMT for SSI benefits 16
NEW CSTATS Old CSTATS: Performance across the division Compares branch-to-branch New CSTATS: Performance across the division And performance by quadrants Compares branches by quad Month, YTD 17
IMPLEMENTATION Phased Roll-Out Roadshows Feedback/Workgroups New Performance Measures Reset every FFY 18
QUESTIONS 19