Predictive Analytics for Life Insurance: How Data and Advanced Analytics are Changing the Business of Life Insurance Seminar May 23, 2012 Session 2 How to Build a Risk Based Analytical Model for Life Insurance Presenter James C. Guszcza, FCAS, MAAA
Predictive Modeling From A to B Society of Actuaries Los Angeles James Guszcza, PhD, FCAS, MAAA Deloitte Consulting LLP May 23, 2011 Analyzing Analytics Three thoughts capture a major reason why predictive analytics is ubiquitous... The whole of science is a refinement of everyday thinking. Albert Einstein Decision-making is at the heart of administration. Herbert Simon Human judges are not merely worse than optimal regression equations; they are worse than almost any regression equation. Richard Nisbett & Lee Ross Human Inference: Strategies and Shortcomings of Social Judgment 2 Deloitte Analytics Institute 2011 Deloitte LLP 1
Agenda Introduction Why Business Analytics is Now Ubiquitous Predictive Modeling: Levels of Discussion Strategy Methodology Technique Why Business Analytics is Increasingly Ubiquitous 2
Is Business Analytics New? Not really Easy example: credit scoring is an early example of business analytics and has been around for many decades. 5 Deloitte Analytics Institute 2010 Deloitte LLP Is Business Analytics New? Predictive Modeling in Insurance isn t new but it s rate of adoption continues to increase. Property-Casualty insurers have been intensively using predictive models for ~ 15 years now. 1990s: Personal credit used to price, underwrite personal insurance. 2000s: scoring models used to price, underwriter commercial and professional liability insurance. Today: Life insurers are adopting or preparing to adopt predictive modeling techniques. 6 Deloitte Analytics Institute 2010 Deloitte LLP 3
Why Now? (Answer #1) Technology (Moore s Law) Cost of storage and computing power has decreased exponentially Data Third-party data is becoming increasingly available Companies are learning to do more with their internal data Software and algorithms Great analytic ideas keep coming from statistics, economics, machine learning, marketing, Free tools like R 7 Deloitte Analytics Institute 2011 Deloitte LLP Exponential Growth of Data Availability More and more data is gathered, stored, and available to be analyzed Time Challenges Challenging Times for Future Innovation Search http://www.dlib.org/dlib/may09/mestl/05mestl.html 8 Deloitte Analytics Institute 2011 Deloitte LLP 4
Exponential growth of Statistical Computing Availability The open-source R statistical computing environment has experienced exponential growth The number of contributed packages has grown exponentially 9 Deloitte Analytics Institute 2011 Deloitte LLP Why Now? (Answer #2) OR: Clinical vs Actuarial Judgment the Motion Picture 10 Deloitte Analytics Institute 2011 Deloitte LLP 5
There s Something About Linda To illustrate, think about this person: Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations 11 Deloitte Analytics Institute 2011 Deloitte LLP There s Something About Linda Think about this person: Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations Now rank these possible scenarios in order of probability: Linda is active in the feminist movement Linda is a bank teller Linda is a bank teller and is active in the feminist movement 12 Deloitte Analytics Institute 2011 Deloitte LLP 6
There s Something About Linda Daniel Kahneman and Amos Tversky posed precisely this question to a group of people They found that 87% of the people thought that feminist bank teller was more probable than bank teller But this is logically impossible. Mathematical fact: So in particular: Prob(A&B) < Prob(B) Prob(feminist & bank teller) < Prob(bank teller) 13 Deloitte Analytics Institute 2011 Deloitte LLP Analytics Everywhere Neural net models are used to predict movie box-office returns based on features of their scripts Decision tree models are used to help ER doctors better triage patients complaining of chest pain. Predictive models are used to predict the price of different wine vintages based on variables about the growing season. Predictive models to help commercial insurance underwriters better select and price risks. Predict which non-custodial parents are at highest risk of falling into arrears on their child support. Predicting which job candidates will successfully make it through the interviewing / recruiting process and which candidates will subsequently retain and perform well on the job. Predicting which doctors are at highest risk of being sued for malpractice. Predicting the ultimate severity of injury claims. (Deloitte applications in green) 14 Deloitte Analytics Institute 2011 Deloitte LLP 7
A Practical Conclusion There is no controversy in social science which shows such a large body of quantitatively diverse studies coming out so uniformly in the same direction as this one. When you are pushing over 100 investigations, predicting everything from the outcome of football games to the diagnosis of liver disease, and when you can hardly come up with half a dozen studies showing even a weak tendency in favor of the clinician, it is time to draw a practical conclusion. -- Paul Meehl, Causes and Effects of my Disturbing Little Book 15 Deloitte Analytics Institute 2011 Deloitte LLP Predictive Analytics: Fundamental Concepts 8
What Business Analytics Isn t One view of predictive modeling: We assemble a database, explore it, and fit models to it. DATA MODEL Is this a full conception of analytics? 17 Deloitte Analytics Institute 2011 Deloitte LLP What Business Analytics Is No: analytics projects do not begin with data they begin with a strategic issue or problem STRATEGY DATA MODEL 18 Deloitte Analytics Institute 2011 Deloitte LLP 9
What Business Analytics Is No: analytics projects do not begin with data they begin with a strategic issue or problem STRATEGY DATA MODEL DECISIONS And they do not end with a model they end with a model being used to make better decisions 19 Deloitte Analytics Institute 2011 Deloitte LLP What Business Analytics Is No: analytics projects do not begin with data they begin with a strategic issue or problem STRATEGY DATA MODEL DECISIONS And they do not end with a model they end with a model being used to make better decisions Therefore, the best analytics projects projects begin with business strategy and end with IT implementation, training, creating analytics-aided decision rules, and change management. 20 Deloitte Analytics Institute 2011 Deloitte LLP 10
Predictive Analytics: Three Levels of Discussion Strategy Profitable growth More consistent, accurate and economical underwriting Retaining, i cross-selling to preferred policyholders ld Marketing / prospecting Promoting wellness Methodology Model design Modeling process ( actuarial science ) ( data science ) Technique Supervised and unsupervised GLM vs. decision trees vs. support vector machines 21 Deloitte Analytics Institute 2011 Deloitte LLP Methodology and Technique How does data science need actuarial science? Target variable specification, preparation Model design Domain knowledge Variable creation, variable selection Model evaluation Knowing which ambiguities to focus on and which to ignore How does actuarial science need data science? Advances in computing, modeling techniques Programming with data Exploratory data analysis, data visualization ideas Ideas from other fields can be applied to insurance problems 22 Deloitte Analytics Institute 2011 Deloitte LLP 11
Semantics: Data Mining vs Predictive Modeling Data Mining : exploration and discovery Find interesting patterns in databases Open-ended, cast the net wide Data visualization often an important t part Can involve brute-force machine learning algorithms Example: discovery that personal credit correlates with risky insurance behavior. Predictive Modeling : apply statistical techniques in the light of the EDA, knowledge discovery phase Quantify & synthesize relationships found during knowledge discovery Example: use credit information in a traditional predictive model 23 Deloitte Analytics Institute 2011 Deloitte LLP At the Center of It All: Data Science Or: The Collision between Statistics and Computation Today, it is arguably appropriate to call certain aspects of propertycasualty ypractice data science. This often involves more than just knowing how to run regression models. Data science goes beyond: Traditional statistics Business intelligence [BI] Information technology Image borrowed from Drew Conway s blog http://www.dataists.com/2010/09/the-data-science-venn-diagram 24 Deloitte Analytics Institute 2011 Deloitte LLP 12
Predictive Analytics: Understanding the Strategy A Great Book About Predictive Analytics 26 Deloitte Analytics Institute 2011 Deloitte LLP 13
The Strategic Use of Analytics in Life Insurance Like baseball, insurance is a numbers game. Both fields are awash in data and a lot of money is at stake! As with baseball, so in insurance: All of this data until fairly recently has not been used in strategic ways. Percent Declined 80% 70% 60% 50% 40% 30% Underwriting Model Lift Curve (Hypothetical) Life insurance example: Models can be built to assign probabilities of (e.g.) being declined or (e.g.) being assigned to the best underwriting class based on readily available data. 20% 10% 0% 1 2 3 4 5 6 7 8 9 10 Model Decile (Predicted Value) 27 Deloitte Analytics Institute 2011 Deloitte LLP The Strategic Use of Analytics in Life Insurance Strategic application: data-driven underwriting 80% Underwriting Model Lift Curve (Hypothetical) 70% Models estimate the probabilities that a risk will fall 60% into the best / worst 50% underwriting classes. In this hypothetical example Worst 10% of risks have a >70% decline rate Percent Declined 40% 30% 20% 10% 0% 1 2 3 4 5 6 7 8 9 10 Model Decile (Predicted Value) 28 Deloitte Analytics Institute 2011 Deloitte LLP 14
The Strategic Use of Analytics in Life Insurance Strategic application: data-driven underwriting 80% Underwriting Model Lift Curve (Hypothetical) 70% Models estimate the probabilities that a risk will fall 60% into the best / worst 50% underwriting classes. In this hypothetical example Worst 10% of risks have a >70% decline rate Best 10% of risks have <5% decline rate Percent Declined 40% 30% 20% 10% 0% 1 2 3 4 5 6 7 8 9 10 Model Decile (Predicted Value) 29 Deloitte Analytics Institute 2011 Deloitte LLP Predictive Analytics: Understanding the Methodology 15
Three Central Concepts of Predictive Analytics Scoring engines aka algorithmic solutions aka predictive models Lift / Gain analysis Lift: how much better/worse than average are the observations with the best/worst scores? Gain: what percent of the outcomes are captures in the 10% of observations with the highest scores Out-of-sample tests Estimate how the model is likely to perform once implemented Unbiased estimate of model s predictive power 31 Deloitte Analytics Institute 2011 Deloitte LLP Scoring Engines Scoring engine: a formula that classifies or separates risks (or accounts or households ) into: Expected longevity Churn likelihoodlih Profitable vs unprofitable Utilization... (non)-linear equation f() of several predictive variables Produces continuous range of scores: Y = f(x 1, X 2,,X N ) Common model form: Y = g -1 ( + 1 X 1, + 2 X 2,+ + N X N ) 32 Deloitte Analytics Institute 2011 Deloitte LLP 16
What Powers a Scoring Engine? Scoring engine: Y = f(x 1, X 2,,X N ) The {X 1, X 2,,X N } are at least as 1 2 N important as the f(). Domain expertise, creativity, insight, all important A major part of the modeling process consists of variable creation and selection It is possible to generate many 100s of variables Knowing what is important t to create, how to create them, and what to do with them once they are created is the steepest part of the learning curve Why predictive modeling is an art as well as a science Tacit knowledge vs textbook knowledge 33 Deloitte Analytics Institute 2011 Deloitte LLP Model Evaluation: Lift Curve Analysis Sort the data by the model score Underwriting Model Lift Curve (Hypothetical) Break into 10 equal pieces Best decile : 10% of risks with the best scores Worst decile : 10% of risks with the worst scores Lift: average measured difference in the quantity we are trying to predict or estimate Lift = segmentation power Lift translates into the ROI of the modeling project Unlike such concepts as R 2 or AIC, lift has operational meaning Percent Declined 80% 70% 60% 50% 40% 30% 20% 10% 0% 1 2 3 4 5 6 7 8 9 10 Model Decile (Predicted Value) 34 Deloitte Analytics Institute 2011 Deloitte LLP 17
Data Mining and The Curse of Dimensionality Data Mining has both positive and negative connotations. Done well, it can lead us to new insights that result in more powerful models Done naively, it can cause our models to be fooled by randomness The curse of dimensionality the more variables we consider the harder it becomes to search through the space of possible models. As the dimension increases, so does the side-length of the cube needed to cover r% of the data space. In 10 dimensions we need to cover 80% of the range of each coordinate to capture 10% of the data. (From The Elements of Statistical Learning by Hastie, Tibshirani, Friedman) 35 Deloitte Analytics Institute 2011 Deloitte LLP The Bias-Variance Tradeoff and the Problem of Overfitting Predictive modeling techniques cannot distinguish true patterns from random variation in the data. Error measured on the dataset used to fit the model is overly optimistic The Bias-Variance Tradeoff 8 9 10 11 0.15 0.20 high bias low variance <----- low bias high variance -----> 4 5 6 7 4 6 8 10 12 14 Which model would you choose? error prediction e 0.05 0.10 train data test data 40 60 80 100 120 140 160 36 Deloitte Analytics Institute complexity (# neural net cycles) 2011 Deloitte LLP 18
The Bias-Variance Tradeoff and the Problem of Overfitting Low bias : The model fit is good on the training data. The model s predicted value is close to the data s expected value High variance : The model might not make accurate predictions upon implementation Bias alone is not the name of the game. 5 0.20 The Bias-Variance Tradeoff high bias low variance <----- low bias high variance -----> game. 40 60 80 100 120 140 160 Out of sample validation is a way of guarding against overfitting. prediction error 0.05 0.10 0.15 train data test data complexity (# neural net cycles) 37 Deloitte Analytics Institute 2011 Deloitte LLP Out-of-Sample Model Testing / Validation A Classic methodology: Divide the data into 3 pieces Randomly and/or using time Test data: use to measure gain/lift of candidate model Train Test Val Training data: used to fit models E.g. estimate parameters of a Generalized Linear Model or Cox Hazard Model Test data: used to measure gain/lift of candidate models Perform train/test steps iteratively until you have a model you are happy with During this iterative phase, validation is held in cold storage Validation Data: used measure lift on final selected model The test data was implicitly used during model selection process, therefore it is preferable to measure lift on a truly blind test sample Validation data is often an out-of-time sample 38 Deloitte Analytics Institute 2011 Deloitte LLP 19
So far Therefore we want to: Build a scoring model that results in a continuous range of scores that Produces valuable segmentation power on a blind-test data sample How do we get there? 39 Deloitte Analytics Institute 2011 Deloitte LLP Predictive Analytics: Understanding the Process 20
The High-Level Process Data scrubbing and Variable Creation Data Exploration and Modeling Model Analysis and Implementation The modeling phases we will discuss fit into the Cross-Industry Standard Process for Data Mining [CRISP-DM] 41 Deloitte Analytics Institute 2011 Deloitte LLP Data Scrubbing and Variable Creation Research possible data sources Extract/purchase data Check data for quality (QA) Messy! (still deep in the mines) Create Predictive and Target Variables Opportunity to quantify tribal wisdom and come up with new ideas Can be a very big task! Steepest part of the learning curve 42 Deloitte Analytics Institute 2011 Deloitte LLP 21
Types of Predictive Variables Insurance data sources can be viewed in an outward inward hierarchy. Geodemographic Population Density, traffic, weather, crime, education, Policy Characteristics Term, whole-life, Policyholder, Household Characteristics Age, gender, martial status, #kids, income, education Prior History / Behavior Medical history, life events, credit history, education/career choices, 43 Deloitte Analytics Institute 2011 Deloitte LLP Knowing Me, Knowing You Consumer data is rapidly increasing in: Volume Scope Specificity it REALITY The nature of available data is evolving rapidly. A 3-stage evolutionary process 44 Deloitte Analytics Institute 2011 Deloitte LLP 22
Data Exploration and Variable Triage The variable creation phase was hypothesis-driven expansion. This phase is contraction : begin the process of analyzing and winnowing the space of candidate variables. Data visualization / Exploratory Data Analysis is a key part of this. Use EDA to: Discard weak or poorly distributed variables Consider how to handle missing variables Transform variables categoricity, linearity, Investigate correlations amongst the candidate predictors Explore dimension reduction techniques (remember curse of dimensionality) Take note of variables relative predictive power 45 Deloitte Analytics Institute 2011 Deloitte LLP Model Design Model design is where actuarial science meets data science. Translate business problem into algorithmic solution. Common considerations: Which target variable to use? Which data points / types of policies to include/exclude? Which data sources / variable types to consider? What is the level of the model (policy? HH for cross-sell?) How should model be evaluated? Lift curves / gains charts Broken down by certain segments of business Compare with existing practice How to split data into train / test / validation samples Is data sufficiently credible? 46 Deloitte Analytics Institute 2011 Deloitte LLP 23
Multivariate Modeling Key initial step: appropriate model design Translate ultimate use of the model into appropriate model design With the model design in place, the iterative ti modeling process begins Picks up where EDA process leaves off: process of triaging, transforming, interacting candidate predictors continues. Multiple candidate models are built/tested Sample techniques: Regression / GLM / proportional hazard models CART / MARS / Random Forests, Neural nets, Support Vector Machines 47 Deloitte Analytics Institute 2011 Deloitte LLP Multivariate Modeling: In More Detail 1) Pair down full collection of candidate predictive variables to a more manageable set 2) Iterative modeling process Build candidate models on training data Evaluate on test data Many things to tweak Alternate modeling techniques Hybrid models, ensemble models Different target t variables Different predictive variables Data filtering / sampling criteria Technique-specific modeling options: distributional assumptions, weights and offsets, cost-complexity tuning parameters, neural net topology, decision tree splitting method, multiple random samples, 48 Deloitte Analytics Institute 2011 Deloitte LLP 24
Considerations Do signs/magnitudes of parameters make sense? Statistically significant? Sensible from a business point of view? Readily explainable Consistent with corporate philosophy? Is the population reasonably homogenous? If not does the model behave consistently / reasonably on both groups? Does the predictive power hold up within sub-segments Continuity Are there small changes in input values that might produce large swings in scores Make sure that end-users can t game the system 49 Deloitte Analytics Institute 2011 Deloitte LLP Model Analysis and Implementation Perform model analytics Necessary for end-users to gain comfort with the model Calibrate Models Create user-friendly scale client dictates Implement models Technology implementation deep IT skills crucial Business implementation plan use of business rules, reason messages, education Monitor performance Distribution of scores over time, predictiveness, usage of model... Collect reactions, suggestions from end-users Collect list of desired improvements for v2.0 Plan model maintenance schedule 50 Deloitte Analytics Institute 2011 Deloitte LLP 25
Predictive Analytics: Understanding the Techniques Supervised vs Unsupervised Learning Supervised Learning Estimate expected value of Y given values of X. Generalized Linear Models (GLM) Generalized Additive Models (GAM) Survival Models (Cox Proportional Hazard) Classification and Regression Trees (CART) Multivariate Adaptive Regression Splines (MARS) Random Forests Support Vector Machines Neural Nets Unsupervised Learning Attempt to find interesting patterns amongst the X. No outcome ( target ) variable Clustering, Self-organizing maps Correlation / Principal Components / Factor Analysis 52 Deloitte Analytics Institute 2011 Deloitte LLP 26
Classification vs Regression Classification Problems Goal is to segment the observations of interest into 2 or more distinct categories. Average, above average, below average profitability customers Likely fraudsters vs likely non-fraudsters Likely defectors vs likely non-defectors Regression Problems Goal is to predict a continuous amount. Dollars of loss for a policy Ultimate size of claim Years of retention Days out of work 53 Deloitte Analytics Institute 2011 Deloitte LLP Parametric vs Non-Parametric Parametric Statistics Involves the specification of a probabilistic model governing the process that generated the data. e.g. Poisson Regression: assume that the process generating g the number of claims for a given policy is a Poisson process. The mean ( λ ) of each policy s Poisson distribution is a linear combination of various attributes. Non-Parametric Statistics No probability model specified Purely algebraic e.g. classification trees use brute-force computing to create relatively pure segments of a dataset. No assumptions made about the target variable. 54 Deloitte Analytics Institute 2011 Deloitte LLP 27
Regression and its Relations Generalized Linear Models [GLM] Relax assumptions of Ordinary Least Squares [OLS] regression Normality exponential family Linearity linearity on scale link function scale (e.g. log scale) Logistic regression: binary outcomes Poisson regression: count outcomes Multivariate Adaptive Regression Splines [MARS] and Neural Nets Clever automatic ways of transforming and interacting candidate input variables Automates difficult job of selecting and transforming dozens or hundred of variables Account for fact that true relationships aren t necessarily linear Universal approximators: can model any functional form Downside: are black-box models (though MARS is much more interpretable) Classification and Regression Trees [CART] Tree-structured regression Simplification of MARS; easier to interpret 55 Deloitte Analytics Institute 2011 Deloitte LLP This presentation contains general information only and is based on the experiences and research of Deloitte practitioners. Deloitte is not, by means of this presentation, rendering business, financial, investment, or other professional advice or services. This presentation is not a substitute for such professional advice or services, nor should it be used as a basis for any decision or action that may affect your business. Before making any decision or taking any action that may affect your business, you should consult a qualified professional advisor. Deloitte, its affiliates, and related entities shall not be responsible for any loss sustained by any person who relies on this presentation. Copyright 2011 Deloitte Development LLC. All rights reserved. Member of Deloitte Touche Tohmatsu Limited 28