Big Data and Health Insurance Product Selec6on (and a few other applica6on) Jonathan Kolstad UC Berkeley and NBER

Transcription

1 Big Data and Health Insurance Product Selec6on (and a few other applica6on) Jonathan Kolstad UC Berkeley and NBER

2 Introduc6on Applica6ons of behavioral economics in health SeIng where behavioral assump6ons and/or informa6on fric6ons characterize the market (e.g. Arrow, 1963) Key applied micro field where applica6ons depend on modeling actual behavior and (poten6ally) designing intervena6ons Exci6ng direc6on for applied micro more generally Not behavioral vs. neoclassical but a variety of tools to explain what is observed (post behavioral) Big data/machine learning in economics Early but growing area Two way opportunity for research (econ ß à CS/biostat/stat) Key applica6ons: data driven treatment effect heterogeneity and non- linear models of behavior

3 Outline Why machine learning in health and behavioral? Example: Addressing choice errors in insurance using ML and economics Treatment effect heterogeneity and health care demand (Brot- Goldberg, et al., 2015) If there s 6me: Causal inference with tree models, an applica6on to physician decision making

4 WHY BIG DATA METHODS IN (HEALTH, BEHAVIORAL, APPLIED) ECONOMICS (E.G. MACHINE LEARNING)?

5 Sufficient Sta6s6cs with Treatment Effect Heterogeneity Typical focus on LATE/ITT es6mates of a treatment effect Average likely masks important heterogeneity and limits out of sample/counterfactual analysis Alterna6ve, imposing structure requires assump6ons of behavioral model that are difficult to test Challenge in mapping to welfare Integrate with sufficient sta6s6cs for welfare analysis (e.g. Chedy, 2011) impose few restric6ons on behavioral response and allow for heterogeneity in key parameter/suff stat

6 Sufficient Sta6s6cs for Welfare Analysis Source: Chedy (2011)

7 Treatment Effect Heterogeneity with Sufficient Sta6s6cs Machine learning approaches rely on data driven assessment of response to treatment (e.g. Athey and Imbens, 2015) Flexible capture responses without imposing structure and, condi6onal on sufficient support over observables, predict out of sample Rely on cross valida6on to test OOS performance Integrate causal inference into ML models (e.g. Kolstad and Swason, 2015) Flexible es6ma6on on observables can: Capture heterogeneous responses without imposing structure (e.g. some behavioral actors and some raional types) Es6mate out of sample based on high dimension observables (depends on share of varia6on explained à move to usine R^2)

8 Big Data in Healthcare Many unique and large scale data sets Poten6al to explain a great deal of varia6on using observables Advent of new wearables will increase amount of data on health directly to get at big ques6ons (e.g. stress and the rela6onship between health and SES)

9 Non- linear models Important ques6ons of decision making in behavioral econ and health (e.g. doctors choosing treatments, pa6ents taking meds) Decision models are inherently non- linear but adhere closely to tree structure of some ML methods Predic6ve performance of non- linear models in explaining key health care outcomes is drama6cally higher (e.g. spending)

10 Challenges to Applying ML Selec6ng a model from an event large toolbox can be a challenge Interpretability is typically low with new methods that enhance predic6ve performance (not with Lasso though) Fundamental issue in health where clinical mechanisms can be an important constraint (e.g. ACG vs. black box machine learners) Source: Has6e, T, et al., 2009, Elements of Sta6s6cal Learning.

11 COMBINING ECONOMICS AND MACHINE LEARNING TO IMPROVE CONSUMER DECISION MAKING

12 Choice Errors and Health Insurance Large and growing literature on choice errors in health insurance Medicare Part D (e.g. Abaluck and Gruber, 2011, 2013; Kling, et al., 2012; Ho, Hogan and Scod- Morton, 2015; Hiess, Mcfadden and Winter, 2010) Employer- based (e.g. Handel, 2013; Handel and Kolstad, 2015; Bhargava, Loewenstein and Sydnor, 2015) Medicaid (e.g. Marton, et al., 2014) Exchanges (e.g. Ericson and Starc, 2015) Typically viewed as a mean es6mate for a behavioral error (e.g. weight on premiums vs. OOP cost), structural behavioral (e.g. Barseghyan, et al., 2013) Ac6ve and informed choice is key to reliance on private provision of insurance (e.g. Medicare Part D and C and ACA) (e.g. Handel and Kolstad, 2015)

13 Understanding Choice Errors in Insurance: A Simple Model Consider simple model in Handel and Kolstad (2015)

14 Understanding Choice Errors in Insurance: A Simple Model Assume consumers may not understand details of plan op6ons and lack complete forward looking knowledge of health

15 Understanding Choice Errors in Insurance: A Simple Model Assume consumers may not understand details of plan op6ons and lack complete forward looking knowledge of health

16 Wedge Between Value and Choices Important for posi6ve and norma6ve assessment of behavior

17 Puzzle and empirical seing 35-70% of the popula6on would be beder of in the HDHP ex post Puzzle: 7% of the popula6on choose the HDHP in 2011, 15% in 2012 Standard model loads everything on risk aversion Exact same provider network in both plans Hypothesis: With limited informa6on may think HDHP has worse newtork Less than 50% can correctly answer this Many are not sure but 20% are in line with the hypothesis Model es6mates show those repor6ng PPO has a more generous are willing to give up $2,277 on average

18 Es6mates and implica6ons for policy and welfare Structural model of consumer choice incorpora6ng survey responses: Model future spending and choices between plans Iden6fies risk aversion and preferences Welfare: Presence of informa6on fric6ons impacts policy and welfare Consider two counterfactual scenarios: i) move popula6on to HDHP and ii) op6mality of HDHP given moral hazard Welfare loss due to move to HDHP is $1,168 in naïve model and $786 in the full model (risk neutral is $722)

19 Using the full big data toolbox to address choice errors Addressing informa6on fric6ons/choice errors requires: 1. Compu6ng the necessary expecta6ons (predicted cost) 2. Mapping those predic6ons into a u6lity func6on 3. Summarizing those findings in a way that facilitates choices

20 Recommenda6ons to Behavioral Actors Predict non- parametric distribu6on outcomes using a minimum set of inputs (e.g. drugs, age, gender, zip, income) Map outcomes into a risk aversion decision makers u6lity Rely on informed matched individuals to use revealed preference with some behavioral actors

21 Non- linear models capture interac6ons well 28 mean: $2,424 median: $250 PrescripGon Drug Use 60 mean: $9,043 median: $2,995 No drugs ADHD Type II diabetes & hypertension An6- anxiety mean: $1,204 median: $95 mean: $3,538 median: $2,380 OutpaGent Visits mean: $6,521 median: $5,536 mean: $6,160 median: $4,713 No visits 5-10 visits 3-5 visits 6-10 visits mean: $3,348 median: $2,220 mean: $4,091 median: $2,689 mean: $5,586 median: $4,435 mean: $6,849 median: $5,768

22 Across Plans by Individual Personal characteris6cs??? Under ??? Male Female??? Male Female No Drugs Drug 1 Drug 2 No drugs Drug 1 Drug 2 Under and no other drugs And Drug 2 and and no other drugs and Drug 2 And No Drugs Drug 1 Drug 2 No outpa6ent visit 1 outpa6ent visit last year Male Female and no other drugs and Drug 2 and Under No outpa6ent visit 1 outpa6ent visit last year No outpa6ent visit 1 outpa6ent visit last year

23 Predic6ng Health Spending Accurate es6mate for health spending in each plan that can be rapidly recovered Random forest performs well

24 Evidence from State Exchanges Overcome lack of linked data (e.g. APCD) by using high dimensional matching and simula6on Match individuals using moments of actual plan choice distribu6on from available Xs (individual and by plan share) Compare actually chosen plans to alterna6ves based on predicted cost

25 Choice Errors on State Exchanges Actual vs Lowest RealCost Choices 19% 7% 26% 27% 61% 22% 25% 12% 34% 2% 60% 29% 29% 27% Actual Lowest RealCost Actual Lowest RealCost 11% Exchange 1 Exchange 2 Catastrophic Bronze Silver Gold Pla6num

26 Incorpora6ng Risk Aversion Share of Enrollees Choosing Best Plan by Levels of Risk Aversion 25% 28% 28% 28% 20% 19% 9% 8% 7% 10% 9% 8% Best Plan Top 5 Plan Best Plan Top 5 Plan Exchange 1 Exchange 2 Low Medium High

27 How well could decision support do? Predict for individuals and assume they take the advice Comparison of Plan Choices with and without Decision Support 91% 71% 53% 49% 24% 24% 10% 10% Exchange 1 Exchange 2 Exchange 1 Exchange 2 Lowest Cost Plan Top 5 Lowest Cost No Decision Support Decision Support

28 Addressing Consumer Choice Errors

29 Demand Side Incen6ves Heavily studied ques6on (e.g. RAND, Oregon HIE) LATE es6mate of impact in study popula6ons à key parameters used for most health policy decisions Elas6ci6es are sufficient sta6s6cs for op6mal insurance (e.g. Zeckhauser, 1970) and VBID (Baicker, et al., forthcoming) Poten6ally important heterogeneity How does offering HDHP plans impact cost/health care u6liza6on (Brot- Goldberg, Chandra, Handel and Kolstad, 2015)? Look at the forced switch of an en6re large firm from complete first dollar coverage (PPO) to HDHP Study: i) impact on cost and u6liza6on ii) price shopping iii) response to non- linear contract

30 How is spending impacted by move to HDHP? Move from 100% actuarial value to 73%

31 Impact by Illness Level

32 Are People Shopping for Lower Prices? Decomposi6on of changes into price infla6on, price shopping and quan6ty change Rely on detailed data for one large markets (see en6re sample of procedures, doctors and prices) Limit sample to doctors who have at least 30 observa6ons in both 2012 and 2013

33 Response to Different Components of Non- linear Contract Do people respond to the expected end- of- year price (economic theory) or spot prices Compare spending based on people who are at the same level of spending at a certain point in the year to equivalent people in the prior year at the same level Assume 0 price spending trajectory captures counterfactual

34 Understanding Price Response Price and spending are a complex func6on of spending up to period t Poten6ally high degree of correla6on between independent price variables Rely on Lasso to assess rela6ve importance flexibly (similar to approach in Backus, et al., 2015 who study pricing on ebay)

35 Lasso Results

36 Conclusion Machine learning holds a great deal of promise for economics (e.g. Lasso is the new OLS) Applied work on topics with poten6ally behavioral models is a great applica6on Consider sufficient sta6s6cs for welfare analysis in modeling combined with heterogeneity in treatment effects Integra6ng machine learning and economics has great poten6al to not only find but to address choice errors and other internali6es in health care (e.g. Baicker, Mullanaithan and Schwartzstein, forthcoming) Economics can contribute to ML with casual inference, amongst other things

37 EXTRA SLIDES

38 CAUSAL INFERENCE WITH TREE MODELS, AN APPLICATION TO PHYSICIAN DECISION MAKING

39 Background High average returns to health care spending with low marginal returns (e.g. Cutler, 2002) Typical assessment of health care treatments relies on mean impact (e.g. RCTs) Variance (and distribu6on) of outcomes may mader to physicians and pa6ents (e.g. back surgery, drugs for stage 4 cancer) ML yields drama6c improvements in modeling OOS R^2 using observable pa6ent characteris6cs

40 Simple Model

41

42

43

44 Model Study c- sec6on vs. vaginal delivery Model difference in mean and variance as a func6on of observables from detailed claims data

45 Models of Mean and Variance Linear approach follows tradi6onal econ literature (e.g. Fang, Keane and Silverman, 2008) Machine learning approach integrates detailed claims data Endogeneity of treatment requires IV 2SLS in linear model Integrate control func6on into tree models (economics à ML)

46 Demonstra6ng Control Func6on Approach to Inference with Tree Models In general, no asympto6cs in ML; Monte Carlo instead (Athey and Imbens NBER lecture) Replicate Varian (2014) example from the Titanic data Two steps: 1 First stage regression 2 Incorporate residuals in tree 3 Set residuals to zero in predic6on

47 Alternate Models

48 Results

49 Results on Treatment Choice Only model that shows the right sign on both is boosted model with control func6on Mean and variance impact treatment decisions