Application in Predictive Analytics. FirstName LastName. Northwestern University

Transcription

1 Application in Predictive Analytics FirstName LastName Northwestern University Prepared for: Dr. Nethra Sambamoorthi, Ph.D. Author Note: Final Assignment PRED 402 Sec 55 Page 1 of 18

2 Contents Introduction... 3 Company Background... 3 Survey Population:... 3 Hypothesis... 4 Sample Frame... 4 Sampling Size... 5 Sampling Method... 6 Strategy to reduce bias... 6 Strategy for missing or poor quality data:... 7 Analysis methods... 8 Modeling Plans: Questionnaire Check List References Page 2 of 18

3 Introduction Insurance companies are facing a tough economy, rising cost of operations and increasingly demanding customers who want personal and seamless customer experience. Insurance is a unique industry where services are delivered only when a claim is filled. When a customer files a claim, he/she has loses something personal. One bad claims experience is enough for him/her to switch and possibly never return to the same insurance company. Therefore claims customer service and experience becomes an important part of the services offered by an insurance company. In this case study, we will design customer satisfaction survey for X Insurance Company. Company Background (Due to client confidentiality the name X insurance company is used) X insurance company is North America s leading personal auto insurance company (by premiums). It also is the leading home insurer and offers non-medical health and life insurance through its subsidiary companies. Its products are marketed via more than 18,000 agents in the US and Canada. X insurance company diversified businesses also includes a federal savings bank that offers consumer and business loans, savings and checking accounts, and investment products. X insurance company services on average 3,000 claims per day and reducing claims operation cost and improving customer service is core part of the strategy. Survey Population: For our auto claims satisfaction survey, the total population equates to all the claimants who have settled a claim with X insurance company in the last 3 months. The insurance company has settled over 320,000 claims in the last 3 months. Page 3 of 18

4 Hypothesis Claimant experience impacts loyalty and every insurance company s profitability. Therefore it is very important for every insurance company to provide exceptional and personalized customer experience at the time of a claim. The hypothesis is believed to be that among claimants who are delighted with their claims experience will definitely renew with X insurance company and "definitely will" recommend the company to others. Among claimants who are "pleased" by their customer experience, they may" renew their policy and among claimants who have a bad claim experience, they is a high probability of them switching the insurance company during their next renewal or possibly sooner. Sample Frame In our case we are using simple random sampling whereas sample frames are used widely to improve the design for stratified sampling technique. Unlike traditional survey techniques where sample frames can include electoral register, telephone directory, public database type sources but X insurance company will have all the information readily available that can be sourced from the internal claims database. We will have the sample frames from individuals who have filled for a claim amount greater than $2000 with the X insurance company and will excludes claimants whose vehicle incurred only glass/windshield damage or was stolen, or who only filed roadside assistance claims. Page 4 of 18

5 Sample Frame below: Claim Policy Customer Age Customer Number Number Name Claim Amount J Smith T Moody J Lopez A Lee G Patel Y Upad T yao F Lonzie J Jimmy h Jiines G Jines K Deapn F rosie L Kumar F sharma P Shrmae I Raofd R Martini J Smith Sampling Size As we know that the sample size increases as the margin of error and confidence interval decreases. We have chosen a margin of error of 2 % with 95% confidence level. Based on the formula (T 2 ) [p (1-p)]/M 2, X insurance company will require minimum of 2401 responses. We also know that larger sample size lead to increased precision when estimating a population therefore for our survey, we will decided to target more than 5000 auto insurance customers who have filled a claim within the past 3 months. Page 5 of 18

6 Sampling Method We have chosen simple random sample (SRS) as a method of choice so that all subsets of the frame are given an equal probability. This will also minimize bias and simplifies analysis of results. The SRS method that we chose is not vulnerable to sampling error because it is possible to identify every member of the population. The randomness of the selection reflects the makeup of the population since we are only targeting around 5000 claimants from a population of 320,000 claimants. Strategy to reduce bias Design and data collection bias Below are guidelines that will be followed to reduce the issue: Design easy to understand questions so that they can be understood by the entire sample in the same way. Question should be kept simple and should not include any presupposition. Survey questions should be kept applicable to all the respondents No questions should be asked where the answer may not be known. Options for the questions should be kept balanced and easily interpretable Technical jargon should be avoided from the questions. Non response Bias Nonresponse bias can be a major contributing source of total survey error which is very common phenomenon and can cause skew survey results tremendously. They happen because sometimes individuals chosen for the sample are unwilling or unable to participate in the survey. In our case, we are targeting claimants who have filled a claim within the last 3 months. It is possible that majority of the claimants who respond to the survey are Page 6 of 18

7 individuals who had good claims experience so they may not bother to respond. In order to deal with this issue, X insurance company has decided to offer 10% premium discount on the next policy renewal for every survey participants. This incentive will ensure that claimant have an additional motive to reply to the survey. Planning/Voluntary Bias Voluntary response bias can occurs when sample members are selfselected volunteer. The resulting sample tends to over represent individuals who have strong opinions. This will be a very common influence in our claim survey as the respondents who had a bad claim experience are more like to respond. This issue will be solved through random sampling in which all probabilities are equally likely to happen. Interviewer Bias Fortunately for our survey implementation this issue is avoided as the surveys are administered via web right after the claim is settled. Strategy for missing or poor quality data Please Note By design the web survey has 10 questions and all answers are mandatory. This will take care of a lot of missing data issues. Missing data is almost part of all survey and data collection methodologies. The strategy for missing data is based on the nature of the missing data itself as there are several reasons why data may be missing. Missing completely at random (MCAR) This missing data types are not a huge issue since the analysis remains unbiased. We will use the most common means of dealing with missing data which is list wise deletion. If the data are missing completely at random, then list wise deletion does not add any bias, but it does decrease the power of the analysis by decreasing the effective sample size,. We have taken a huge sample size of 5000 claim respondents therefore this should not be an issue. Page 7 of 18

8 Missing not at Random (MNAR) - When we have data that are MNAR we have a problem. For our survey we will use the below techniques to address missing data issues - Missing value can be imputed from a randomly selected similar record - We may be replacing any missing value with the mean of that variable for all other cases, which has the benefit of not changing the sample mean for that variable - Regression imputation - A regression model is estimated to predict observed values of a variable based on other variables, and that model is then used to impute values in cases where that variable is missing - Propensity score method - In the propensity score method, a propensity score is generated for each variable with missing values to indicate the probability of the observation being missing. The observations are then grouped based on these propensity scores and an approximate Bayesian bootstrap imputation is applied to each group. Page 8 of 18

9 Analysis methods We will use several graphical means to analyze and report the data to our stakeholders. Since this assignment is more focused on the survey design itself, survey analysis and reporting is discussed briefly using graphics and cross tabulations reports for the first 3 questions. 1. Approximately, how long did it take to settle the claim? Claim Time 10% 15% 20% 25% 2 wks 2-4 wks 4-6 wks 6-8 wks Over 2 months 30% 2. What is the most important feature you like in an insurance company? Preference Percentage Score Age Demographics a. Flexible Products 10% b. Superior customer service 16% c. Technology & ease of use 30% d. Premium Discounts 44% All Ages Page 9 of 18

10 3. How would you rate the claim intake process for the FNOL (First Notice of Loss)? Excellent 2 Good 5 Average Poor 2 3 Claim Amount > >Claim Amount <5000 Claim Amount <2000 Very Dissappointed Modeling Plans X insurance company wants to predict the impact on customer retention due to customer claims experience. For this the company will be using 1- Hypothesis testing using F and T tests there are several hypotheses that are being developed. For e.g. Customer interaction with the agent is directly correlated to the level of customer service. Another hypothesis is that the amount of time it takes to settle a claim is directly correlated to high claims experience. We will do several other hypotheses testing on the survey data. 2- ANOVA Analysis of variance 3- Cluster Analysis 4- Text Analytics Page 10 of 18

11 Questionnaire PLEASE NOTE: X insurance company has all the demographics (Age, Sex, Address etc.) Behavioral and the Claims data available in the other source systems that will have to be integrated with this survey data before the analysis. The survey is conducted via web and all the fields are mandatory in order to avoid missing data issue. Also remember that there is 10% premium reduction discount offered by X insurance company in order to improve the response rate and thus the overall effectiveness of this claim satisfaction survey 1. Approximately, how long did it take to settle the claim? a. Less than 2 weeks. b. 2-4 weeks. c. 4-6 weeks. d. 6-8 weeks. e. More than 2 months 2. What is the most important feature you like in an insurance company? a. Flexible Products b. Superior customer service c. Technology and ease of use d. Premium Discounts Page 11 of 18

12 All the questions below are on a scale from 1 (Very Disappointed) to 5 (excellent): 3. How would you rate the claim intake process for the FNOL (First Notice of Loss)? 1. Very Disappointed 2. Poor 3. Average 4. Good 5. Excellent 4. How would you rate the explanation of the claim process to you by the assigned claim rep.? 1. Very Disappointed 2. Poor 3. Average 4. Good 5. Excellent 5. How would you rate our communication & over all attentiveness to your needs during your claims process? 1. Very Disappointed 2. Poor 3. Average 4. Good 5. Excellent Page 12 of 18

13 6. The insurance appraiser/adjuster was friendly and courteous. 1. Very Disappointed 2. Poor 3. Average 4. Good 5. Excellent 7. From start to finish, the insurance appraiser/adjuster processed your claim in an efficient manner? 1. Very Disappointed 2. Poor 3. Average 4. Good 5. Excellent 8. How would you rate your overall claims experience? 1. Very Disappointed 2. Poor 3. Average 4. Good 5. Excellent 9. Would you recommend us to your friend and family? 1. Definitely Yes 2. Maybe 3. Definitely No Page 13 of 18

14 10. If you could change one thing about the claims process, what would it be? Free Text. Page 14 of 18

15 Page 15 of 18

16 Check List Survey Meta Data Sponsor, broad objective to achieve Definition Claims Vice President X insurance company services nearly 5,000 claims per day. Improving customer service is core part of the strategy; therefore company would like to capture customer claims experience through claims satisfaction survey so that they can predict which customers are more likely to leave the company thus reducing customer retention. Third party No. Internal to X insurance company only collaboration Purpose The purpose of the survey is to capture claims satisfaction of the customer. Survey Time Period Survey Design Open for 2 months One time panel Target population Customer that have a closed claim in the last 6 months. 65,000 approximately Sampling Method Sample Size Use of Interviewer? Mode of administration Type of computer Simple Random Sampling 5000 claimants No Web Based survey offered immediately after claim settlement Not Applicable assistance Page 16 of 18

17 Reporting/Respondent 1 person Frequency of survey Interview per round of Once after claim is settles Not Applicable survey Levels of observations Link to the survey Personal claimant who files the claim Not Applicable. It will be sent automatically once the claim is settled. Page 17 of 18

18 References 1) 2) 3) 4) 5) 6) 7) 8) 9) 10) sfactionsurvey 11) Page 18 of 18