AN ANALYSIS OF INSURANCE COMPLAINT RATIOS Richard L. Morris, College of Business, Winthrop University, Rock Hill, SC 29733, (803) 323-2684, morrisr@winthrop.edu, Glenn L. Wood, College of Business, Winthrop University, Rock Hill, SC 29733, (803) 323-2599, woodg@winthrop.edu ABSTRACT Evaluating the quality of service from a prospective insurer is a formidable challenge even for the most experienced financial advisors. One approach to evaluating service is to assess the insurer s complaint ratio. Using data that are not readily available, this study describes the complaint ratios for six basic lines of insurance and analyzes the unique problems inherent in the complaint ratios for small insurers. In addition, the authors analyze the relationship between insurer size and complaint ratios, and investigate the question of whether bad complaint ratios tend to be followed by improved ratios. INTRODUCTION The purpose of this paper is to perform a macro (aggregate) analysis of complaint data for six lines of business for the years 2004-2006. Unless otherwise noted, all the data for this paper were provided by material supplied by the National Association of Insurance Commissioners (NAIC) apart from information available from the Internet website [1]. The lines of business under scrutiny are: 1. Private passenger auto 2. Homeowner s 3. Group life 4. Individual life 5. Individual accident and health, and 6. Group accident and health. More specifically, we will: 1. describe the complaint ratios for the above six lines of business, 2. analyze the relationship between insurer size and complaint ratios [2], 3. investigate the question of whether bad complaint ratios tend to be followed by improved ratios, and 4. determine whether there is a relation between complaint ratios of companies operating in similar lines of business. There is no intention of analyzing the statistical problems inherent in the data as this has been done previously [3], but we will, by necessity, address a few of the major questions that arise in the interpretation of the information provided.
THE SMALL COMPANY PROBLEM For purposes of this paper complaint ratios are calculated as follows: Complaint Ratio= # of complaints Premium Volume x1,000,000 (1) In analyzing premium volume data, it becomes evident that large insurers write most of the business in every line and many, many small companies compete for the remainder. This market fact raises a very important problem with the interpretation of complaint ratio data for small companies. Specifically, the complaint ratios for small companies can be extremely misleading; they can make small companies look much better or much worse than they really are. Consider the following simple example for two companies in the same line of business: Company A Company B Expected Number Of Complaints 800 2 Premium Volume $5,000,000,000 $12,500,000 The term Expected Number of Complaints is the number of complaints we would expect from each firm if it is operating as usual. An equivalent definition is that it is the long-run average number of complaints that each firm would incur if it is operating as usual. Company A s expected complaint ratio is then: Expected Complaint Ratio = and Company B s is: Expected Complaint Ratio = 800 x1,000,000 =.16 5,000,000,000 2 x1,000,000 =.16 12,500,000 So both companies are operating at a similar level of service as measured by their expected complaint ratios. However, it is very easy to see that Company B, just by chance variation, could have zero complaints in a year, resulting in a complaint ratio of 0. Likewise, if it has six complaints in a year, just by chance, it would have a complaint ratio of.48. Using the Poison distribution we can model the distribution of complaints for both companies. Using this approach, the probability of zero complaints for Company B is about 13.5%. The probability of having six or more complaints in a year is about 1.7%. We can use a reverse analysis for Company A to get the same probabilities. The probability of Company A s having 769 or fewer complaints is about 13.7%. This would yield a complaint ratio of.154. The probability of Company A s having 861 or more complaints is about 1.7%, with a complaint ratio of.172.
Looking at this example in another way, the probability of each company operating in its range above is about 85% [100 - (13.5 + 1.7) = 84.8% for Company B; 100 - (13.7 + 1.7) = 84.6% for Company A]. So Company B s complaint ratio will lie within the range 0 to.48 while Company A s will be in the much smaller range of.154 to.172, both with equivalent probabilities. The problems of scale in evaluating complaint ratios are very serious. The following table illustrates average complaint ratios and the variability of the ratios as premium volume increases: TABLE 1. MEANS AND STANDARD DEVIATIONS OF COMPLAINT RATIOS BY LINE OF BUSINESS Private Passenger Homeowner s Quintile x s Quintile x s 1 0.130 0.014 1 0.125 0.022 2 0.251 0.048 2 0.150 0.030 3 0.332 0.072 3 0.234 0.098 4 0.312 0.090 4 0.250 0.107 5 0.514 0.338 5 0.524 0.365 Group Life Individual Life Quintile x s Quintile x s 1 0.013 0.004 1 0.032 0.004 2 0.023 0.003 2 0.038 0.006 3 0.027 0.006 3 0.042 0.009 4 0.023 0.007 4 0.076 0.015 5 0.289 0.254 5 0.269 0.170 Individual Accident and Health Group Accident and Health Quintile x s Quintile x s 1 0.090 0.003 1 0.081 0.026 2 0.231 0.154 2 0.097 0.029 3 0.226 0.161 3 0.079 0.023 4 0.248 0.074 4 0.117 0.048 5 0.687 0.507 5 0.182 0.106 This table shows the mean and standard deviation of the complaint ratio by quintile of premium volume for each line of business. In the table, the simple arithmetic average of each measure is calculated for the first through fifth quintile in each line of business. From Table 1 we see, for every line of business, a very strong tendency for the average complaint ratio to increase as premium volume decreases. This relationship is clear and almost perfectly consistent. A reasonable, or at least possible, explanation is that the larger companies do a better job of providing higher-quality service to their customers.
The greater variability in complaint ratio for smaller companies is likely due to the effects of a small absolute number of complaints for smaller companies versus larger absolute numbers of complaints for larger ones. After a fair amount of analysis we concluded that there is no magic cutoff size, i.e., a company size where the number of complaints becomes much more (or less) meaningful. To address the issue further we determined the number of zero complaint ratios in each quintile. We found that they were highly concentrated in the fifth quintile for each line of business with just a smattering of them in the fourth quintile. In recognition of the above problems some states list only the companies with a number of complaints that exceed a specified minimum, such as ten. Other states do not list the complaint ratios of companies that have a premium volume less than a certain minimum. Unfortunately there is no method that solves the small company statistical problem without introducing additional problems. There is no natural dividing line between large and small companies in any line of insurance. This means any classification by size will be somewhat arbitrary and subject to criticism. Nonetheless, financial advisors (and consumers) who use complaint ratios for very small companies should understand that these ratios have little or no meaning. IMPROVEMENTS IN COMPLAINT RATIOS An insurer might be concerned if its complaint ratio increases substantially from year to year or is too high compared to comparable size companies. In other words, a company might be concerned if the complaint ratio has increased, or the company might be concerned if the level of complaints is viewed as unacceptable. That is, a bad complaint ratio might be viewed as one that is increasing or it could be defined as one that is higher than some standard set by the company. Consequently, in examining the question of whether companies tend to improve after bad ratios, we tested both concepts. To test the idea that a company might take steps to improve its complaint ratio after experiencing a bad ratio, we defined a bad ratio as one that had increased by 5% or more from the previous year. With three years of data, we arbitrarily decided that if a company s complaint ratio increased by 5% from 2004 to 2005, then 2005 would be labeled a bad year. Then we determined whether or not those companies with a bad year in 2005 improved their complaint ratio in 2006. In the Private Passenger line of business, for example, 67% of the companies experiencing a bad year in 2005 improved their complaint ratios in 2006, suggesting that there may be a pattern of improvement after a bad year. To determine if there was a significantly higher proportion of firms with bad complaint ratios in 2005 that improved versus those that did not have bad complaint ratios in 2005 we performed a test of two proportions. The hypotheses we tested were: H o : The proportion of companies showing improvement from 2005 to 2006 is the same regardless of whether they experienced bad complaint ratios in 2005. H a : The proportion showing improvement is higher for those having bad complaint ratios in 2005.
We performed this test on each line of business using a z-test on proportions. Because of the high variability inherent in very low numbers of complaints, we based the analysis only on companies whose number of complaints for each year was greater than ten. The results are shown in Table 2. As shown in the table, the z-test statistic for the Private Passenger line of business is 1.785, with a p-value of.037. The null hypothesis will be rejected if the p-value is less than the significance level,. If we choose the customary.05, we would reject the null and can conclude that there was a significant improvement effect in the Private Passenger line. TABLE 2. IMPROVEMENTS AFTER A BAD RATIO #of Bad CR's in 2005 # of Good CR's in 2005 Proportion Proportion z- Line of Business Improving Improving statistic p-value Private Passenger 67 67.2% 249 55.0% 1.785 0.037 * Homeowner's 25 76.0% 109 53.2% 2.079 0.019 * Group Life 3 33.3% 8 50.0% -0.494 0.689 Individual Life 35 65.7% 71 47.9% 1.731 0.042 * Individual Accident and Health 39 56.4% 95 53.7% 0.288 0.387 Group Accident and Health 64 65.6% 137 59.1% 0.881 0.189 * Significant at the.05 level ** Significant at the.01 level Note that the sample sizes for Group Life insurance are too small to draw any meaningful conclusions. This is true of the remaining tables in this section also. Therefore, we will eliminate this line of business from any further consideration. In Table 2, the second column lists the number of bad complaint ratios for 2005 in each line of business. The next column is the proportion of those with bad complaint ratios that improve from 2005 to 2006. The next two columns list the number of firms with good complaint ratios in 2005 and the proportion of those that improved from 2005 to 2006. These firms act as a control group we can compare to those with bad ratios. From this table, it can be seen that while the proportion improving was higher for those companies having bad complaint ratios in 2005 in five of the lines of business, only three, Private Passenger, Homeowner s and Individual Life, had significantly higher proportions at the 5% level of significance. Taking the other approach, we reasoned that some companies might look at other similar companies as a means of deciding whether their complaint ratios need improvement. A very rough way of doing this is to look at whether a company s complaint ratio is high relative to the mean of the quintile it is in. We used the same 5% figure as in the previous analysis. That is, if a firm s complaint ratio is 5% higher than its quintile average, then it would be classified as a bad ratio. We then determined the proportion of those with bad complaint ratios that improved and
compared this to the proportion of those companies not classified as having bad ratios that improved. Table 3 shows the proportions of companies in both categories that improved from 2004 to 2005 and Table 4 shows the same information for improvements from 2005 to 2006. As before, we eliminated from consideration those companies with ten or fewer complaints in any of the three years considered. TABLE 3. IMPROVEMENTS RELATIVE TO SIMILAR SIZE COMPANIES (2004 AND 2005) #of Bad CR's in 2004 # of Good CR's in 2004 Proportion Proportion z- Line of Business Improving Improving statistic p-value Private Passenger 157 86.0% 159 61.0% 5.026 0.000 ** Homeowner's 76 84.2% 58 67.2% 2.308 0.010 ** Group Life 5 80.0% 6 66.7% 0.494 0.311 Individual Life 54 64.8% 52 51.9% 1.347 0.089 Individual Accident and Health 113 68.1% 21 52.4% 1.397 0.081 Group Accident and Health 100 69.0% 101 58.4% 1.560 0.059 * Significant at the.05 level ** Significant at the.01 level TABLE 4. IMPROVEMENTS RELATIVE TO SIMILAR SIZE COMPANIES (2005 AND 2006) #of Bad CR's in 2005 # of Good CR's in 2005 Proportion Proportion z- Line of Business Improving Improving statistic p-value Private Passenger 124 67.7% 192 51.0% 2.933 0.002 ** Homeowner's 58 67.2% 76 50.0% 2.000 0.023 * Group Life 3 33.3% 8 50.0% -0.494 0.689 Individual Life 52 65.4% 54 42.6% 2.353 0.009 ** Individual Accident and Health 112 58.0% 22 36.4% 1.866 0.031 * Group Accident and Health 93 72.0% 108 51.9% 2.929 0.002 ** * Significant at the.05 level ** Significant at the.01 level
Table 2, as discussed previously, suggests that companies try to improve after a bad year, but the results are not particularly dramatic. Tables 3 and 4 give stronger results. In Table 3, only two lines of business yield significant results, but the rest produce p-values fairly close to the.05 level of significance. Table 4 produces significant results (at the.05 level) in all of the five lines of business after excluding Group Life insurance. Three of these five are significant at the.01 level. Do the results tend to support the idea that companies try to improve after experiencing bad complaint ratios? The preceding analysis suggests that they do. We cannot say with certainty that they use the complaint ratios themselves to decide if they receive too many complaints; they may use more informal measurements or they may use different criteria than we used for detecting improvements. Nevertheless, the results strongly suggest that insurers are concerned about getting too many complaints and that they do indeed take steps to improve bad complaint experience. CORRELATIONS BETWEEN LINES OF BUSINESS AND YEARS We are also interested in seeing if complaint ratios are correlated between different lines of business. To analyze this question, we calculated the Spearman rank correlation coefficient (r S ) for Personal Passenger and Homeowners insurance for each of the years for which we have data, shown in the next table: TABLE 5. CORRELATIONS FOR PERSONAL PASSENGER AND HOMEOWNER S INSURANCE Year 2006 2005 2004 r S 0.496 0.593 0.582 p-value 2.24E-08 4.44E-12 1.41E-12 So, for example, the correlation relating complaint ratios of Personal Passenger insurance and Homeowners insurance is.593 for 2005. This bears a little explanation. To begin with, the correlation coefficient itself lacks a reasonably good interpretation. However, its statistical companion, the coefficient of determination, or r s 2, can be interpreted as the amount of variability in one variable that can be explained or accounted for, by the other. The r s 2 for 2005 is.352 (.593 2 ). So 35.2% of the variability in the complaint ratios for Personal Passenger insurance can be explained by the variation in complaint ratios for Homeowners insurance (and vice-versa; the relationship holds in both directions).
Personal Passenger Ranking It might help also to show a graph of the complaint ratios for the two lines of business for 2005: FIGURE 1. COMPLAINT RATIO RANKINGS OF PERSONAL PASSENGER AND HOMEOWNER'S INSURANCE, 2005 120 100 80 60 40 20 0 0 20 40 60 80 100 120 Homeowner Ranking From this it can be seen that the correlation between the two isn t that dramatic. The third row of Table 5 contains the p-values of the coefficients. The p-values are extremely low, indicating that the correlations are highly significant, but this doesn t mean that they re particularly meaningful, just that there s a relationship (maybe a small one) that can be detected in a statistical test of hypotheses. So if we re trying to answer the question of whether or not companies tend to have comparable complaint ratios in separate lines of business, the answer is Yes, but it s not a particularly close relationship. CONCLUSIONS Complaint ratios vary greatly by line of business. Complaints in property and liability insurance are consistently much higher than in life insurance. This might be explained by the fact that most complaints arise from the handling of claims, with underwriting, policyholder service, and marketing and sales all together accounting for a minority of all complaints. All insurance markets are dominated by large insurers, and the analysis of complaint ratios for small companies is very difficult because of statistical problems. With a small premium volume, small fluctuations in the absolute number of claims will cause the complaint ratio to fluctuate wildly. Accordingly, the complaint ratios of very small companies are essentially meaningless. The analysis supports the conclusion that larger companies have lower complaint ratios than smaller companies. This was true for every line of business in every year of our analysis. Using two different definitions of bad complaint ratios, we found statistically significant results at the.05 level in all lines of business (not including Group Life insurance, which had too few complaints to analyze). In three of the five lines of business the results were significant at the.01 level. This is strong evidence that insurance companies are concerned about their complaint ratios and it seems probable that they take steps to improve their service if complaints increase.
We also looked at the correlations between lines of business and found that there was a significant correlation in rankings of complaint ratios for companies engaged in different lines of business. However, while significant, the correlations were not particularly meaningful. REFERENCES [1] National Association of Insurance Commissioners (NAIC) Consumer Information Source https://eapps.naic.org/cis/ [2] Query, J. T., Hoyt, R, E, & He, M. Service quality in private passenger automobile insurance. Journal of Insurance Issues,2007, 30(2), 155-165. [3] Venezian, E. Complaint ratios: What (or where) is the beef? Journal of Insurance Regulation, 2002, 20(4) 19-46.