The Effect of Report Cards on Consumer Choice in Health Insurance Market

Transcription

1 Journal of Health Economics 21 (2002) The effect of report cards on consumer choice in the health insurance market Gerard J. Wedig a,, Ming Tai-Seale b a William E. Simon Graduate School of Business Administration, University of Rochester, Rochester, NY 14627, USA b School of Rural Public Health, Texas A & M Health Science Center, College Station, TX, USA Received 1 April 2001; accepted 6 June 2002 Abstract We test the effect of report cards on consumer choice in the HMO market. Federal employees were provided with report cards on a limited basis in 1995 and then on a widespread basis in Exploiting this natural experiment, we find that subjective measures of quality and coverage influence plan choices, after controlling for plan premiums, expected out of pocket expenses and service coverages. The effect is stronger within a small sample of new hires compared to a larger sample of existing federal employees. We also find evidence that report cards increase the price elasticity of demand for health insurance Elsevier Science B.V. All rights reserved. JEL classification: D83; I11; L15 Keywords: Health insurance; Report cards; Consumer choice 1. Introduction Report cards help consumers make better-informed choices among the products and services they consume. In the case of health insurance, report cards seek to translate complex data about plan benefits and treatments into a small number of dimensions that are understandable and useful to consumers. 1 Effective report cards expand the consumer s information set and enable them to select plans that offer the best tradeoff between quality Corresponding author. Tel.: ; fax: address: wedig@simon.rochester.edu (G.J. Wedig). 1 Perhaps the best-known example of a health plan report card is the health plan employer data and information set survey (HEDIS). Consumer reports has also published health plan report cards /02/$ see front matter 2002 Elsevier Science B.V. All rights reserved. PII: S (02)

2 1032 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) and cost. The effect of report cards on consumer decision-making depends on the information consumers have about health insurance plans in the absence of report cards and on report cards conveying meaningful additional information to consumers. The purpose of this paper is to test the hypothesis that consumer report cards influence the consumer s choice of health plan. In this paper we provide an empirical test of the joint hypothesis that report cards provide relevant information to consumers and that consumers are less than fully informed in their absence. Our evidence is based on the plan selections of a sample of new and existing US federal employees in the years 1995 and Federal employees select their health insurance under a program called the federal employees health benefits plan (FEHBP), 2 which provides them with multiple plan choices. Starting in 1995, the FEHBP s administrative arm, the Office of Personnel Management (OPM), began to compile report cards on each included plan, although these were not widely distributed. The following year, 1996, the report cards were also widely distributed. We are able to use the natural experiment created by OPM to test the joint hypothesis. Our tests are based on analyses of plan choice in 1995 and The 1995 choice regressions establish a baseline of consumer information with respect to financial and quality dimensions, prior to the widespread introduction of report cards. Re-estimation of the choice regressions using 1996 data is used to assess the net impact of report cards on consumer choice. We focus on the differences in key coefficients for plan quality and coverage between 1995 and 1996 in order to measure the impact of report cards on consumer-decision making. Our results contribute to the emerging literature that considers the role of health plan report cards in consumer-decision making (Edgman-Levitan and Cleary, 1996; Hibbard and Jewett, 1996, 1997; Sainfort and Booske, 1996; Tumlinson et al., 1997; Chernew and Scanlon, 1998; Spranca et al., 2000; Beaulieu, 2002; Farley et al., 2002; Harris et al., 2002; Scanlon et al., 2002). Early work in this area concentrated on hypothetical questions about the kinds on information consumers would find useful. More recent work has focused on the effect that distributing health plan report cards to consumers has on plan choice. Our work provides additional evidence on this important question. Our results also bear on the more widely researched price elasticity of demand for health insurance. It is possible that in the absence of report cards consumers will use prices to partially infer plan quality, thus reducing the price elasticity of demand. This is consistent with an argument made by Royalty and Solomon (1999), who argue that vigorous price competition between plans can only occur where consumers are confident that plans have comparable quality and coverage. Our results test the proposition that report cards makes choices more responsive to price. The paper is organized as follows. In Section 2, we review the prior work in the area of health plan choice and health plan report cards. In Section 3, we discuss the natural experiment that resulted from OPM s policies. Data and methods are given in Section 4 2 The FEHBP dates back to 1959 when the government first instituted health benefits for federal employees. Under the FEHBP, federal employees select their health insurance from a number of insurers in their local market. Insurers are included in the choice set provided that that they have contracts with the government. The government pays 71% of the plan premium, regardless of the plan selected.

3 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) with results reported in Section 5. The paper concludes in Section 6 with a discussion of its implications. 2. Literature review 2.1. Existing empirical literature on plan choice There exists an extensive literature on the determinants of health plan choice, including the effects of price, coverage, convenience and consumer-specific characteristics. Scanlon et al. (1997) provides a review of this literature. Most existing research has focused on estimating the price elasticity of plan choice decisions. Notable examples include McGuire (1981); Holmer (1984); Welch (1986); Long et al. (1988); Feldman et al. (1989); Short and Taylor (1989); Barringer and Mitchell (1994) and Buchmueller and Feldstein (1996). The major methodological issues addressed in this literature include: defining price properly, devising methods to create meaningful statistical variation in price and applying the proper econometric framework for discrete choice. 3 Studies consistently find a significant and negative impact of price with elasticities ranging from 0.1 to 0.8. More recently, Royalty and Solomon (1999) argue that price elasticities are larger where plan benefits are standardized. This is the competitive model envisioned under managed competition (Enthoven, 1978). Their study of the Stanford health plan finds price elasticities in the range of 0.4 to 0.7. Our study also tests the proposition that the price elasticity is larger if consumers have reliable information about overall plan quality. There are few studies that examine the effect of coverage on plan choice and even fewer studies that examine the effect of plan quality on choice. Those studies that have examined the effects of providing specific benefits, including dental, drug, mental health and preventive care have not found a consistent pattern of significant effects (Welch, 1986; Short and Taylor, 1989; Feldman et al., 1989). Studies that have used measures of convenience as a proxy for quality have also found insignificant results (Juba et al., 1980; Feldman et al., 1989) Effects of report cards on plan choice Consumer report cards, as a measure of plan quality, are only beginning to be tested in a systematic way. Several earlier studies conclude that consumers care about report card measures and would use them to assist with their plan choices (Edgman-Levitan and Cleary, 1996; Hibbard and Jewett, 1996, 1997; Sainfort and Booske, 1996; Tumlinson et al., 1997). Several surveys also indicate that consumers find process measures of consumer satisfaction to be the most useful, largely because they can easily infer the meaning of measures such as overall satisfaction, time spent with the physician, etc. Measures of health outcomes (e.g. mortality rates) are assigned less value by consumers. Spranca et al. (2000), took the additional step of conducting an experiment with 391 adults to test the hypothetical effects of varying plan quality on plan choice. Most notably, 3 Interestingly, none of the articles discuss another potential bias, namely omitted variable bias resulting from the omission of plan quality measures.

4 1034 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) they found that reporting plan quality increases the likelihood that lower priced plans will be chosen over higher priced plans. This has the effect of increasing the price elasticity of demand, consistent with the results of Royalty and Solomon. More recent work has focused on the effects that report card distribution has on plan choice. Chernew and Scanlon (1998) studied the open enrollment decisions of employees already enrolled in an existing plan and found no consistent or plausible pattern of choices. Consumers in the Chernew and Scanlon faced switching costs, which may have muted the impact of report card information. Farley et al. (2002), also find that report card dissemination has little effect on the plan choices of Medicaid enrollees. More recently, Beaulieu (2002), and Scanlon et al. (2002) have found evidence that the dissemination of report card scores does influence consumer choice. Beaulieu finds a small effect of reported quality and satisfaction on the likelihood of plan switching while Scanlon et al. find that consumers avoid plans with many below average ratings. 3. The experiment 3.1. Description The Office of Personnel Management (OPM s) survey of federal employees enrolled in participating health plans created a quasi or natural experiment of the effects of report card distribution on plan choice. 4 In 1994, OPM cooperated with the Center for the Study of Services to conduct the first satisfaction survey of federal employees who were enrolled in FEHBP health plans. In total, federal employees enrolled in 261 different plans participated in the mail survey, with an individual employee response rate of 62%. Separate surveys for enrollees of prepaid plans and fee for service (FFS) plans were conducted. The following aspects of care were rated by enrollees: overall satisfaction, access to medical care, overall quality of care, doctors available through the plan, coverage and information provided by the plan, customer service and simplicity of paperwork. Results of the satisfaction survey were made available to federal employees in the form of an eight-page booklet (report card) intended to help them compare health plans nationwide (McCormack et al., 1996). Survey results were distributed to FEHBP benefit managers in late 1994 for 1995 open enrollment. Based on a subsequent telephone survey of federal employees, OPM learned that only about a quarter of federal employees actually received the booklet during 1994 (United States Office of Personnel Management, 1995a,b). In 1995, another patient satisfaction survey was conducted. Federal employees enrolled in 301 different plans were surveyed with respect to the same dimensions of satisfaction. This time, the survey results were appended to the FEHB Guide and distributed free of charge to all federal employees during the 1996 open enrollment season. The different distribution rates of survey results created a natural experiment for testing the effect of distributing report card information on health plan decisions. 4 The details of the survey and the resulting natural experiment are discussed in McCormack et al. (1996) as well as in the OPM s fall 1995 publication, 1995 FEHB Guide (United States Office of Personnel Management, 1995a,b).

5 3.2. Hypotheses G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) The natural experiment described above gives rise to the study s three hypotheses. H1. Dissemination of report cards influences consumers choice of health plan consumers are less than fully informed in their absence. Report cards can only influence choices where consumers are less than fully informed. H2. Report cards have a greater impact on decisions of new federal employees. New federal employees may have less prior information about health plans and weaker ties to providers. H3. Report cards increase the measured price elasticity of demand. Price is a weaker signal of quality, increasing the measured price elasticity of demand. 4. Data and methods 4.1. Data The primary data for the study were provided by the OPM. OPM provided us with the health plan choices of all federal employees in the years 1995 and We restrict this sample to employees who chose single person HMO coverage and who resided in counties where the set of federal employees made five or fewer unique plan selections in the given year. 5 This yields a sample of 649 new hire choices in 1995 and 713 new hire choices in We match the new hire sample with a sample of existing federal employees. Using the same set of counties that the new hires work in, we select a stratified random sample of 4150 existing hire choices in 1996 and 3650 existing hire choices in 1995 to complement the new hire sample. After sampling we are left with 231 unique counties in which federal employees selected five or fewer unique plans. We examine the counties to see whether they are broadly representative of the US, both geographically and based on population density. We find the 5 We restrict the sample to those employees with five or fewer choices for three reasons. First, we believe that in such counties it is more likely that all plans are viable choices and that the assumption of independence of irrelevant alternatives will hold. Conversely, in counties where the number of choices exceeds five, it is likely that a subset of choices is not in each enrollee s actual choice set due, for example, to the geographic location of the employee in relation to the plan s provider network. Second, we believe that choice sets of this size are more representative of the size of choice sets faced by individuals in the private sector, which helps us to generalize our results. Finally, we believe that markets with a relatively small number of choices are most likely to be influenced by report cards since comparisons can be made between plans with little difficulty. Using this rule reduces our sample of new hires to 649 in 1995 and 713 in On the other hand, the sample of existing employees in the restricted county set remains quite large and our final empirical work on existing employees draws upon a stratified random sample of existing federal employees.

6 1036 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) counties to be broadly representative of the US geographically, drawn from 40 different states. The median county in our sample is located in a metropolitan area and has a population between 250,000 and 1,000,000 people. The list of all counties in our study is available from the authors. The raw data files provide us with information on employee age, sex, pay grade, county of employment and plan choice. The second main data source comprises the FEHB Guides for 1995 and 1996 and report cards supplied to federal employees in these years. The Guides provide information on plan premiums and the report cards provide information on subjective measures (federal employee ratings) of overall satisfaction, access, overall quality of care, doctor availability, coverage and simplicity of paperwork. The ratings are adjusted for the respondents age, education and health status. (United States Office of Personnel Management, 1994a,b). A third data source provides more detailed information on the financial costs of plan membership, including expected out of pocket costs due to copays, deductibles and uncovered services as well as plan quit rates. This information is available in a commercial publication, Checkbook Guide to Health Insurance Plans for Federal Employees (referred to as the checkbooks) and could be purchased by federal employees in both 1995 and 1996 (Francis, 1995, 1996). We use the checkbooks to gather detailed information on the financial consequences on plan membership. Our complete specification includes report card measures, enrollee plan costs, specific plan coverages, organizational model dummies and other controls. In some specifications, these measures are interacted with the personal characteristics of age and sex. 6 To infer the local plan choice set for each employee, we form the set of all plans that were selected by federal employees in the county, including the choices of both existing federal employees and new hires. The independent variables used to test our three hypotheses are the report card measures and plan cost. We restrict our set of included report card measures to overall quality of care and plan coverage. Other available report card measures are highly correlated with overall quality of care. We employ two measures of enrollee cost as additional controls. One is the individual s monthly premium while the second is a measure of annual expected out of pocket costs, defined as the sum of expected copays and deductibles paid by an enrollee over the course of a year. We refer to this variable as out of pocket cost (OOP cost) and obtain this measure from the aforementioned checkbooks (not from the report cards). Specific plan coverages may be attractive to individuals with specific conditions. We include dummy variables for unlimited coverage of outpatient mental health services, prescription drugs and dental accidents. We also include dummies for plan type including staff, group, network, mixed and IPA models. These variables control for consumer preferences for specific organizational models of HMOs. These variables may also pick up unmeasured quality attributes that are correlated with specific delivery models. Finally, we include a measure of each plan s quit rate, defined as the percentage of plan enrollees who quit a plan during open enrollment. The quit rate is another proxy for plan 6 We also try interactions of some variables with pay grade but find no statistically significant impacts. These results are not reported here.

7 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) quality over and above the report card measures. If consumers communicate information efficiently to one another, the quit rate may act as a proxy for negative aspects of plan quality that are communicated by word of mouth Empirical methods Following Feldman et al. (1989), we use a nested multinomial logit (MNL) approach to estimate the parameters of the expected utility model. The MNL model consists of a decision tree in which the employee chooses whether to select an IPA or other model of HMO and, conditional on the model type selected, the specific HMO. To test the need for a nested approach we conduct tests of the independence of irrelevant alternatives (IIA). This allows us to analyze whether there are unmeasured preferences for the IPA model. Our tests of the independence of irrelevant alternatives indicate that the IPA and other nests can be combined, so that final estimates encompass the entire choice set of prepaid plans. To test hypothesis one, we measure the differential impact of report card measures in 1995, when report cards were not widely disseminated, compared to 1996, when they were widely disseminated. A larger report card impact in 1996 supports the hypothesis that report card dissemination matters and that markets are not fully informed in their absence. Hypothesis two is tested using MNL regressions on a stratified random sample of existing federal employees from the same set of counties. According to hypothesis two, we expect a smaller difference in the report card scores between 1995 and 1996 for this sample. This would indicate that report cards have a smaller influence on existing federal employees. Finally, hypothesis three is tested by examining changes in the price elasticity of demand with respect to plan premiums between 1995 and According to hypothesis three, the price elasticity should be larger in 1996, after report cards were widely distributed. 5. Results 5.1. Descriptive statistics Variable definitions are provided in Table 1. Means and S.D. are provided in Table 2. The means are provided separately for 1995 and We also report individual characteristics, (age and gender) separately for the new and existing federal employee samples. Plan characteristics are identical across the two samples. Several characteristics of the data are worth noting. Employee premiums average about US$ 500 on an annual basis (US$ on a monthly basis) and fell slightly from 1995 to The relatively small individual premium reflects a 71% contribution made by the federal government to premium costs. Unlimited outpatient mental health coverage was provided by few plans, while prescription drugs were covered by most plans. A little more than half of all plans provided dental accident coverage. The quit rate for all plans averaged about 9% and about half of all enrollees selecting an HMO chose an IPA. The average quality of care and coverage report card scores were in the mid to upper eighties on a scale of 100. Evidently, most individuals were generally satisfied with their plan s quality of care and

8 1038 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) Table 1 Variable definitions Variable Individual-level Age: new hires Female: new hires Number of plan choices Plan-level Premiums Expected out of pocket costs Mental health coverage Drug coverage Dental coverage Quit rate IPA Network Group Mixed Staff Quality score Coverage score Variable definition Age of employee measured in years Dummy variable gender is female Number of unique plans selected by federal employees in the county Employee s required monthly contribution Expected out of pocket costs if enrolled in this plan, including copays, deductibles and payments for uncovered services Dummy variable plan provides unlimited coverage for outpatient mental health services Dummy variable plan provides unlimited coverage for prescription drugs Dummy variable plan provides coverage for dental accidents Percentage of enrollees who withdraw from the plan annually Dummy variable IPA model Dummy variable network model Dummy variable group model Dummy variable mixed group and network model Dummy variable staff model Percentage of surveyed individuals who rate the overall quality of care as good, very good or excellent Percentage of surveyed individuals who rate the coverage of services as good, very good or excellent coverage. Finally, in comparing existing federal employees with new hires, we find a significant difference in average age, with the existing employee sample averaging 10 years older Tests of IIA The first step in the estimation process is to test the hypothesis of independence of irrelevant alternatives (McFadden, 1978). The IIA hypothesis states that preferences among subsets of choices are uncorrelated. In our case, we test whether individuals have unmeasured preferences for the selection of an IPA versus alternative HMO models, following the arguments of Feldman et al. (1989). If the IIA hypothesis is rejected, this violates a basic assumption of the MNL procedure. The process of testing IIA requires: (1) a division of the sample into IPA and non-ipa subsets (2) estimation of model parameters using the smaller choice set, (3) computation of inclusive value measures for both the IPA and non-ipa choice set and (4) estimation of a binomial logit model to predict the choice of IPA and non-ipa choices. The coefficient on the difference in inclusive values should not be statistically different from one if the hypothesis of IIA is to be supported. We estimate the IIA procedure on a random sample of existing federal employees for the years 1995 and The results are given in Table 3. The results provide mixed support 7 We judge the new hire sample to be too small to support a powerful test of IIA.

9 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) Table 2 Variable means individual and plan-level Variable 1995 mean 1995 S.D mean 1996 S.D. Individual-level Age New hires Old hires Female New hires Old hires Number of plan choices Plan-level Premiums Expected out of pocket costs Mental health coverage Drug coverage Dental coverage Quit rate IPA Network Group Mixed Staff Quality score Coverage score This table presents means and S.D. of the analytic variables. The top half of the table gives the relevant statistics for consumer characteristics, age and gender. The bottom half of the table provides plan-related data for the 125 plans that make up choices in our data set. for the IIA hypothesis. The coefficients on the inclusive values are not statistically different from one in They are only slightly different from one for Based on this limited evidence, we choose not to estimate the model using a nested structure. Instead, the choice among plans combines IPA and other choices into a single decision. Table 3 Tests of independence of irrelevant alternatives (IPA vs. other models) Coefficient S.E. Coefficient S.E. Intercept (0.0341) (0.0534) Inclusive value (0.2948) (0.0374) Number of choosers Log likelihood This table presents the tests of the independence of irrelevant alternatives (IIA). The specific hypothesis tested is that the IPA model choices do not have correlated errors in the choice structures. The test is applied to a randomly-selected group of old hires. Coefficient values for the inclusive value variables close to one support the IIA hypothesis. The IIA hypothesis is accepted for 1996 and rejected for 1995.

10 1040 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) Results for 1995 prior to report card dissemination Regressions of 1995 plan choices for both sets of federal employees are found in Table 4. Results for 1995, prior to the widespread dissemination of report cards, are used to establish a baseline in order to measure the net impact of widespread report card dissemination in To establish robustness, Table 4 includes two different specifications. 8 Both specifications include the report card measures, but different covariates. The first specification includes only the report card measures, the premium and HMO model dummies. The second specification introduces all of the other covariates. The estimated models provide a reasonably good fit. The analogy to R 2, based on the log likelihood function, shows fits in the range of 26 34%. In addition, the results for the new and existing federal employees are quite similar, so they are discussed together. Our chief finding for 1995 is that report card quality of care relates negatively to the enrollment decisions of both new and existing employees. The result holds for both specifications in the two samples and is robust to controls for premium, out of pocket costs, plan coverages, the plan quit rate and two interaction terms. Consumers may have been poorly informed about this dimension of quality during the baseline period. The negative coefficient also suggests that report card quality correlates negatively with unmeasured plan attributes that attract enrollments. For example, one possibility is that report card quality of care correlates negatively with visible plan attributes such as marketing expenses or specific benefit coverages that we do not measure. 9 The coefficient for report card coverage is positive in three of the four specifications shown. However, the coefficient is either insignificant or negative in the two full specifications. Thus, the market appears to be poorly informed with respect to report card coverage also. In short, there is little evidence that consumers are informed with respect to either of the report card measures, quality or coverage. Plan premium is the other variable of interest. In 1995 there are modest premium effects in the expected (negative) direction that become statistically insignificant in the full specifications. The calculated price elasticity is in the low range of what has previously been found in the literature, 0.14 for new hires and 0.02 for existing federal employees. 10 As suggested in the literature, a low price elasticity is consistent with a market that is relatively uninformed with respect to plan quality. It is also consistent with a market in which consumers use the plan premium, in part, to infer quality. Within the set of other controls, the most surprising result is found for expected plan out of pocket cost. The coefficient for this variable is positive and significant and robust to added controls for specific plan coverages. It appears that the market is also poorly informed with respect to this measure of plan expense. On the contrary, plans that increase enrollee out of pocket expense are able to systematically increase their enrollments. Plans with high out 8 The results are also robust to several other specifications not shown here. 9 The alternative hypothesis is that consumers are informed about this dimension of quality but that they regard it as a poor, invalid measure of expected utility. Our results for 1996, discussed below, contradict this explanation. 10 The elasticity is computed by averaging the fitted impact of a 1% change in prices across the entire data set and measuring its impact on the log probability of choice. All observations in the sample are weighted equally for these purposes.

11 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002)

12 1042 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) of pocket expenses may funnel some of the resulting plan savings into more visible plan attributes, such as marketing and advertising that are used to attract enrollments. 11 Other control variables in the regressions behave as expected. Specific benefit coverages are generally positive and significant and the quit rate has a negative coefficient. The interaction terms are insignificant with the exception of the drug age interaction in the existing employees regression. Finally, the plan type dummies are also significant. We speculate that these dummies capture unmeasured plan attributes as well as preferences for model types per se Results for 1996 and tests of hypotheses Tables 5 and 6 provide the regressions of plan choice for 1996, after the full dissemination of report card scores. Table 5 reports the results for new hires while Table 6 reports the results for existing federal employees. For both specifications, we present both the 1996 coefficient, as well as the difference between the 1995 and 1996 coefficients, together with the estimated S.D. and statistical significance of this difference. 12 Compared to 1995, the greatest coefficient difference exists for report card quality of care. Whereas the effect is negative in the 1995 results, in 1996 the effect is positive and highly significant for both new and existing federal employees. This provides support for hypothesis one, indicating that report card quality of care influences plan selection once report cards are disseminated. In addition, the change is greatest for new federal employees, indicating a greater report card impact for this population. This supports hypothesis two. The magnitude of report card quality s impact on plan choice is also worth noting. The impact of all right side variables is given in Table 7. According to Table 7, a one S.D. increase in the report card measure of quality of care increases the likelihood of plan selection by more than 50%. 13 The 1996 results for report card coverage are somewhat different. The effect of report card coverage is insignificant for new hires but is positive and significant for existing federal employees. Thus, with respect to plan coverage, hypothesis one is verified only for existing federal employees and hypothesis two is refuted. A possible explanation is that existing federal employees use more medical services because they are older on average and thus place a greater weight on plan coverage. The other variable of interest is plan premium. Hypothesis three predicts a greater price elasticity in 1996, conditional on increased information with respect to quality and coverage. Tables 5 and 6 provide support for this hypothesis. The price elasticity is greater in 1996 for both new hires and existing employees. The measured price elasticity rises from 0.14 in 1995 to 1.05 in 1996 for new hires and from 0.02 to 0.13 for existing federal 11 One reviewer suggests, alternatively, that expected out of pocket costs may be a poor construct for the expected out of pocket expense of most consumers. This is because it is derived actuarially based on the experiences of a representative consumer and most individuals vary by health status and expected service mix. 12 The differences are obtained from a regression in which we combine the 1995 and 1996 data and include interaction terms for the product of year dummies and each variable. 13 Given that report cards were already distributed to 25% of employees in 1995, the 50% figure may represent a lower bound on the actual effects of distributing report cards.

13 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002)

14 1044 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002)

15 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) Table 7 Odds ratios Variable Odds ratio 1995 new hire 1995 old hire 1996 new hire 1996 old hire Quality score Coverage score Premiums OOP Mental health Drugs Dental Quit rate Staff Group Network Mixed This table presents odds ratios for the variables in four complete specifications. In each case, the odds ratio is defined by computing the probability of plan choice with the relevant variable set equal to its mean plus one sample S.D., relative to the likelihood of choice with the relevant variable set equal to its mean. All other variables in the specification are set equal to the sample mean. employees. 14 These results support the hypothesis that informed consumers demands are more price elastic. We conclude that the main hypotheses set out in Section 3 are supported, with the exception of hypothesis two applied to report card coverage. Among the remaining variables, the results provide additional support for our main hypotheses. The variable for out of pocket expense remains positive, albeit with a smaller coefficient. Thus the market remained uninformed with respect to out of pocket expense in This is expected, because plan report cards provide no information on expected out of pocket expense. The other noteworthy changes are reduced coefficients for specific plan coverages and the measured quit rate. We speculate that where consumers are more informed with respect to quality of care and coverage measures, consumers reduce their reliance on surrogate measures of expected utility A robustness check using 1996 report card values for 1995 plan choices The results reported above provide support for our three hypotheses. However, the relatively poor performance of the 1995 models and the negative findings for quality of care in 1995 are disturbing. To test the robustness of our findings, we re-estimate the 1995 models using the 1996 values of the report card measures. 15 If consumers are truly uninformed 14 The elasticity is computed by averaging the fitted impact of a 1% change in prices across the entire data set and measuring its impact on the log probability of choice. All observations in the sample are weighted equally for these purposes. The individual elasticities vary according to the number of choices. For example, in the new hires sample, where there are five choices the elasticities are 0.25 and 0.03 in 1996 and 1995, respectively. Where there are only two choices the elasticities are and in 1996 and 1995, respectively. 15 We also examine the pair wise correlation of report card measures across years. The pair wise correlation across years for the quality score is The pair wise correlation for the coverage score is 0.54.

16 1046 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) about quality of care in 1995, as we hypothesize, then the coefficient for quality in the 1995 regression should remain negative or statistically insignificant even using the 1996 values of quality. Our results show this to be the case, differing little from those reported in Table Conclusion There have been few efforts to assess the effects of subjective report card measures. Feldman et al. (1989) found that subjectively reported waiting times influenced plan choice in the Twin Cities. On the other hand, Chernew and Scanlon (1998) did not find that subjective plan measures influenced plan choice in any regular, sensible way. Our work differs from that of Chernew and Scanlon in that we significantly reduce the number of report card measures in our regressions. In the process, we obtain significant results. Our results suggest that making report cards available influences consumer choice. However, it doesn t necessarily follow that report cards improve market outcomes unless it can also be verified that report cards provide valid measures of quality of care. Significantly, we also find that many report card measures are highly correlated. 17 This may be an asset from the consumer perspective. Consumers can apparently get a complete picture of most subjective dimensions of quality by focusing on just a few report care measures (two in our case). Providing that these few measures are valid indicators of quality, reporting them may have a significant impact on market efficiency. Our data and results are subject to some limitations. Our data are for a selected set of counties where the number of plan choices is reasonably small. Thus, our results cannot be generalized to the entire FEHBP. Still, the limited number of choices may be a good representation of what may occur in large firms in the private sector. Future work may be aimed at validating our findings on a larger sample of new hires. One difficulty in doing so will be replicating the natural experiment exploited in this work. Most importantly, it would be useful to test the validity of report card measures. One way to do this would be through a study of plan level quit rates in relation to lagged report card measures. Acknowledgements We are grateful to Mike Chernew, Tom McGuire, Tom Rice, Larry Van Horn and two anonymous referees for helpful comments on a previous draft. All remaining errors are our own. Wedig received financial support from the Indiana Hospital and Health Association. 16 These results are available on request from the authors. In particular, the fitted effect of quality remains negative in all four reported specifications. The values of other coefficients are also similar to the values reported in Table 4. These results are consistent with the hypothesis that the market was relatively uninformed about quality of care in 1995, regardless of the measure of report card quality used. 17 Specifically, we measure partial correlations between report card quality of care and several different report card measures in the Checkbook Guide to Health Insurance Plans for Federal Employees, including coverage, access, choice of PCP, physician interest in patient, time with physician, preventive care and overall results of care. We find partial correlations in the range of with report card quality for every measure.

17 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) References Barringer, M., Mitchell, O., Workers preferences among company-provided health plans. Industrial and Labor Relations Review 48, Beaulieu, N.D., Quality information and consumer health plan choices. Journal of Health Economics 21, Buchmueller, T., Feldstein, P., Consumers sensitivity to health plan premiums: evidence from a natural experiment in California. Health Affairs 15, Chernew, M., Scanlon, D., Health plan report cards and insurance choice. Inquiry 35, Edgman-Levitan, S., Cleary, P., What information do consumers want and need? Health Affairs 15, Enthoven, A., Consumer choice health plan. New England Journal of Medicine 12/13, and Farley, D., Short, P., Elliot, M.N., Kanouse, D.E., Brown, J.A., Hays, R.D., Effects of CAHPS health plan performance information on plan choices by New Jersey Medicaid beneficiaries, Health Services Research, in press. Feldman, R., Finch, M., Dowd, B., Cassou, S., The demand for employment-based health insurance plans. The Journal of Human Resources 24, Francis, W., Checkbook s Guide to Health Insurance Plans for Federal Employees, Center for the Study of Services, Washington, DC. Francis, W., Checkbook s Guide to Health Insurance Plans for Federal Employees, Center for the Study of Services, Washington, DC. Harris, K., Schultz, J., Feldman, R., Measuring consumer perceptions of quality differences among competing health benefit plans. Journal of Health Economics 21, Hibbard, J., Jewett, J., What type of quality information do consumers want in a health care report card? Medical Care Research and Review 53, Hibbard, J., Jewett, J., Will quality report cards help consumers? Health Affairs 16, Holmer, M., Tax policy and the demand for health insurance. Journal of Health Economics 3, Juba, D., Lave, J., Shaddy, J., An analysis of the choice of health benefits plans. Inquiry 17, Long, S., Settle, R.F., Wrightson, C.W., Employee premiums, availability of alternative plans and HMO disenrollment. Medical Care 26, McCormack, L., Garfinkel, S., Schnainer, J., et al., Consumer information development and use. Health Care Financing Review 18, McFadden, D., Modelling the choice of residential location. In: Karlquist, A., et al. (Eds.), Spatial Interaction Theory and Residential Location, North-Holland, Amsterdam, pp McGuire, T., Price and membership in a prepaid group medical practice. Medical Care 19, Royalty, A., Solomon, N., Health plan choice: price elasticities in a managed care setting. The Journal of Human Resources 34, Sainfort, S., Booske, B., Role of information in consumer selection of health plans. Health Care Financing Review 18, Scanlon, D., Chernew, M., Lave, J., Consumer health plan choice: current knowledge and future directions. Annual Review of Public Health 18, Scanlon, D., Chernew, M., McLaughlin, C., Solon, G., The impact of health plan report cards on managed care enrollment. Journal of Health Economics 2002, Short, P., Taylor, A., Premiums, benefits and employee choice of health insurance options. Journal of Health Economics 8, Spranca, M., Kanouse, D., Elliott, M., Short, P.F., Farley, D., Hays, R., Do consumer reports of health plan quality affect health plan selection? Health Services Research 35, Tumlinson, A., Bottigheimer, H., Mahoney, P., Stone, E., Hendricks, A., Choosing a health plan: what information will consumers use? Health Affairs 16, United States Office of Personnel Management, FEHB Guide: 1995 open season for federal civilian employees. US OPM, Washington, DC. United States Office of Personnel Management, 1994b. How members rate their health plans, 1994 consumer satisfaction survey results. US OPM, Washington, DC.

18 1048 G.J. Wedig, M. Tai-Seale / Journal of Health Economics 21 (2002) United States Office of Personnel Management, FEHB Guide: 1996 open season for federal civilian employees. US OPM, Washington, DC. United States Office of Personnel Management, 1995b. How members rate their health plans, 1995 consumer satisfaction survey results. US OPM, Washington, DC. Welch, W.P., The elasticity of demand for health maintenance organizations. Journal of Human Resources 21,