Accuracy of Medicare Expenditures in the Medical Expenditure Panel Survey

Transcription

1 Samuel H. Zuvekas Gary L. Olin Accuracy of Medicare Expenditures in the Medical Expenditure Panel Survey This paper examines underreporting and underrepresentation of high expenditure cases in the Medical Expenditure Panel Survey (MEPS) and their implications for analyses. Our data come from a sample of Medicare beneficiaries in the MEPS who were matched to their Medicare claims and enrollment files, with supplemental data from the Medicare Current Beneficiary Survey (MCBS). Underreporting of expenditures affected all groups of Medicare beneficiaries in the matched sample, but uniformly so that behavioral analyses were largely unaffected. Straightforward adjustments to the MEPS expenditure estimates could align them with aggregate sources, such as the National Health Expenditure Accounts, while preserving underlying relationships between expenditures and key correlates. The Medical Expenditure Panel Survey (MEPS) is a widely used national resource for conducting descriptive, behavioral, and simulation analyses and informing health care policy. Conducted annually since 1996, the MEPS is the only source of health care data that combines detailed information about health care spending with individualand family-level characteristics for the U.S. community population as a whole. The survey was instrumental, for example, in research and policy analyses leading to enactment of the Medicare Part D prescription drug benefit, evaluations of the State Children s Health Insurance Program (SCHIP), and state health care reform initiatives, including those in California. The MEPS is also central to the recent effort to construct a national health account that combines expenditures and outcomes (Rosen and Cutler 2007; Cutler, Rosen, and Vijan 2006). However, household surveys such as the MEPS are not without weaknesses. A common finding in fields as diverse as employment and crime is that estimates based on survey data do not match aggregate data. Health care expenditures are no different. In detailed comparisons of the MEPS to the National Health Expenditure Accounts (NHEA), produced by the Centers for Medicare and Medicaid Services (CMS), Sing et al. (2006) found an estimated gap in expenditures of 14% between the MEPS and NHEA. This gap exists even after adjustments for differences in the populations and health care services included in the two sources. Many analysts have noted differences between the MEPS and NHEA and use various means of adjusting the MEPS so that individual- or Samuel H. Zuvekas, Ph.D., is an economist at the Center for Financing, Access and Cost Trends at the Agency for Healthcare Research and Quality (AHRQ). Gary L. Olin, Ph.D., is an economist who recently retired from the Center for Financing, Access and Cost Trends. Address correspondence to Dr. Zuvekas at the Agency for Healthcare Research and Quality, 540 Gaither Road, Rockville, MD Samuel.zuvekas@ahrq.hhs.gov 92 Inquiry 46: (Spring 2009) Excellus Health Plan, Inc /09/

2 Medicare Expenditures in the MEPS family-level analyses may be conducted while still aligning with NHEA expenditure totals. The primary factors driving the 14% gap between the MEPS and NHEA are underreporting of health care utilization by survey respondents and underrepresentation of high expenditure cases in the MEPS. Householdreported health care use is the basis for national estimates of health care expenditures in the MEPS. The survey asks households about each of their hospital stays, hospital outpatient, emergency department, and office visits, prescription drugs, home health care use, and other health care services and supplies during the year. Additional information about spending for health care events reported by the household are collected in follow-back surveys as part of the Medical Provider Component (MEPS-MPC) and Pharmacy Component (MEPS-PC). However, data from providers and pharmacies in the MEPS are used solely to supplement or replace payment information collected from the household. Quantities come strictly from the household, but the reliance on householdreported utilization as the basic building blocks of MEPS total expenditure estimates raises concerns about their accuracy. A recent review of 42 studies evaluating the accuracy of self-reported health care utilization data, for example, found that the most common problem is underreporting (Bhandari and Wagner 2006). It is also believed that people with high health care expenditures may be underrepresented in the MEPS. Actuaries and others comparing the MEPS to nonrepresentative claims data have reported the relative absence of cases above $25,000 and especially of very high expenditure cases. Comparisons of the MEPS to MarketScan claims data, which are largely from firms in the Fortune 200, also support this finding (as reported in Sing et al. 2006). Some MEPS data users have expressed reluctance to use MEPS data because of this problem. Household underreporting (or overreporting in some cases) and underrepresentation of high expenditure cases are not the only potential source of error in MEPS expenditure estimates. Households report their own out-of-pocket payments fairly well (Machlin et al. 1999), but may not know third-party payments at all or report them inaccurately because of confusion about discounts, adjustments, and contractual arrangements between providers and third-party payers. This is the main rationale for the follow-back surveys of providers (MEPS-MPC) and pharmacies (MEPS-PC). However, provider and pharmacy payment data are available for only a subset of health care services reported by households, and also may be subject to error in the data collection process. Either householdreported payment data or imputation must be used for the remainder of household-reported events (see MEPS Public Use File Documentation for event files HC-094A through HC- 095H for information on current MEPS imputation procedures). Imputation unavoidably introduces random error in MEPS expenditure data, which only reduces precision when expenditures are used as the dependent variable. Moreover, nonrandom error the more serious concern for estimation purposes is also possible. We seek to better understand the accuracy and validity of expenditure data reported in the MEPS public use files and the potential implications for their use in descriptive, behavioral, and simulation analyses. While reasons for differences between the MEPS and other sources have been identified, quantifying them is another matter. The NHEA aggregate expenditures are widely considered gold standard benchmarks, but the adjustments for services and populations needed to make comparisons with the MEPS involve considerable uncertainty. Moreover, regardless of the accuracy of the adjustments, these comparisons cannot tell us why there are gaps. Previous benchmark comparisons to claims and other administrative data also have been hampered by difficulties in obtaining truly comparable populations and definitions of service use. We use data from a sample of MEPS Medicare beneficiaries for whom we obtained complete Medicare claims and enrollment data for 2001 through 2003 to investigate how underreporting by households and other potential sources of error in the data collection and estimation process affect MEPS expenditure estimates. We quantify how 93

3 Inquiry/Volume 46, Spring 2009 reporting varies by population subgroups and the extent to which reporting variation affects descriptive and behavioral analyses in our matched sample. We further assess the potential impact of misreporting in behavioral analyses by estimating a standard model of health care expenditures. Then, to assess the impact of potential underrepresentation of people with high expenditures in the MEPS, we use expenditures from the Medicare Current Beneficiary Survey to decompose the expenditure shortfalls that are due to general underreporting of health care use versus underrepresentation of high expenditure cases. We conclude with a discussion of methods for adjusting or aligning MEPS expenditure data based on the findings with the matched MEPS sample and comparisons to the MCBS. Methods MEPS-Medicare Claims Matched Sample The MEPS has a rotating panel design with two overlapping cohorts (Cohen 2003; Cohen, Monheit, and Cohen 1996/1997; J. Cohen 1997; S. Cohen 1997). A new cohort (panel) is initiated each year and interviewed five times to collect two calendar years of data. We pooled data for calendar years 2001 through 2003, and initially restricted the sample to people covered by Medicare at any point during a year. This sample included 9,015 people, or 13,680 person-year observations since some of the beneficiaries were in the survey for two years. Under a data use agreement with the CMS, we initially matched MEPS respondents who provided their Medicare health insurance claim number (HICN) or Social Security number (SSN) to Medicare enrollment and claims data. Of the 3,788 beneficiaries providing a complete HICN or SSN in the years MEPS, 91% (3,463 unique individuals or 5,376 person-year observations) matched exactly to the same HICN or SSN, sex, and date of birth in the Medicare administrative records. These exact matches represent 39% of all eligible Medicare beneficiaries in the MEPS for the years A logistic regression found that these matched individuals were more likely to be the household informant (self-respondent), to live in the Midwest or South compared to the West and East regions, to reside in a non- Metropolitan Statistical Area (MSA), to report their race as white compared to nonwhite, and to be at least age 65, compared to those who did not match or to Medicare beneficiaries who did not provide their HICNs or SSNs. Whether adjusting for these differences or not, we found no statistically significant differences between mean Medicare expenditures reported in the MEPS for the 5,376 person-year observations matched to Medicare administrative records and the mean Medicare expenditures reported in the MEPS for the remaining 8,304 person-year observations with Medicare coverage in the MEPS. We restricted the initial matched sample to survey respondents who were in-scope the entire calendar year, did not die, and were not institutionalized as defined in the MEPS (that is, the person did not enter a prison, nursing home, assisted living facility, residential treatment facility, or other institution), leaving 5,169 person-year observations. In part, this was to ensure that our final analytic matched sample contained complete Medicare expenditure data from the MEPS (which does not collect utilization and expenditure data for the period that respondents are outof-scope for the survey) and the CMS claims. We also excluded decedents and beneficiaries who entered institutions because they were underrepresented in our matched sample to a high degree and because we lacked sufficient power to conduct analyses of these cases. Because claims data are unavailable for Medicare beneficiaries enrolled in managed care plans, we further restricted the matched sample to beneficiaries with Part A and Part B Medicare fee-for-service coverage for the entire calendar year based on their monthly Medicare enrollment data. These restrictions allowed us to compare expenditures for survey respondents who were asked about their health care for the entire year and also had Medicare claims for all their covered services regardless of what they reported in the survey. After all restrictions, the final analytic matched sample contained 4,045 person-year observations (2,649 unique individuals) and was the sample 94

4 Medicare Expenditures in the MEPS used for all subsequent analyses described and reported in this paper. Medicare claims for physician and hospital care were the benchmark in our comparisons with the MEPS expenditure data. Expenditure measures. We focus on Medicare expenditures in our comparisons for two reasons. First, the CMS administrative records contain complete data on just the services covered by the traditional fee-forservice Medicare program. Noncovered services, such as certain types of elective surgeries, prescription drugs (prior to 2006), and services provided at Veterans Administration or military hospitals are not included in the Medicare claims even if reported by MEPS respondents. Second, the Medicare claims records contain exact payments by Medicare but only estimates for other payers, including the patient. We first constructed a measure of total Medicare expenditures on hospital and officebased physician and nonphysician services from the MEPS public use full-year consolidated files for calendar years 2001 through We then computed total Medicare expenditures from the claims for each person in our analytic sample using the Inpatient, Outpatient (including emergency department visits not included in the inpatient file), and Carrier (physician/supplier) Standard Analytic Files (SAF) for the years We subtracted out claims for separately billed laboratory (SBL) services identified in the Carrier SAF because the MEPS does not have a standard protocol for capturing SBL services. For example, a patient may have blood drawn at a physician s office that is sent for testing to an independent laboratory, which then separately bills the patient. In addition, we did not use in the comparisons durable medical equipment and supplies (DME) covered by Medicare, such as wheelchairs, beds, and oxygen supplies, because the MEPS generally does not capture these expenditures. Medicare covers DME services under the Part B benefit but they are separated from other Part B claims in the Carrier SAF and appear in their own SAF. Thus, we simply summed Medicare payments in the Inpatient, Outpatient, and Carrier SAFs (less SBL services) to construct the benchmark Medicare expenditures on hospital and office-based services for each person. In order to pool the expenditure data for calendar years , we inflated expenditures for 2001 and 2002 to constant 2003 dollars using the personal health care expenditures (PHCE) price index developed by the CMS, Office of the Actuary (2006), to account for medical price inflation. Additional details about the construction of the Medicare expenditure variable from the Standard Analytic Files are available online (Olin et al. 2008). Control variables. We created the following variables from the MEPS data on socioeconomic characteristics of each household member. Age is categorized as under 65, 65 to 74, 75 to 84, and 85 and older. Binary indicators represent the following categories: female, nonwhite including Hispanics, married, region (North, South, Midwest, and West), and urban (living in an MSA). Family income is coded as below the federal poverty line (FPL), 100% to 199% of the FPL, and 200% or more than the FPL. Education is categorized as less than 12 years, 12 years, and more than 12 years. Binary indicators represent the five categories of perceived health: excellent, very good, good, fair, or poor. A cognitive limitation indicator was coded 1 for people who experienced confusion or memory loss, had problems making decisions, or required supervision for their own safety. An activity limitation indicator was coded 1 if the person had limited ability to work in a job, do housework, or go to school because of impairment or physical or mental health problem. We also included binary indicators taken from the MEPS for private insurance and Medicaid coverage, respectively, representing coverage at the time of the first MEPS interview for the calendar year. We included several binary indicators to describe the types of interviews that could affect the quality of utilization and expenditure data reported by household respondents. Interview language was coded 1 if any of the interviews were in a language other than English (2% of the analysis sample). A code of 1 for nonresident proxy interview indicates that someone outside the household completed the interview (1%). A code of 1 95

5 Inquiry/Volume 46, Spring 2009 for self-respondent indicates that the sample person was the household informant in all of the interviews (68%). Finally, the survey year for the individual was coded 0 for the first year, and 1 for the second year in the MEPS survey. Weighted means for the sociodemographic and interview characteristics are reported in Table 1. MEPS-Medicare Claims Analyses We first compared univariate means of Medicare expenditures reported in the MEPS to the actual Medicare expenditures in the CMS claims, as well as the ratios of the means. We also calculated Lin s concordance statistic, a standard measure of agreement between two continuous variables (Lin 1989). Lin s statistic is scaled from 21 (perfect disagreement) to 1 (perfect agreement). We also conducted bivariate analyses of the expenditure means and ratios and the agreement and concordance statistics by sociodemographic and interview characteristics. We next investigated whether reporting and estimation errors in the MEPS lead to systematic biases in behavioral analyses by estimating a pair of expenditure regressions using, respectively, the MEPS and claimsbased Medicare expenditures as the dependent variable. Each regression included an identical set of socioeconomic characteristics typically included in health care demand models. We used a simple ordinary least squares (OLS) regression specification to model health expenditures for the Medicare population. That is, each person s actual Medicare expenditure (not logged or otherwise transformed) was entered on the lefthand side. Previous analyses using data from the MEPS (Hill and Miller 2007) have found that in the elderly Medicare population, the one-part OLS model outperforms traditional two-part log models, variants of one- and two-part generalized linear models (GLM) (Manning and Mullahy 2001), and generalized gamma models (GGM) (Manning, Basu and Mullahy 2005), and is at least as good if not better than extended estimating equations (EEE) models (Basu and Rathouz 2005). Using data from the 1996 MCBS, Buntin and Zaslavsky (2004) also found the one-part OLS model performs well compared to alternatives (although no one model dominated in their analyses). The one-part OLS model has the added attraction that the coefficient estimates are directly interpretable as incremental changes in Medicare expenditures. All analyses used MEPS full-year personlevel sampling weights, which take disproportionate sampling and nonresponse into account. Standard errors were adjusted for the complex sampling design of the MEPS using Stata 9.2. The method of balanced repeated replications (BRR) was used for all tests involving Lin s concordance statistic (see MEPS Public Use File HC-036BRR documentation). All other test statistics, including between-model comparisons, used the Taylor Series approach. Both the BRR and Taylor Series methods also corrected for repeated observations of individuals (Williams 2000). MEPS-MCBS Comparisons We further compared Medicare expenditures from the MEPS analytic matched sample (both the MEPS reported Medicare expenditures and actual Medicare expenditures recorded in the CMS claims for the matched sample) to Medicare expenditures in the MCBS for calendar years We restricted the MCBS sample to people living in the community all year (defined in the MCBS as not residing in an institution at any of the three interview rounds for the calendar year) who did not die. To match our MEPS analytic sample, we also restricted the MCBS sample to people with traditional Medicare coverage for all 12 months of each year they were in the survey. We note that in the MCBS, the estimates of expenditures for the population enrolled in traditional Medicare are derived from actual claims data for the sampled beneficiaries. Our measure of Medicare expenditures from CMS claims records in the MEPS analytic matched sample was comparably constructed. Sample sizes for our MCBS analytic sample were 8,548, 8,698, and 8,731 for calendar years 2001, 2002, and 2003, respectively, or a total of 25,977 cases in the pooled analyses. We calculated Medicare expenditures in the MCBS as the sum of the inpatient hospital, outpatient hospital, and physician/supplier 96

6 Medicare Expenditures in the MEPS Table 1. Comparison of Medicare expenditures reported in the MEPS and Medicare claims, by personal characteristics, pooled analytic matched sample Mean sample Mean expenditure ($) characteristic MEPS Medicare claims MEPS/Claims ratio Concordance Overall 3,788 4, Age, ,607 5, ,240 3,771*** ,168 4,738*** ,845 3, Race/ethnicity White.86 3,893 4,263***.91 ###.86 Nonwhite.14 3,147 4,494*** Sex Male.45 3,869 4,356*** Female.55 3,723 4,246*** Marital status Not married.47 4,003 4,618*** Married.53 3,595 4,005*** Region Northeast.19 3,877 4,793***.81 ##.86 Midwest.27 4,418 4,898*** South.41 3,547 3, West.14 3,125 4,072*** MSA status Non-MSA.30 3,819 4,124**.93.93ˆˆˆ MSA.70 3,775 4,367*** Family income,100% FPL.13 4,271 5,459***.78 ### % FPL.28 3,479 3,976*** $ 200% FPL.59 3,831 4,194*** Education,12 years.32 3,658 4,460***.82 # years.35 3,767 4,138** years.33 3,954 4,300* Private insurance No.43 3,959 4,734***.84 ##.85 Yes.57 3,657 3,958** Medicaid No.90 3,730 4,159***.90 ##.86 Yes.10 4,305 5,516** Self-perceived health status Excellent.16 2,246 2, Very good.25 2,773 2,982* Good.31 3,543 4,066*** Fair.19 4,965 5,957*** Poor.09 7,916 8,968** Cognitive limitation No.89 3,644 4,194*** Yes.11 4,905 5, Activity limitation No.71 3,187 3,689***.86.81ˆˆ Yes.29 5,230 5,750*** Interview language English.98 3,794 4,253***.89 ###.86 Non-English.02 3,543 6,150** Nonresident proxy No.99 3,744 4,200***.89 ###.85 Yes.01 6,937 11,131***

7 Inquiry/Volume 46, Spring 2009 Table 1. (continued) Mean sample Mean expenditure ($) characteristic MEPS Medicare claims MEPS/Claims ratio Concordance Self-respondent No.32 3,948 4,429*** Yes.68 3,715 4,233*** Year in survey First.54 3,653 3,953***.92 ##.88 Second.46 3,945 4,692*** Note: All estimates are weighted using MEPS full-year sample weights. All statistical tests adjust for the complex survey design of the MEPS. * p,.10; ** p,.05; *** p,.01 for difference in means between CMS and MEPS. # p,.10; ## p,.05; ## p,.01 for difference in ratio by group characteristic. ˆ p,.10;ˆˆˆ p,.05;ˆˆˆ p,.01 for difference in concordance by group characteristic. services from the MCBS event-type files (RICs IPE, OPE, and MPE, respectively). Physician/supplier services in the MCBS include DME and SBL services, which we excluded from our comparisons between the MEPS Public Use Files and Medicare claims for our matched analytic sample. Because the MCBS event files are organized differently from the Medicare claims SAFs, we were not able to pull these DME and SBL services out of the MCBS estimates of Medicare expenditures. Instead, we used the DME and carrier SAF to add DME and SBL expenditures to the estimates of Medicare hospital and officebased expenditures for our matched analytic sample to facilitate comparisons between our matched sample and the MCBS. We used these measures to compare mean Medicare expenditures reported in the MEPS Public Use Files and in the Medicare claims for our matched analytic sample to the MCBS estimates, as well as their distributions. Included were comparisons of the concentration of expenditures. We further decomposed the differences between the MEPS and MCBS expenditure estimates that are explained by the underreporting of events in the MEPS versus the absence of high expenditure cases. Results MEPS-Medicare Claims Matched Sample Mean Medicare expenditures for hospital and office-based care reported in the MEPS were $3,788, compared to $4,295 in the claims for the analytic matched sample (Table 1), a difference of 12%. As measured by Lin s statistic, overall concordance between the MEPS and CMS reported expenditures at the individual level was high.88 on a scale from -1 to 1. The high overall concordance reflects the fact that, while Medicare expenditures reported in the MEPS were lower than in the claims, they were systematically lower across individuals and population subgroups. However, while underreporting in the MEPS for the matched sample affected all population groups (with only a few minor exceptions), there were important differences in the extent of this underreporting (Table 1). Survey underreporting was relatively greater for nonwhites compared to whites in the analytic matched sample. Seventy percent of actual Medicare expenditures (as measured by the claims) were reported by nonwhites compared to 91% for whites in the MEPS (Table 1). Probit regressions on the probability of underreporting, controlling for Medicaid status and other characteristics contained in Table 1 (not shown), similarly found underreporting by nonwhites compared to whites. However, mean Medicare expenditures for nonwhites were actually slightly higher (but not statistically different) than for whites when claims were compared. In other words, there was no disparity in Medicare expenditures using claims even though the mean expenditure was $650 lower in the MEPS ($3,147) for nonwhite Medicare beneficiaries compared to white beneficiaries ($3,893). The Lin s concordance statistic was 98

8 Medicare Expenditures in the MEPS also lower for nonwhites but did not have a statistically significant difference from whites. Somewhat surprisingly, there was significant geographic variation in the reporting of Medicare expenditures in the analytic matched sample. A relatively greater proportion of expenditures were reported in the MEPS for people living in the South (.95) and Midwest (.90) compared to the Northeast (.81) and West (.77). Income and education also played a role, with relatively better reporting for those with more income and higher educational levels (12 years or more). However, differences in education were not statistically significant in multivariate analyses (not shown). Reporting was also higher for beneficiaries with private insurance and lower for those with Medicaid. Although there was some variation in reporting by health status, these differences were generally not statistically significant. There were also substantial reporting differences by type of interview obtained (Table 1). MEPS-reported expenditures were 58% of claims for persons in households where interviews were conducted in a language other than English, and 62% for interviews conducted with nonresident proxies (differences were not statistically significant in multivariate analyses, not shown). There were no differences in the quality of reporting for Medicare beneficiaries who responded for themselves compared to beneficiaries who had another household member respond for them, possibly because MEPS interviewers are directed to conduct interviews with the person most knowledgeable about health and health care use in the household. However, mean reporting was lower for those in their second year of the MEPS (.84) compared to those in their first year (.92). We used the side-by side regressions to assess the extent to which differences in reporting by population subgroups affect behavioral models of health care expenditures. Table 2 presents results of these estimates for a representative health care demand model, with the MEPS-based estimates of Medicare expenditures in the first set of columns and the claims-based estimates in the second set. Again, the OLS coefficient estimates are directly interpretable in real dollar terms. Both regressions showed the same general pattern with respect to the determinants of health care expenditures. In particular, health expenditures were strongly monotonic with respect to perceived health status, although the MEPS coefficient estimates were uniformly lower by 18% to 25%. For example, with the claims-based (MEPSbased) expenditure measure, those in fair health had $4,116 ($3,108) higher Medicare expenditures than those in excellent health while those in poor health had $7,621 ($6,251) higher expenditures, all else equal. However, there were two notable differences between the regressions. First, the magnitude of the coefficient for the South census region (relative to the Northeast region) was more than twice as large in the claims regression ($1,274 vs. $516). Second, and more importantly, the MEPS-based regression implies that, all else equal, nonwhites had $890 lower Medicare expenditures than whites in the analytic matched sample; however, in the claims-based regression, the nonwhite coefficient estimate was smaller in magnitude (2$260), although still the same sign, and not statistically significant. MEPS-MCBS Comparisons The comparisons of the MEPS and claimsbased expenditures capture underreporting and how it varies in the analytic matched sample, but cannot address the absence of some high expenditure cases in the MEPS. For this, we turn to the results from the MEPS-MCBS comparisons. Claims-based mean expenditures drawn from linked claims for the full-year community fee-for-service MCBS population were $5,100, compared to $3,788 in the MEPS (Table 3), a sizable difference. However, much of this gap is explained by the absence of most DME and SBL expenditures in the MEPS. DME and SBL expenditures are technically in-scope for MEPS, but the survey is not designed to capture them fully. When we included other medical expenditures from the MEPS public use files, the mean MEPS expenditure increased only $31 to $3,819. However, the claims-based measure for the MEPS sample increased $351 from $4,295 to 99

9 Inquiry/Volume 46, Spring 2009 Table 2. Comparison of OLS Medicare expenditure regressions using MEPS and claims-based expenditure measures, pooled MEPS Medicare claims Difference Coefficient S.E. Coefficient S.E. p-value Age , Age , ** 1, **.365 Age 85+ 1, , Nonwhite ** Female Married Midwest South * 21, **.009 West Urban % 199% FPL *.095 $200% FPL years education years education * Very good health * **.506 Good health 1, *** 2, ***.044 Fair health 3, *** 4, ***.008 Poor health 6,251 1,095*** 7,621 1,254***.033 Cognitive limitation Activity limitation Private insurance Medicaid Constant 969 1,007 1,648 1,290 R-squared Note: All estimates are weighted using MEPS full-year sample weights. Standard errors (S.E.) and statistical tests adjust for the complex survey design of the MEPS. n54,045. * p,.10; ** p,.05; *** p,.01. $4,646 once the DME and SBL claims were added. When we added this estimate of missing DME/SBL services into the MEPS ($351 minus $31), the gap between MEPS and MCBS was $961, or approximately 19% of the MCBS estimate of $5,100 (Table 4). Just over half of this gap (53%) is explained by the underreporting of hospital and ambulatory services in the MEPS, which was calculated directly from Table 3 as the difference in the MEPS Public Use Files estimates and the analytic matched claims estimate ($4,295 minus $3,788). The remaining 47% of the expenditure gap between the MEPS analytic matched sample and the comparable MCBS sample is likely due to an insufficient number of high expenditure cases in the analytic matched MEPS sample. To see the underrepresentation of high expenditure cases more clearly, we graph the cumulative distribution of Medicare expenditures in Figure 1 (up to $25,000) and Figure 2 ($25,000 and above). Medicare expenditures (including DME/SBL expenditures to facilitate comparisons with the MCBS sample) are on the X-axis and the cumulative (weighted) percentages of the respective samples are on the Y-axis. The MEPS Public Use Files expenditure distribution (shown in the solid black line) is to the left and above the claimsbased expenditure distribution for the analytic matched sample (shown in the dotted gray line). This is the direct result of the underreporting of ambulatory care, DME, and SBL services in the MEPS. A simple adjustment of multiplying MEPS-reported Medicare expenditures by 1.09 for people in the analytic matched sample with inpatient Medicare expenditures and by for those without inpatient expenditures nearly perfectly replicates the claims-based expenditure distribution (including DME/SBL expenditures). 100

10 Medicare Expenditures in the MEPS Table 3. Mean Medicare expenditures on hospital and physician/supplier services: comparison of MCBS to MEPS-Medicare claims matched sample, full-year community fee-for-service population, Expenditures for matched MEPS sample ($) Excluding DME and SBLs Including DME and SBLs Expenditures for MEPS a CLAIMS b MEPS c CLAIMS d MCBS ($) e ,503 4,288 4, ,484 4,282 4, ,066 4,874 5, f 3,788 4,295 3,819 4,646 5,100 Notes: All estimates are weighted using MEPS full-year sample weights. Standard errors and statistical tests adjust for the complex survey design of the MEPS. Sample includes Medicare population residing in the community all three interview rounds in the MCBS and MEPS, respectively, having Medicare FFS in all 12 months, and not deceased. The MEPS and MCBS samples include individuals with short-term skilled nursing facility and other subacute care stays (for example, rehabilitation hospitals). a Calculated as the sum of opfmcryy, erfmcryy, ipfmcryy, opdmcryy, erdmcryy, ipdmcryy, and obvmcryy where yy501, 02, and 03 from the respective full-year consolidated MEPS files. b Calculated as the sum of Medicare expenditures from the following Medicare Standard Analytic Files: Inpatient Hospital file, Outpatient Hospital file, and Carrier file (excluding separately billing labs). c Calculated as the sum of opfmcryy, erfmcryy, ipfmcryy, opdmcryy, erdmcryy, ipdmcryy, obvmcryy, and othmcryy where yy501, 02, and 03 from the respective full-year consolidated MEPS files. d Calculated as the sum of Medicare expenditures from the following Medicare Standard Analytic Files: Inpatient Hospital file, Outpatient Hospital file, Carrier file (including separately billing labs), and Durable Medical Equipment File. e Calculated as the sum of Inpatient Hospital, Outpatient Hospital, and Physician/Supplier services from the person-level MCBS files. f Three-year average expenditure in 2003 constant dollars using the CMS Personal Health Care Expenditure price index. Table 4. Breakdown of gap between MEPS Public Use File and MCBS estimated Medicare expenditures Explanation Underreporting of hospital and physician services Residual: differences in definitions of community population ; survey sampling variation, nonresponse, and attrition (MEPS/NHIS b, MCBS) Percent of gap explained Percent of expenditure difference Total Adjustment factors to MEPS that eliminated differences between MEPS and MCBS Multiply hospital and physician service Medicare expenditures by 1.09 if beneficiary had inpatient hospital stay a 2. Multiply hospital and physician service Medicare expenditures by if beneficiary did not have hospital stay a Multiply person-level weight of those with Medicare expenditures on hospital and physician services $$25,000 by Multiply person-level weight of those with Medicare expenditures,$25,000 by.9884 (number sufficient to keep total Medicare population constant after reweighting) Note: All estimates are weighted using the MEPS full-year sample weights. Standard errors and statistical tests adjust for the complex survey design of the MEPS. a Also adjusts for nonreporting of durable medical equipment and separately billed lab services in the MEPS. b New panels in the MEPS are drawn from the previous year of the National Health Interview Survey (NHIS). 101

11 Inquiry/Volume 46, Spring 2009 Figure 1. Comparison of cumulative distributions of Medicare expenditures in the MEPS analytic matched sample to the MCBS, (Scope of expenditures includes hospital, physician, and supplier services. All estimates from the analytic matched sample are weighted using MEPS full-year sample weights. The MEPS adjusted distribution was derived by multiplying MEPS-reported expenditures in the analytic matched sample by 1.09 for persons with inpatient Medicare expenditures and for persons without inpatient expenditures to adjust for underreporting of ambulatory, DME, and SBL services in the MEPS. The MCBS distribution of Medicare expenditures is restricted to Medicare beneficiaries with traditional fee-for-service coverage and living in the community all year.) This adjustment was derived by dividing the mean Medicare expenditures reported in the MEPS by the Medicare expenditures in the CMS claims for those with and without inpatient stays. The adjustment shifts the MEPS Public Use Files expenditure distribution (shown in solid black) downward and to the right to the adjusted MEPS expenditures shown with the dotted black line in Figures 1 and 2. The adjusted MEPS expenditures (dotted black line) closely tracks the claimsbased expenditure distribution (dotted gray line) shown in Figures 1 and 2. However, even with the adjustment for underreporting the expenditure distribution (shown with the dotted black line) is shifted slightly upward relative to the MCBS comparison sample (shown with the solid gray line). Below $25,000 (Figure 1), we find negligible differences between the adjusted MEPS expenditure distribution (dotted black line) and the MCBS expenditure distribution (solid gray line); however, the relative absence of people with more than $25,000 in Medicare expenditures (Figure 2) leads to the 9% gap in mean expenditures between the analytic matched MEPS sample (corrected for underreporting) compared to the MCBS sample (Table 4). Because of the high expenditures involved, it takes only a small underrepresentation of high expenditure cases in the MEPS analytic matched sample to generate these differences. For example, 3.6% of the MCBS sample had expenditures between $25,000 and $50,000, while 2.9% of the MEPS analytic matched sample did; 1.2% had 102

12 Medicare Expenditures in the MEPS Figure 2. Expansion of Figure 1 above the 95 th percentile (Scope of expenditures includes hospital, physician, and supplier services. All estimates from the analytic matched sample are weighted using MEPS full-year sample weights. The MEPS adjusted distribution was derived by multiplying MEPS-reported expenditures in the analytic matched sample by 1.09 for persons with inpatient Medicare expenditures and for persons without inpatient expenditures to adjust for underreporting of ambulatory, DME, and SBL services in the MEPS. The MCBS distribution of Medicare expenditures is restricted to Medicare beneficiaries with traditional fee-for-service coverage and living in the community all year.) expenditures above $50,000 in the MCBS sample compared to.8% in the MEPS sample (Figure 2). We also compared the distribution of Medicare expenditures in the MEPS analytic matched sample to the comparable MCBS full-year, community-only fee-for-service sample concentration of expenditures (Table 5). The first three sets of columns were derived from the MEPS matched analytic sample, the fourth and last set of columns from the full-year community fee-for-service population in MCBS. Concentration was measured by first ranking Medicare beneficiaries by their Medicare expenditures and then calculating how much of all Medicare expenditures were accounted for by the top 1%, 2%, 5%, 10%, 20%, and 50%, respectively. For example, in the first two columns the top 1% of Medicare beneficiaries in the analytic matched sample accounted for 17% of Medicare expenditures using the MEPSreported measure of expenditures. In the second set columns, using the CMS claimsbased measure instead (without DME and SBL expenditures added in), the top 1% accounted for 16% of all Medicare expenditures. The concentration among the top 1% further dropped to 15% in the third set of columns, when DME and SBL expenditures were added in from the CMS claims. Thus, we see from comparing within the matched analytic sample that MEPS estimates of concentration (17% for the top 1%) are biased upward from the true concentration of expenditures for the analytic sample measured from the CMS claims (15% for the top 1%) because of both underreporting 103

13 Inquiry/Volume 46, Spring 2009 Table 5. Comparison of estimates of the concentration of Medicare expenditures on hospital and physician supplier services, full-year community fee-for-service population, Population ranked by Medicare expenditures MEPS Matched analytic MEPS sample Claims MCBS Without DME/SBL Without DME/SBL With DME/SBL With DME/SBL Percent of all Medicare expenditures Mean $ Percent of all Medicare expenditures Mean $ Percent of all Percent of all Medicare Medicare expenditures Mean $ expenditures Mean $ Top 1% 17 64, , , ,270 Top 2% 26 50, , , ,505 Top 5% 45 34, , , ,800 Top 10% 63 23, , , ,701 Top 20% 81 15, , , ,008 Top 50% 96 7, , , ,824 Bot 50% Notes: All estimates are weighted using MEPS full-year sample weights. Standard errors and statistical tests adjust for the complex survey design of the MEPS. Sample includes Medicare population residing in the community all three interview rounds in the MCBS and MEPS, respectively, having Medicare FFS in all 12 months, and not deceased. The MEPS and MCBS samples include individuals with short-term skilled nursing facility and other subacute care stays (for example, rehabilitation hospitals). of ambulatory services and the absence of most DME and SBL expenditures. However, at the same time, the absence of some high expenditure cases in the MEPS leads to a downward bias in the estimates of concentration based on the MEPS. We see this in the fourth and final set of columns based on the MCBS; the top 1% accounted for 17% of all Medicare expenditures, compared to 15% in the MEPS analytic matched sample if ambulatory, DME, and SBL services were fully captured, as in the CMS claims. The downward bias from too few high-expenditure cases in the MEPS is offset by the upward bias from underreporting in the MEPS. Thus, the degree of concentration of Medicare expenditures as reported in the MEPS (first two columns) is remarkably similar to the concentration estimated from MCBS (last two columns). Discussion and Conclusions We found an overall gap of 19% between the MEPS estimates of Medicare expenditures in our matched analytic sample and a comparably defined sample of beneficiaries in the MCBS. About half of the gap is due to underreporting and the other half appears due to underrepresentation of high expenditure cases in our matched analytic sample. Underreporting affected all groups of Medicare beneficiaries in this sample and, most importantly, was fairly uniform. As a result, behavioral analyses were largely unaffected, with the notable exception of racial and ethnic differences in expenditures. Estimates of the concentration of expenditures in the matched analytic sample of Medicare beneficiaries also were surprisingly close to those obtained from the MCBS, despite the absence of some high expenditure cases. These results suggest that simple adjustments can be made to MEPS expenditure estimates to align them with aggregate sources, such as the NHEA, while still preserving underlying relationships between expenditures and key correlates, such as health insurance coverage. Our estimated gap in Medicare hospital and physicians expenditures is somewhat larger than that reported by Sing et al. (1996) in their comparisons of the MEPS to the NHEA. In part, this may reflect the greater uncertainty in aligning particular payment sources than total expenditures in aggregate benchmark comparisons. The authors of that study are currently making technical refinements to the methodologies used to align and adjust the MEPS and NHEA, in part based on the analyses 104

14 Medicare Expenditures in the MEPS contained in this study, as well as other information from the MCBS. This is part of a broader effort by MEPS staff to produce an NHEA-aligned set of MEPS data for use by researchers (Selden and Sing 2007). We note several other potential limitations in using the results of our comparisons between the MEPS public use and claimsbased measures of expenditures in our matched analytic sample of Medicare beneficiaries. While we matched a large sample of Medicare beneficiaries in the MEPS to CMS files, the matched analytic sample itself is not nationally representative of Medicare beneficiaries. However, we found no differences in average expenditures in the MEPS between those who matched to Medicare administrative records and those who did not. Similarly, we found no differences in our results when using propensity-score based weights that adjust for observed differences between the matched and unmatched groups. Even so, our findings for Medicare beneficiaries may not generalize to the rest of the U.S. population. Elderly and disabled Medicare beneficiaries use substantially more health care than other Americans (Ezzati-Rice, Kashihara, and Machlin 2004), and previous studies suggest underreporting is greatest among high use groups (Ritter et al. 2001; Wallihan, Stump, and Callahan 1999; Roberts et al. 1996; Cleary and Jette 1984). To this extent, our findings may provide an upperbound estimate of underreporting for the full MEPS sample, but there may be other offsetting factors because the elderly and disabled Medicare populations differ in other important ways from the rest of the population. Our matched sample may also include better reporters on average because it is limited to those people providing a valid Medicare HICN or SSN and includes a higher proportion of self-responders. However, we note that we found no differences in reporting between self-responders and others within the matched sample. We are most concerned about generalizing the results concerning racial and ethnic differences because the racial and ethnic composition of the elderly Medicare population is quite different from that of younger Americans. Finally, we excluded several types of health care services in the analyses presented. Prescription drugs in general, were not covered by Medicare until the Medicare Part D benefit went into effect in 2006, three years following the endpoint for our matched sample. Medicare-covered services in skilled nursing facilities (SNFs) are considered out-of-scope for the MEPS, and therefore were excluded. Medicare-covered home health care services will be the subject of a future analysis with our matched sample. Likewise, there are limitations in using our comparisons of the matched sample to MCBS to understand the underrepresentation of high expenditure cases in the MEPS. Our estimates of this underrepresentation are based on residual calculations between expenditures in the comparably defined MCBS sample and the MEPS public use and claims-based measures in our matched sample. Important factors likely driving this residual calculation are survey nonresponse and attrition. MEPS panels are taken from the previous year s National Health Interview Survey (NHIS), and then interviewed five times to cover two full years in the MEPS. High expenditure cases may be lost due to initial nonresponse to the NHIS or to attrition from either survey over the next three years. Our calculations suggest that a loss of a few dozen higher-expenditure cases at any point out of the thousands of Medicare beneficiaries in the MEPS is enough to lead to underestimates of mean and total expenditures in this population. However, we cannot rule out other factors, such as differences in how the community and institutionalized populations (which are out-of-scope in MEPS) are determined in the MEPS and MCBS. On the face of it, these definitions are virtually identical, but may be implemented differently in subtle ways between the two surveys. Finally, we lack sufficient power with our matched sample to examine the extent to which the expenditures of Medicare beneficiaries who died during the year are underrepresented in the MEPS, and even less power to examine the very small group of high-expenditure beneficiaries who moved between the community and long-term care institutions during the year. 2 Our analyses suggest a three-fold approach for better aligning the MEPS to other sources to improve research and policy analyses 105

15 Inquiry/Volume 46, Spring 2009 where capturing the full distribution of expenditures is important. First, the MEPS should be adjusted to account for services that it does not capture well, especially durable medical equipment. Second, the MEPS should be adjusted to account for underreporting of utilization of hospital and office-based services. We found that a simple adjustment could simultaneously correct for these first two issues, and almost perfectly replicate Medicare claims in our matched sample. Specifically, we multiplied MEPSreported hospital and physician Medicare expenditures by 1.09 for people with inpatient Medicare expenditures and for those without inpatient expenditures (Table 4). Third, the MEPS needs to be adjusted for underrepresentation of high-expenditure people. Significant divergence was observed above $25,000 or the 95% level of Medicare expenditure in our matched sample of Medicare beneficiaries. For those in the MEPS with Medicare expenditures on hospital and physician services above the $25,000 level, we found that a simple adjustment of multiplying person-level weights by a factor of 1.3, and down-weighting those below the $25,000 level by a proportionate amount to keep the total population of Medicare beneficiaries constant (.9884) eliminated the remaining differences between the MEPS and MCBS (see Table 4). However, a more sophisticated nonlinear approach may be needed to better replicate true expenditure distributions in the MEPS. Research using micro-level and aggregate benchmarks will continue to be an area of emphasis with the MEPS. This methodological research serves both to improve collection of raw data in the MEPS and to refine methods for aligning the MEPS to aggregate expenditure data for policy-related analyses. Notes The views expressed in this paper are those of the authors; no official endorsement by the Agency for Healthcare Research and Quality or the Department of Health and Human Services is intended or should be inferred. The authors wish to thank Virendar Kumar, Brian Taiffe, Kitty Williams, Diana Wobus, and Pat Ward of Westat for the preparation of the Medicare claims and MEPS analytic files, and Doris Lefkowitz (AHRQ) for arranging the Data Use Agreement (15816) with the Centers for Medicare and Medicaid Services. Approval to conduct this study was granted by the Westat Institutional Review Board (IRB) on October 11, Office-based and hospital expenditures are calculated as the sum of the variables IPFM- CRyy and IPDMCRyy from the MEPS inpatient event files, ERFMCRyy and ERDMCRyy from the MEPS emergency room event files, OPFMCRyy and OPDM- CRyy from the MEPS outpatient department event files, and OBVMCRyy from the MEPS office-based event files, where yy501, 02, 03 for each person from the respective full-year consolidated MEPS Public Use Files (HC-060, HC-070, and HC-079). 2 The weights of people who died during the year are post-stratified in the MEPS to corresponding estimates derived using data obtained from the Medicare Current Beneficiary Survey and vital statistics information provided by the National Center for Health Statistics (NCHS). Similarly, the weights of those entering nursing homes are post-stratified (MEPS Public Use File HC-105 documentation). Thus, the MEPS likely maintains adequate representation of Medicare beneficiaries who die during the year and enter nursing homes. However, expenditures for Medicare beneficiaries who die during the year may be underreported to a larger degree than other Medicare beneficiaries because of the potential difficulties of collecting these data post-mortem in the MEPS. The point estimates of the ratio of MEPS-reported Medicare expenditures to the expenditures in their CMS claims for decedents in MEPS was.76 (n5128) compared to.88 for living beneficiaries in the analytic matched sample (n54,045), but the difference is not statistically significant. Using propensity-score reweighting to adjust for the substantial underrepresentation of decedents in the analytic matched sample (because SSNs or HICNs were less likely to be provided for them) suggests that including decedents would increase the gap between the MEPS and MCBS by 2.5 percentage points. Again, the difference between the estimated gap with and without decedents is not statistically significant and no strong conclusions can be drawn from this analysis. There may be similar difficulties in obtaining complete utilization and expenditure data for beneficiaries living in the community who 106

16 Medicare Expenditures in the MEPS enter nursing homes, assisted living facilities, or are otherwise considered institutionalized in the MEPS at some point during the year. The population institutionalized part of the year is about one-quarter the size of decedents and the MEPS sample size is too small (n525) to support analysis. Potentially offsetting the likely downward bias from excluding decedents and beneficiaries living part of the year in the community and part of the year in nursing homes and other institutions is sampling variation in the MCBS. Over time, MCBS estimates of total Medicare expenditures on personal health care services match those reported from Medicare administrative data in the National Health Expenditure Accounts on average. However, during the particular time period we examine, , the MCBS estimates are 2.2 percentage points higher than aggregate administrative totals (NHEA 2008). References Agency for Healthcare Research and Quality (AHRQ) MEPS HC-036BRR: MEPS Replicates for Calculating Variances File. download_data_files_detail.jsp?cbopufnumber5 hc-036brr&prfricon5yes. Accessed January 7, Basu, A., and P. J. Rathouz Estimating Incremental and Marginal Effects on Health Outcomes Using Flexible Link and Variance Function Models. Biostatistics 6: Bhandari, A. M., and T. Wagner Self- Reported Utilization of Health Care Services: Improving Measurement and Accuracy. Medical Care Research and Review 63: Buntin, M. B., and A. M. Zaslavsky Too Much Ado About Two-Part Models and Transformation? Comparing Methods of Modeling Medicare Expenditures. Journal of Health Economics 23: Cleary, P. D., and A. M. Jette The Validity of Self-Reported Physician Utilization Measures. Medical Care 22: Centers for Medicare and Medicaid Services (CMS), Office of the Actuary National Health Expenditures Accounts: Definitions, Sources, and Methods, hhs.gov/nationalhealthexpenddata/downloads/ dsm-06.pdf Accessed March 12, Cohen, J. W., A. C. Monheit, and S. B. Cohen. 1996/1997. The Medical Expenditure Panel Survey: A National Health Information Resource. Inquiry 33(4): Cohen, J. W Design and Methods of the Medical Expenditure Panel Survey Household Component. MEPS Methodology Report no. 1, AHCPR Pub. no Rockville, Md.: Agency for Health Care Policy and Research (AHCPR). files/publications/mr1/mr1.shtml. Cohen, S. B Sample Design of the 1996 Medical Expenditure Panel Survey Household Component. MEPS Methodology Report no. 2, AHCPR Pub no Rockville, Md.: AHCPR. data_files/publications/mr2/mr2.pdf Design Strategies and Innovations in the Medical Expenditure Panel Survey. Medical Care 4(Suppl):III-5 III-12. Cutler, D. M., A. B. Rosen, and S. Vijan Value of Medical Innovation in the United States: New England Journal of Medicine 355: Ezzati-Rice, T. M., D. Kashihara, and S. R. Machlin Health Care Expenses in the United States, MEPS Research Findings no. 21, AHRQ Pub no Rockville, Md.: Agency for Healthcare Research And Quality (AHRQ). Data_Files/Publications/Rf21/Rf21.Shtml. Hill, S. C., and G. E. Miller Health Expenditure Estimation and Functional Form: Applications of the Generalized Gamma and Extended Estimating Equations Models. Unpublished mimeograph. Rockville, Md.: Center for Financing, Access and Cost Trends, AHRQ. Lin, L A Concordance Correlation Coefficient to Evaluate Reproducibility. Biometrics 45: Machlin, S. R., J. W. Cohen, S. H. Zuvekas, and J. M. Thorpe Accuracy Of Household Reported Payments For Physician Visits In The 1996 Medical Expenditure Panel Survey Proceedings of the American Statistical Association of the Section on Survey Research Methods. Manning, W. G., and J. Mullahy Estimating Log Models: To Transform or Not to Transform? Journal of Health Economics 20: Manning, W. G., A. Basu, and J. Mullahy Generalized Modeling Approaches to Risk Adjustment of Skewed Outcomes Data. Journal of Health Economics 24: Olin, G., S. Zuvekas, V. Kumar, P. Ward, K. Williams, and D. Wobus Medicare-MEPS Validation Study: A Comparison of Hospital and Physician Expenditures. Working paper. Rockville, Md.: AHRQ. Ritter, P. L., A. L. Stewart, H. Kaymaz, et al Self-Reports of Health Care Utilization Compared to Provider Records. Journal of Clinical Epidemiology 54:

17 Inquiry/Volume 46, Spring 2009 Roberts, R. O., E. J. Bergstrahl, L. Schmidt, et al Comparisons of Self-Reported and Medical Records Health Care Utilization Measures. Journal of Clinical Epidemiology 49: Rosen, A. B., and D. M. Cutler Measuring Medical Care Productivity: A Proposal for U.S. National Health Accounts. Survey of Current Business 87(6): Selden, T. M., and M. Sing Aligning the Medical Expenditure Panel Survey to Aggregate U.S. Benchmarks. Working paper. Rockville, Md.: AHRQ. Sing, M., J. Banthin, T. Selden, et al Reconciling Medical Expenditure Estimates from the MEPS and NHEA, Health Care Financing Review 28: Wallihan, D. B., T. E. Stump, and C. M. Callahan Accuracy of Self-Reported Health Services Use and Patterns of Care among Urban Older Adults. Medical Care 37: Williams, R. L A Note on Robust Variance Estimation for Cluster-Correlated Data. Biometrics 56(2):