Introduction. Developing and Evaluating Performance Measures. Defining and Measuring Quality

Transcription

1 This issue brief, the third in a series prepared by Breakaway Policy Strategies for FasterCures, discusses how quality is defined in health care. It explains how performance is measured by various public and private entities that pay for health care, identifies the key organizations that develop and evaluate measures, and discusses the main uses of performance data and concerns with the validity and usefulness of the data in practice. Introduction As discussed in A Closer Look at Alternative Payment Models, the second brief in the FasterCures Value and Coverage Issue Brief Series, the United States is transitioning from a fee- for- service (FFS) payment system one that pays doctors and hospitals for each individual service provided into a system that rewards providers for quality while controlling costs. Studies have shown that at least 20 to 50 percent of all prescriptions, visits, procedures and hospitalizations in the U.S. are inappropriate either as the overuse, underuse or misuse of 1,2,3 what has been shown to be effective and beneficial care. This has fueled demand by consumers and health care policymakers for information regarding the performance of health care providers and institutions. Agencies like CMS and the Agency for Health Care Research and 7 Quality (AHRQ), as well as non- profit private developers such 8 as the Joint Commission and the National Committee for 9 Quality Assurance (NCQA) are the organizations responsible for the development of many of the performance measures in use today. Professional societies such as the American Heart Association, the American College of Cardiology, and the American College of Surgeons also develop measures. After measures are developed, they sometimes undergo evaluation by the National Quality Forum (NQF) a public/private, multi- stakeholder organization that endorses general standards for 10 measurement and specific measures. The most widely used performance measures are those endorsed by the NQF or those developed by the NCQA, known as the Healthcare Effectiveness Data and Information Set (HEDIS) which have been vetted by stakeholders and tested for reliability, feasibility and validity. Developing and Evaluating Performance Measures Performance measurement is a quantitative way to measure quality, and can be defined as whether or how often a process and outcome of care occurs. 4 In certain clinical areas, such as cardiac and intensive care, performance measurement has been associated with improvements in providers use of evidence- based strategies 5,6 and patients health outcomes. Aside from quality improvement, capturing performance data can also increase transparency in the quality of care provided and serve as the basis for accreditation or certification for provider groups or organizations. Defining and Measuring Quality The Institute of Medicine (IOM) describes quality as multidimensional and inclusive of concepts that go well beyond safety. IOM defines high- quality care as care that is safe, effective, patient- centered, timely, efficient, and equitable (with no disparities between racial or ethnic groups)11. In the late 1990 s, the American Medical Association (AMA) began a program to develop physician- level performance measures to be used for quality improvement. By bringing together physicians and experts from various medical specialties, clinical process measures for several areas of

2 medical practice were developed. The AMA s program, now known as the Physician Consortium for Performance Improvement (PCPI), continues to lead efforts in developing, testing and implementing performance measures for use at the 12 point of care. Organizations that define and measure Quality Organization Involvement with Quality Measurement NCQA A private, nonprofit that reviews and accredits health insurance plans. Created the Healthcare Effectiveness Data and Information Set (HEDIS), a set of health plan performance measures used for both public reporting and accreditation. AHRQ The Joint Commission NQF Federal agency within HHS which aims to improve quality, safety, efficiency, and effectiveness. AHRQ initiatives include: the National Quality Measures Clearinghouse (NQMC), which provides information on specific evidence- based health care quality measures, and the Consumer Assessment of Health Providers & Systems (CAHPS), a comprehensive series of patient satisfaction surveys regarding health care services. Process measures, which look at improvement and assess the performance of activities shown to contribute to positive patient outcomes, are the most commonly used of the quality measures. The measures can be used for any number of things, including quality improvement, accountability or research or some combination of the three. The entity performing the measurement must determine the purpose and intended use of the measure and can then use the measurement to help identify problems, establish baseline results, and/or drive quality and performance improvement. Measurement results can be expressed as a rate, ratio, frequency distribution or score for average performance and is often interpreted in comparison to a set standard. Primary Uses of Performance Data Performance measurement data is used primarily for quality improvement initiatives and accountability. Both the public and private sectors have An independent not- for- profit that accredits more than 20,000 health care organizations and programs in the United States. States and CMS require hospitals and other health care organizations to be accredited by the Joint commission in order to participate in Medicare and Medicaid. A private, nonprofit that builds consensus around quality improvement priorities and evaluates and endorses quality standards and measures. There are several quality metrics that allow a user to quantify the quality of health care services by comparison to specific criteria. These metrics include: process measures, outcome measures, patient experience measures, and structure 13 measures. Detailed descriptions of these measures is included in A Closer Look at Alternative Payment Models, the second brief in the FasterCures Value and Coverage Issue Brief Series. developed quality improvement initiatives, such as pay- for- performance programs (P4P). P4P is a term that describes payment models that offer financial incentives to providers who achieve or exceed specified quality benchmark. (P4P initiatives are discussed in greater detail in A Closer Look at Alternative Payment Models.) Under most payment models, payments to physicians and hospitals are adjusted on the basis of whether the providers achieve a pre- determined set of quality measures. In theory, providing the public with access to performance data should allow patients to make informed choices about their care and be more involved in their medical decision- making. It should also allow providers to identify areas for improvement and motivate them to make those improvements. Indeed, some studies have shown that publicly reporting provider performance data can result in quality improvements.14 Such data, however, is not always readily accessible by patients either patient are not aware that the data is available, it is not the exact information they need, or it is not presented in an 15 understandable way. Commercial health plans also make performance data publicly available by classifying providers into different value tiers and encouraging consumers to choose certain providers by offering lower cost- sharing. Page 2 of 6

3 Performance Measurement in Practice There are many examples of performance measurement in practice. A few examples include: The Physician Quality Reporting System (PQRS), a voluntary reporting program implemented in 2007 by the Centers for Medicare and Medicaid Services (CMS),16 offers a financial incentive to eligible professionals for voluntarily reporting data on specific quality measures applied to the Medicare population. The program uses measures developed by several sources the majority coming from the AMA PCPI. For 2014, there are 285 measures in total, of which are individual quality measures. Though there is a financial incentive to encourage participation in the PQRS, less than 30 percent of eligible providers actually report data to CMS. This low participation rate may be due to the concerns that many physicians have regarding the validity of this data and the credibility and accuracy of public reporting, especially in regards to outcome 18 measures. Despite these concerns, participation in PQRS has been growing. CMS uses performance data to offer bonuses to Medicare Advantage (MA) plans using a star ratings system. The program, implemented under the Affordable Care Act (ACA), pays MA plans bonuses based on the Medicare 5 Page 3 of 6 star program, which rates plans online on a scale of 1 to 5 stars. Stars are awarded based on performance measures taken from CMS administrative data, HEDIS measurement data, and CHAPS survey data. CMS also uses performance measurement data in many of its P4P programs, including the End- Stage Renal Disease (ESRD) Bundled- Payment and 19 Quality Incentive Program. The use of performance measurement in improving specific types of care has also produced some promising results. For example, there have been significant changes in cardiovascular care in last 10 years. Key changes in this treatment area came from a decision by CMS to support the measurement of care provided to patients with an acute myocardial infarction. Using reported performance data, CMS was able to identify gaps in the quality of care and facilitated and supported efforts to improve cardiovascular care. Hospitalizations for acute myocardial infarction dropped by more than 25 percent and hospitalizations for heart failure fell by more than 30 20,21 percent. Post- hospitalization mortality due to acute 22 myocardial infarction also decreased by over 20 percent. CMS has also created online tools to aid in consumer decision- making, including Hospital Compare, Nursing Home Compare and Physician Compare. These sites aid patients in making informed decisions about their health 23 care based on publicly available provider quality data. The program was designed to encourage providers to improve the quality of their care through accountability. Information made available includes, for example, that regarding readmission, complications and death, timely and effective care, use of medical imaging, and surveys of patients experiences.

4 demonstrated improved scores on mostly process measures, the program did little to improve patient outcomes. Assessing the Validity and Usefulness of Performance Measures While the purpose of performance measures is to improve quality and promote transparency, even proponents believe it is important to: ensure that measures are appropriate; understand the scientific basis underlying and the strengths and limitations of each measure; and reduce inaccurate inferences about provider performance. Many agree that scientifically rigorous and valid measures of 24 performance can truly improve value in health care. However, despite the widespread acceptance and use of performance measures in recent years, some argue that there has not been a 25 sufficient corresponding increase in the quality of care. There are a few possible explanations for this: concerns about the strength of the evidence 26 underlying the performance measures, the ways in which measures are used to encourage 27 providers to improve care, 28 limitations of the amount and type of existing data. The measures also may not be suitable for clinically important subpopulations meaning it is easier to achieve in practice, but will have little or no impact on the group of patients who need 29 improvement most, or they may not account for a patient s or 30 clinician s personal preferences for certain services. Another concern is that the majority of the current quality improvement initiatives focus too heavily on process measures instead of outcome measures which do not always result in improved care or value of care for the patient. Process measures look at improvement and assess the performance of activities shown to contribute to positive health outcomes for patients while outcome measures look at the effects that care had on patients. One example of a program that did not improve care was the Medicare Premier Hospital Quality Incentive Demonstration, the largest test of both public reporting and P4P, which failed to have any significant impact on the value of care in three key clinical conditions and neither reduced patient mortality nor cost growth.31,32 While the hospitals However, shifting to using more outcome measures to improve quality is not as easy as it sounds. Patients health outcomes are not solely based on the quality of care they receive, but also their previous risk factors, chance events, or social 33,34 determinants of health. As with process measures, there are also concerns regarding the validity of outcome measures based on the source of data mainly claims data which may fail to identify preexisting conditions and complications that occur 35 after hospital admission. Modifications to Measures over Time Though concerns exist regarding the validity and usefulness of performance measurement, programs and measures can be modified over time. As those developing and evaluating these measures continue to assess how these work in practice, modifications can be made directly to the measures, by finding new or multiple ways for providers to satisfy the measures, or by creating exclusions for specific circumstances as in the example above. The future of evidence- based performance measurement rests on the ability of stakeholders to make such modifications. One example of how performance measures can be modified involves the Medicare National Pneumonia Project. The Project used performance measures related to timing of antibiotic treatment for patients coming to hospitals with community- 36 acquired pneumonia. The Project adopted a measure to administer antibiotics within four hours of a patient first arriving to the hospital. Use of this measure, however, failed to take into account that often, pneumonia cannot be diagnosed during an initial evaluation and that the appropriate standard of care for a stable patient is to withhold treatment until a more certain diagnosis can be made. After studies failed to show that the four hour time window for antibiotic administration 37 decreased mortality in stable patients, the Joint Commission relaxed the time window to 6 hours and created a new carve- out for diagnostic uncertainty that can be used to exclude certain patients from this measure. Page 4 of 6

5 Conclusion 12 Rewarding performance in our health care system is gaining momentum as a way to improve the quality and value of care. Public and private payers continue to measure not only the under and over use of services, but also to assess the quality of certain services and interventions, using the results for public reporting and P4P programs. It is important for performance measures to take into account the perspectives of various health care stakeholders including the patient, the purchaser, and the provider. Though there are hurdles to overcome in perfecting the use of performance measurement in practice, current efforts have demonstrated the potential of these measures if used and modified appropriately along the way. About the PCPI. American Medical Association. n.p., n.d.. Available at: assn.org/ama/pub/physician- resources/physician- consortium- performance- improvement/about- pcpi.page?. Accessed March, Selecting Quality Measures. Agency for Healthcare Research and Quality. n.p., n.d. Available at: Accessed March Ferris RG and Torchiana DF. Public Release of Clinical Outcomes Data Online CABG Report Cards. New England Journal of Medicine 363 (2010): Totten AM, Wagner J, Tiwari A, et al. Closing the Quality Gap: Revisiting the State of the Science (Vol. 5: Public Reporting as a Quality Improvement Strategy, Evidence Reports/Technology Assessments. Agency for Health Care Research and Quality (2012). 16 Berenson R, Kaye D. Grading a physician s value- the misapplication of performance measurement. N Engl J Med. 369 (2013): The Urgent Need to Improve Health Care Quality. Consensus Statement- September 16, Institute of Medicine National Roundtable on Health Care Quality. JAMA. 280 (1998): McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 348 (2003); Kerr EA, McGlynn EA, Adams J. Profiling The Quality Of Care In Twelve Communities: Results From The CQI Study. Health Affairs (2004): Statement preceding each physician measurement set authored by AMA, JCAHO, and NCQA. Web. June Available at assn.org/ama/pub/category/2946.html. Accessed March, Krumholz HM, Wang Y, Chen J, et al. Reduction in Acute Myocardial Infarction Mortality in the United States: Risk- Standardized Mortality Rates from Journal of the American Medical Association, (2009): Chassin MR, Loeb JM, Schmaltz SP, et al. Accountability Measures Using Measurement to Promote Quality Improvement. New England Journal of Medicine, 363 (2010): AHRQ at a Glance. Rockville, MD: Agency for Healthcare Research and Quality, (Accessed March 2014). 8 Facts About The Joint Commission. Oakbrook Terrace, IL: The Joint Commission. N.d. Web. April Measures Codes. Centers for Medicare and Medicaid Servvices. N.p., Available at: Initiatives- Patient- Assessment- Instruments/PQRS/MeasuresCodes.html. Accessed March 2014). 18 Physician performance measurement & reporting introduction. Value- based Purchasing Guide. n.p., Van Lare JM and Conway PH. Value- Based Purchasing National Programs to Move from Volume to Value. New England Journal of Medicine. 367 (2012): Chen J, Normand SL, Wang Y, et al. National and Regional Trends in Heart Failure Hospitalization and Mortality Rates for Medicare Beneficiaries, JAMA (2011): Wang OJ, Wang Y, Chen J, et al. Recent Trends in Hospitalization for Acute Myocardial Infarction. American Journal of Cardiology, (2012): Krumholz HM, Wang Y, Chen J, et al. Reduction in Acute Myocardial Infarction Mortality in the United States: Risk- standardized Mortality Rates from JAMA (2009): Hospital Compare. Nursing Home Compare. Physician Compare. Medicare.gov. n.p., n.d. Available at: ww.medicare.gov/hospitalcompare/search.html; Desirable Attributes of HEDIS. Washington, DC: National Committee for Quality Assurance. n.d. Web. April Available at: (accessed March 2014). Guidance for Measure Testing and Evaluating Scientific Acceptability of Measure Properties. Washington DC: National Quality Forum. January Pronovost PJ and LIlford R. A Road Map for Improving the Performance of Performance Measures. Health Affairs (Millwood) (2011): National Quality Forum. n.p., Available at: (accessed March 2014) st Crossing the Quality Chasm: A New Health System for the 21 Century. Institute of Medicine. Washington, DC: National Academies Press, Tricoci, P., JM. Allen, J.M. Kramer, R.M. Califf, and S.C. Smith, Jr. Scientific Evidence Underlying the ACC/AHA Clinical Practice Page 5 of 6

6 Guidelines. Journal of the American Medical Association (2009): Werner, Rachel and Dudley, R. Medicare s New Hospital Value- Based Purchasing Program is Likely to Have Only a Small Impact on Hospital Payments, Health Affairs (2012): Roski J and McClellan M. Measuring Health Care Performance Now, Not Tomorrow: Essential Steps to Support Effective Health Reform. Health Affairs. 30 (2011): Hayward, R. All or Nothing Treatment Targets Make Bad Performance Measures. The American Journal of Managed Care.13.3 (2007): Tinetti, M.E., et al. Health outcome Priorities Among Competing Cardiovascular, Fall Injury, and Medication- Related Symptom Outcomes. Journal of the American Geriatrics Society (2008): Ryan AM, Nallamouth BK and Dimik JB. Medicare s Public Reporting Initiative On Hospital Quality Had Modest or No Impact on Mortality From Three Key Conditions. Health Affairs (2012): Ryan AM. Effects of the Premier Hospital Quality Incentive Demonstration on Medicare Patient Mortality and Cost. Health Services Research (2009): Bradley et al., Schwarz M Cohen AB, Restucciai JD, et al. How Well Can We identify the High Performing Hospital? Medical Care Research and Review (2011): Werner RM, Bradlow ET, and Asch DA. Does Hospital Performance on Process Measures Directly Measure High Quality Care or Is it a Marker of Unmeasured Care? Health Services Research (2008): Glance LG, Newman M, Martinez, EA, et al. Performance Measurement at a Tipping Point. Anesthesia and Analgesia (2011): Wachter RM, Flanders SA, Fee C, Pronovost PJ. Public reporting of antibiotic timing in patients with pneumonia: lessons from a flawed performance measure. Ann Intern Med.149 (2008): Wachter R, Flanders S, Fee C, Pronovost P. Public reporting of antibiotic timing in patients with pneumonia: lessons from a flawed per formance measure. Annals of Internal Medicine. 149 (2008): Page 6 of 6