The STS Composite Quality Measurement Methodology. Executive Summary

Transcription

1 The STS Composite Quality Measurement Methodology Executive Summary BACKGROUND Cardiothoracic surgery has a long and distinguished history of critical selfexamination to improve the quality of patient care. Twenty years ago our profession was challenged by an unprecedented call for accountability. Recognizing our responsibility to our patients and our profession, STS embarked on one of the most extensive voluntary clinical data collection initiatives in healthcare, the development of the STS National Adult Cardiac Surgery Database (NCD). This Database is now among the largest and most respected clinical data registries in the world, and studies based upon it have substantially advanced patient care, research, and quality initiatives. Because of the early development and continuing evolution of this outstanding Database, it has become the national gold standard for cardiothoracic surgery and has established clearly-defined benchmarks for clinical comparisons. Such information has become the cornerstone of quality assessment in cardiothoracic surgery. Our profession is now at a similar critical juncture as it was 20 years ago. Once again we are faced with a call for greater accountability, but now this attention is directed to the entire medical profession, not just one specialty or particular high-profile procedure. Furthermore, there is now widespread consensus that the keystone of

2 accountability is the objective assessment of quality, and that this is best accomplished through performance measurement. The objective measurement of provider quality has thus become a dominant theme in the re-engineering of American health care. For example, the Institute of Medicine s 2006 quality report titled Performance Measurement: Accelerating Improvement (reference?), and Porter and Teisberg s highly regarded book, Redefining Health Care (reference?), both advocate strongly for performance measurement as a central tenet of health care reform. By enthusiastically embracing this performance measurement paradigm, which is completely consistent with the principles that have always guided the cardiothoracic surgery profession, STS will continue to strengthen its leadership position at the forefront of American health care reform. Furthermore, such a paradigm provides the opportunity to set a rigorous precedent for quality measurement initiatives, thereby precluding systems that may be inferior and less acceptable to all stakeholders. QUALITY MEASUREMENT TASK FORCE MISSION Given the current health care policy environment, the Society s historical commitment to quality, and its extensive experience with clinical data collection, the STS Executive Committee voted in May 2005 to initiate a comprehensive quality measurement program for cardiothoracic surgery. A Quality Measurement Task Force (QMTF) was subsequently created to: (1) select a set of individual quality indicators within multiple domains of care; (2) develop methodologies for combining such measures within and across domains (commonly referred to as composite scores ); and

3 (3) examine various approaches for rating provider performance based upon overall composite quality scores. INDIVIDUAL QUALITY MEASURES AND DOMAINS Following extensive research, deliberations, and pilot studies, the QMTF will publish the results of its work as a two-part supplement to the April issue of The Annals of Thoracic Surgery. Part 1 (Conceptual Framework and Measure Selection) explains the principles underlying the selection of individual measures and provides a detailed description of the eleven specific measures chosen. Initial QMTF reports will focus on isolated CABG procedures, and the level of analysis will be hospitals and programs (referred to in this summary as providers ) rather than individual surgeons. The initial group of quality measures will be restricted to those already endorsed by the National Quality Forum, thus assuring their national acceptance. Only those measures currently available within the STS National Adult Cardiac Database were considered so that additional data collection would not be required. Measures were chosen to include process, structural, and outcomes domains as well as the broad temporal spectrum of the cardiac surgical experience, from preoperative medications to measures that primarily impact secondary prevention and long-term survival. Finally, the QMTF selected measures that would likely be regarded as valid by all major stakeholders, that would be consistent with the recommendations of leading health care policy groups such as the Institute of Medicine, and that would be transparent and actionable by surgeons. Eleven individual measures of CABG quality within four domains were selected:

4 (1) Perioperative Medical Care, a process bundle of 4 medications (preoperative beta blockade and discharge anti-platelet, beta blockade, and lipid-lowering agents). These are considered as an all or none bundle, consistent with current national recommendations in performance measurement. Thus, failure to provide all of the 4 recommended medications results in loss of credit for that domain. The QMTF acknowledges that some patients will be ineligible for one or more of these measures. Efforts to compile a comprehensive list of eligibility criteria were abandoned because of the difficulty in achieving objective consensus on what constitutes a legitimate exclusion, as well as the problematic aspects of documenting and auditing such exclusions. The QMTF chose a simpler and more straightforward approach. We believe that, unlike case-mix differences, there is no reason to suspect that eligibility for process measures should vary systematically or substantially across institutions. Therefore, we recognize only a few straightforward exclusions--- previous cardiac surgery for the internal mammary artery (IMA) measure, operative mortality for the discharge medication measure, and previous cerebrovascular accident (CVA) for the postoperative stroke measure. We assume that all remaining potential exclusions are distributed randomly among institutions, and that all institutions will have a somewhat underestimated percentage of process measure compliance. As this is scaled across institutions, none should be particularly disadvantaged. (2) Operative Care, a single process measure IMA use. Although many variations of surgical technique and cardiopulmonary bypass (or off-pump approaches) have been advocated, use of at least one IMA is perhaps the only generally recognized operative standard of care for CABG.

5 (3) Risk-Adjusted Operative Mortality. This is the traditional risk-adjusted operative mortality measure available to STS NCD participants since the inception of the database. It includes all deaths occurring up to thirty days post surgery regardless of venue, and all deaths occurring in-hospital regardless of the timing. (4) Postoperative Risk-Adjusted Major Morbidity, defined as the risk-adjusted occurrence of any of the following renal failure; deep sternal wound infection; reexploration; stroke; and prolonged ventilation/intubation. As with the Perioperative Medical Care bundle, this is an any or none measure, requiring the absence of all such complications. Once converted to this dichotomous measure, it can be risk-adjusted in a fashion exactly analogous to that for risk-adjusted mortality. COMPOSITE SCORING METHODOLOGY Part 2 of the QMTF report (The Measurement of Adult Cardiac Surgery Quality: Statistical Considerations in Composite Measure Scoring and Provider Rating) describes the methodology for combining these individual measure and domain scores into an overall composite. The scores for each of the four domains of care are first standardized to account for differences in scale (process compliance is a large number; mortality and morbidity are small numbers) and directionality (for process compliance, larger is better; for morbidity and mortality, smaller is better). Applying this standardization approach to actual 2004 STS data, a 1% change in risk-adjusted mortality would be roughly equal to an 8% change in risk-adjusted morbidity, an 11% change in IMA usage, and a 28% change in appropriate perioperative medication use. Finally, the standardized domain scores are added together to obtain an overall composite quality score.

6 PROVIDER RATING The overall composite CABG quality score is next used to generate a provider rating, which has been an mandate from payers. In general, such rating systems can be (a) absolute, in which an externally-derived criterion for acceptable performance is established, or (b) relative, in which providers are judged in relation to their peers. We chose option (b), as there is little empirical basis for establishing normative absolute thresholds. However, the QMTF did not utilize relative rating systems that predetermine a percentage of providers to be in the low performing group (e.g., distribution tails ). Rather, we required that there be a 99% true Bayesian probability that a provider s performance differed from the average performance of other providers. If all groups achieved statistically indistinguishable performance, it is theoretically possible that no provider would be considered above or below average. Our pilot studies based upon actual 2004 STS data suggest that about 13 % of providers in such a rating system would be classified as low performing and another 10% as high performing. The decomposed domain scores (i.e., mortality, morbidity, IMA use, and medication bundle compliance) for these two groups differed quite substantially, yielding at least face validity to this approach. MISSING DATA Missing data are extremely disadvantageous to a provider s overall quality score, as the QMTF currently plans to impute non-compliance to such fields. It is very important that each program and its data managers accept nothing less than complete data

7 for the 11 individual NQF measures that comprise the composite. Similarly, if they have not already done so, providers should consider developing standardized order sheets that establish as a default the four medication process measures from the STS composite score (unless there are obvious contraindications). POTENTIAL UNINTENDED NEGATIVE CONSEQUENCES The voluntary development of this sophisticated performance assessment tool reaffirms the Society s longstanding commitment to quality. However, the QMTF recognizes that even an effort with such noble intentions has potential unintended negative consequences. For example, the fear of unfavorable quality scores could make some providers more risk averse, thus limiting access for the highest-risk patients. However, unlike rating systems that rely solely on operative mortality, these more comprehensive and broad- based quality scores encompass multiple measures of performance, thus mitigating to some extent the impact of any single measure. With the consent of providers, these quality scores may in the near future be made available to payers to ensure accountability or to satisfy pay for performance reporting requirements. It is possible, however, that payers may overly rely on such scores and seek to inappropriately redirect patients based solely on perceived quality differences that are, in reality, small or transient. Conversely, there is substantial historical evidence that many consumers and payers will ignore such scores, which would negate some of their intended value. Both these concerns will be addressed with an aggressive education initiative. Finally, it would be unfortunate if such scores were used exclusively for

8 profiling, as they have tremendous potential to direct the focus of provider-initiated quality improvement activities. IMPLEMENTATION Working in conjunction with Duke Clinical Research Institute (DCRI), the QMTF anticipates that the first hospital/group-level composite quality scores will be reported to providers later this year. The standard report to all STS NCD participants will be expanded to include the overall composite quality score as well as individual measure and domain scores, thus facilitating quality improvement initiatives by providers. Although the first iteration of this methodology does not include scores for individual surgeons, it is highly likely that commercial payers and/or government agencies may require this in the near future, either for accountability or pay for performance initiatives. This further emphasizes the need to submit accurate and complete data, particularly for the measures that comprise the composite score. The QMTF anticipates that these methodologies will undergo modification over time as new knowledge is gained from both academic research and the practical realities of implementation. Our approach is flexible and can easily accommodate such changes. Furthermore, it is our belief that variations of this quality scoring methodology can be translated to many other areas of health care. CONCLUSION Notwithstanding these and other caveats, STS is confident that this bold step charts the proper course for our specialty. As a professional organization whose members

9 are entrusted with their patients lives, we hold a very special position of public trust. With this privilege comes the responsibility to always put the public s interest first, to provide our patients with the highest quality care, and to objectively document our performance. With the quality measurement methodology described in this report, STS continues its long and proud tradition in pursuit of these goals. David M. Shahian, MD Chair, STS Quality Measurement Task Force John E. Mayer, Jr., MD President, The Society of Thoracic Surgeons Frederick L. Grover, MD Immediate Past President, The Society of Thoracic Surgeons Richard P. Anderson, MD Chair, STS Council on Quality, Research and Patient Safety Fred H. Edwards, MD Chair, STS Workforce on National Databases