TRANSPLANT CENTER QUALITY MONITORING. David A. Axelrod, MD,MBA Section Chief, Solid Organ Transplant Surgery Dartmouth Hitchcock Medical Center

Transcription

1 TRANSPLANT CENTER QUALITY MONITORING David A. Axelrod, MD,MBA Section Chief, Solid Organ Transplant Surgery Dartmouth Hitchcock Medical Center

2 Nothing to Disclose Disclosures

3 Overview Background Improving Outcomes Measures CUSUM techniques

4 Transplant deaths blamed on poor leadership Sarah Boseley, health editor Thursday September 13, 2001 The position of the head of the Scottish heart transplant service looked uncertain last night following an official report which criticized him over the unnecessary deaths of patients at the London hospital where he previously worked. Surgeon Andrew Murday was joint coordinator of the heart and lung transplantation program at St George's hospital in Tooting, south London. An investigation by the commission for health improvement (CHI) yesterday found that poor leadership by Mr. Murday was partly to blame for the high death rate of transplant patients, some of whom would have lived if they had been treated elsewhere. Of 11 patients who had heart transplants between December 1999 and September 2000, eight died - a death rate massively out of kilter with the average for other transplant units, which are all supposed to achieve an 80% survival rate within a year of opening. Two out of three patients who received a heart and lung transplant also died. This was not, the CHI report finds, "an unlucky run."

5 The Problem Newspaper reports that St. Georges Hospital in London had an exceptionally high death rate after heart transplant The center had performed 371 transplants from National average survival > 80% St. Georges had 80% mortality in last 10 heart transplants IS THIS BAD LUCK, BAD CARE, OR BOTH?

6 Available Statistical Techniques Basic techniques Average mortality rate Rolling average (last 10 cases) Adjusted average mortality rates # of consecutive failures Cox proportional hazard models Adjusted for clinical characteristics Incorporates risk adjustment Statistical process control Continuous monitoring techniques (e.g. CUSUM)

7 Pitfalls of Episodic Outcomes Assessment Techniques Repeated measures Comparing observed to expected rates multiple times increase the risk of a type I error Can be improved with Bonferroni adjustment, but this reduces likelihood of detection Delay in detecting rapid change in performance Average annual mortality includes period of good performance which offsets a period of significant clinical decline Important to detect failures in real time Difficult to interpret Evaluation of output requires knowledge of statistics and probability

8 Goals of Outcome Monitoring Method Timely Easily interpretable Sensitive to clusters of failures Low likelihood of false positive signals Risk adjusted for important clinical characteristics Clinical relevant and useful

9 CUSUM Monitoring Cumulative Sum (CUSUM) is technique of monitoring outcomes Began as a method to monitor industrial processes Produces a graphical output that can be tracked over time. CUSUM monitoring utility in health care recognized in 1970s Limited by poor data collection Lack of risk adjustment techniques Recent innovations let to expanded interest Steiner et al. Incorporated risk adjustment Axelrod et al. Effective in multi-center assessment in a retrospective study Kalbfleisch and Biswas- now utilizes survival analysis rather than a binary (alive or dead) logistic analysis

10 CUSUM Charts Graphical representation of outcomes for process Can be risk-adjusted charts for important donor and recipient characteristics Plot outcomes over time to compare the results with expected outcomes based on a national model of mortality or graph failure 2 types: O - E charts and One-sided charts Trends in the plot line suggest improving or declining outcomes Once the trend line reaches a certain predefined level (one-sided charts) or exceeds a certain slope (O - E charts) the CUSUM signals

11 Binary CUSUM: Example

12 The Continuous O-E CUSUM The O-E Chart plots the difference between the observed number of deaths and the expected number of deaths (adjusted for the particular patient mix of the center). The expected number is computed using national rates Over an interval, the O-E chart TRENDS UP if the death rate at the center exceeds the risk-adjusted national average. TRENDS DOWN if the death rate at the center is less than the riskadjusted national average. IS HORIZONTAL, if the death rate at the center is approximately the same as the adjusted national rate. The slope of the plot gives an informal estimate of the approximate ratio of the death rate at the center to that for the adjusted national average (see direction arrows).

13 O-E Figure 4: Vmask demo upper arm ( t, Ot Et ) Vertex 10 5 d h 0 lower arm h the rise in the arm corresponding to the distance (d) from origin to vertex Time (years)

14 Risk-Adjusted O-E CUSUM O-E CUSUM chart An Example A.1: Kidney Program O-E CUSUM Chart RR=2 RR=1 RR= Time (years)

15 Risk-Adjusted O-E CUSUM O-E CUSUM chart An Example C.1: Kidney Program O-E CUSUM Chart RR=2 RR=1-6 RR= Time (years)

16 Continuous One-Sided CUSUM Chart Similar to the O-E chart except bounded below by the x-axis Designed to primarily detect deviation from expected practice. Trends upward if the death rate is substantially higher than the national rates. Tends downward or stays close to x-axis, if the death rate is near or lower than the national average.

17 Continuous One-Sided CUSUM Chart Control limit is a horizontal line if the chart touches or crosses this, it registers a signal The signal indicates that death rates are larger than would be expected by chance and a review for possible causes of declining performance is advisable. The level of the control limit is chosen to keep the chance of a false positive small. Facility size (txps/yr) one-sided CUSUM L power ARL

18 Risk-Adjusted CUSUM One-Sided CUSUM Chart Example 1 A.2: Kidney Program one-sided CUSUM Chart Time (years)

19 Risk-Adjusted CUSUM One-Sided CUSUM Chart Example 2 B.2: Kidney Program one-sided CUSUM Chart Time (years)

20 Retrospective Analysis CUSUM charts were constructed using the novel risk adjusted, survival analysis for all kidney and liver transplant centers Included deceased donor txps from (1/05-12/07): 9,783 liver 67 centers kidney 112 centers Centers were flagged using a sized adjusted signaling level targeted to a false positive rate of 7.5% over three years

21 Facility size Retrospective Analysis- Kidney Transplant Results # and ave. time for PSR flagging # and average time for onesided CUSUM # and ave. time for O-E CUSUM transplants/yr # facilities # time # time # time NA

22 CUSUM Conclusion CUSUM charting provides a reliable, risk adjusted method of tracking outcomes of a clinical process Can be tuned to balance the need for sensitively to detect clinical failures with the requirement to limit the number of false positive signals Graphical output is easily interpretable with a minimal amount of training Providing outcomes are promptly reported the CUSUM can provide real time insight into TC outcomes