Demystifying Transplant Performance Reviews Robyn Zernhelt Performance Analyst
Pre-Assessment Transplant Performance Reviews
The PAIS reviews which transplant programs? 1. Kidney, Pancreas, Intestine 2. Kidney, Heart, Liver 3. Kidney, Liver, Lung, Heart, Pancreas 4. Kidney, Liver 25% 25% 25% 25% Kidney, Pancreas, Intestine Kidney, Heart, Liver Kidney, Liver, Lung, Heart, P... Kidney, Liver
How long is the cohort time frame the PAIS uses to review program outcome performance? 1. 1.5 years 2. 2.5 years 3. 3 years 4. 4 years 25% 25% 25% 25% 2.5 years 4 years 1.5 years 3 years
What does the PAIS consider when reviewing a program? 1. Personnel changes 2. History of outcomes/inactivity 3. Raw data 4. SRTR data 5. All of the above 1 1 1 1 1 Personnel changes History of outcomes/inactivity Raw data SRTR data Previous peer visit
A stand alone pediatric hospital will be identified for functional inactivity if a transplant has not been performed in the last 1. 3 months 2. 6 months 3. 9 months 4. 1 year 25% 25% 25% 25% 3 months 6 months 9 months 1 year
Membership and Professional Standards Committee (MPSC) 38 members, 2 HRSA reps Chair and Vice Chair 11 Regional Representatives 25 At Large Representatives Broad community representation Physicians and Surgeons OPO Administrators and Procurement and Quality Staff Transplant Administrators and Coordinators Lab Directors Living donor and Recipient
MPSC Responsibilities Develop and review membership criteria Review applications and make recommendation for Board action Review living donor adverse events Projects suggest improvements to Bylaws, Policies, or review procedures Review transplant program and OPO performance* Review member compliance with OPTN Policies and Bylaws
MPSC Organizational Structure MPSC PAIS Applications OPO Performance PCSC Transplant Program Outcomes & Inactivity Surveys Non-routine routine & allocation issues
Why did you receive a letter from the PAIS? PAIS received data reports from SRTR or UNOS research department Program identified for review PAIS requests more info
PAIS has 2 Categories of Review Outcome Reviews - 1 Year Patient and Graft Survival (kidney, liver, lung, heart, and pancreas patient only): Large Volume Program Reviews Small Volume Program Reviews Functional Inactivity Reviews: Transplant Inactivity
Performance Review Process Program identified, MPSC sends initial inquiry letter Program submits requested information Review by Ad Hoc Subcommittee of 3 PAIS members Recommendation of Ad Hoc Subcommittee approved by PAIS approved by the MPSC MPSC communicates action to program
PAIS Large Volume Outcome Review Program must meet all three of the following for either graft or patient survival (or both): Observed (actual) events (graft failures or deaths) Expected events >3 Observed events/expected events >1.5 Observed survival is significantly lower than expected survival (onesided p<0.05, Poisson distribution)
PAIS Small Volume Outcome Review One death or graft failure identified by SRTR Considered for notification if: Already under review for functional inactivity & flagged as small volume by SRTR At least 1 event in subsequent years
Who receives an inquiry letter? Initial inquiry sent to all programs identified that are not already under outcome review, or released from review within the past 2 meeting cycles.
Reviews and actions taken by PAIS/MPSC PAIS/MPSC reviews submissions from transplant programs, SRTR analysis, and other available data Programs already under review, MPSC may: Release from reporting Continue to report Recommend an expanded outcomes questionnaire Invite for informal discussion Recommend peer visit Voluntarily inactivate OR MPSC will consider: Probation Member Not in Good Standing Referral to Secretary of HHS
Additional actions taken If a policy violation is identified or a significant patient safety concern is identified, the MPSC may: Issue Notice of Uncontested Violation Issue Letters of Warning or Reprimand Consider Probation Member Not in Good Standing Referral to Secretary of HHS
Peer Visits Objective is to help a transplant program improve performance and/or compliance Goals of peer involvement: Clinical and operational experience Program and MPSC receive practical and relevant feedback 2 types of peer visits: MPSC Investigatory MPSC Advisory
PAIS Functional Inactivity Review Program Type Kidney, Liver or Heart Inactive Period 3 consecutive months Pancreas and Lung 6 consecutive months Stand-alone pediatric transplant programs 12 consecutive months
Who receives an inquiry letter? Programs identified for functional inactivity will receive an initial survey, with the following exceptions: Approved for less than 1 year No organ offers declined/no candidates on waiting list during the study period (1 st cycle only) Already under review for outcomes or inactivity Reviewed and released within the past 2 meeting cycles
Functional Inactivity Inquiry The following data are provided for programs identified: The number of active and inactive candidates on the program s waiting list The date of the last transplant before the study period Details on organ offers from the last transplant until the end of the study period
Reviews and Actions taken by the PAIS/MPSC Programs already under review MPSC may: Release from reporting Continue to report Invite for informal discussion Voluntarily inactivate OR MPSC will consider Probation Member Not in Good Standing Referral to Secretary of HHS
Factors Considered by PAIS/MPSC Additional programs under review Recent raw outcomes data Recent changes in key personnel SRTR analyses and MPSC metrics Plan for QI, External Consultation, Internal Analyses
Factors Considered by PAIS/MPSC Organ turndown reports Transplant volume history History of MPSC reviews for inactivity Recent changes in key personnel Plans for increasing volume and quality improvement Additional programs under PAIS review
Transplant Outcomes 2013-2014 July 2013 - Programs Reviewed: 61 Initial survey: 0 Release/conditional release: 13 Continue to monitor: 47 Voluntary inactivation following peer visit: 1 December 2013 - Programs Reviewed: 48 Initial survey: 0 In-person Informal Discussion: 1 Continue to monitor: 34 Release/conditional release: 12 Reenter PAIS review: 2 March 2014 - Programs Reviewed: 78 Initial survey: 32 Enter review upon reactivation: 3 Continue to monitor: 24 Inactivation requested: 1 Skip reporting cycle: 2 Release: 10 Return to monitoring: 3
Functional Inactivity & Waiting List Inactivity 2013-2014 July 2013 - Programs Reviewed: 29 Initial survey: 7 Offer Informal Discussion: 2 Continue to monitor: 10 Release: 10 December 2013 - Programs Reviewed: 27 Initial survey: 5 Offer Informal Discussion: 1 Continue to monitor: 9 Release: 11 Skip reporting cycle: 1 March 2014 - Programs Reviewed: 19 Initial survey: 4 Offer Informal Discussion: 1 Continue to monitor: 8 Release: 5 Skip reporting cycle: 1
New Performance Review Initiatives Post-transplant Outcomes new methodology and thresholds Board Approval June 2014 Effective January 1, 2015 Fall 2014 SRTR reports released in December - Bayesian methodology and new triggers MPSC review using new methodology Spring 2015 using SRTR reports issued in December 2014 Pre-transplant Performance Public Comment Fall 2014
Case Examples
Hospital AABB Liver July 2011: SRTR report showed liver program identified for patient outcomes for transplants performed 1/1/2008 6/30/2010: Patient # of TX s # Failed Actual Survival Exp Survival Obs- Exp Failures Obs/Exp Failures 1-Sided p Value 108 19 81.9% 88.8% 7.944 1.718 0.019
Hospital AABB Liver July 2011: Based on identified program requested to complete an outcomes survey September 2011: Program submitted survey citing reasons for outcomes lower than expected: Listing practices Need to revise protocols, ex: immunosuppression and utilization of DCD and ECD donors
Hospital AABB Liver January 2012: SRTR report showed liver program identified for graft and patient outcomes for transplants performed: 7/1/2008 12/31/2010 Graft # of TX s # Failed Actual Survival Exp Survival Obs-Exp Failures Obs/Exp Failures 1-Sided p Value 112 27 75.2% 83.6% 10.120 1.600 0.014 Patient # of TX s # Failed Actual Survival Exp Survival Obs-Exp Failures Obs/Exp Failures 1-Sided p Value 106 21 77.5% 89.0% 10.798 2.058 0.002
Hospital AABB Liver Jul 2012: MPSC requests program complete an Expanded Survey and offers a peer visit based on: Several recent patient deaths Now identified in 2 cohorts Sept 2012: Hospital accepted peer visit Nov 2012: Hospital undergoes peer visit Dec 2012: Peer visit report issued, MPSC requests a quality improvement plan based on peer visit recommendations
Hospital AABB Liver Program still identified for outcomes subsequent cohorts, but rates are improved Jan 2013-May 2014, hospital continues to submit reports, with updates to the QI Plan All action items on the QI plan are complete; program has not been identified for two cohorts July 2014: MPSC releases program from review
Helpful Hints: Read the letter carefully to make sure you prepare the correct documents Contact staff members listed on your letter Proactively monitor your data and take appropriate corrective actions Submit thorough, timely responses Validate data and correct errors in UNet Confidential medical peer review is UNOS privilege REMINDER: OPTN/UNOS does not require self reporting for inactivity
Post-Assessment Transplant Performance Reviews
The PAIS reviews which transplant programs? 1. Kidney, Pancreas, Intestine 2. Kidney, Heart, Liver 3. Kidney, Liver, Lung, Heart, Pancreas 4. Kidney, Liver 25% 25% 25% 25% Kidney, Pancreas, Intestine Kidney, Heart, Liver Kidney, Liver, Lung, Heart, P... Kidney, Liver
How long is the cohort time frame the PAIS uses to review program outcome performance? 1. 1.5 years 2. 2.5 years 3. 3 years 4. 4 years 25% 25% 25% 25% 2.5 years 4 years 1.5 years 3 years
What does the PAIS consider when reviewing a program? 1. Personnel changes 2. History of outcomes/inactivity 3. Raw data 4. SRTR data 5. All of the above 1 1 1 1 1 Personnel changes History of outcomes/inactivity Raw data SRTR data Previous peer visit
A stand alone pediatric hospital will be identified for functional inactivity if a transplant has not been performed in the last 1. 3 months 2. 6 months 3. 9 months 4. 1 year 25% 25% 25% 25% 3 months 6 months 9 months 1 year
Performance Review Staff Sharon Shepherd Transplant System Performance Manager 804-782-1512 Sheila Foster Performance Analyst Regions 1, 3, 4, 8, 9 804-782-4094 Robyn Zernhelt Performance Analyst Regions 5, 10, 11 804-782-4815 JoAnn Malone Performance Analyst Regions 2, 6, 7 804-782-2975 Crystal Lewis Performance Coordinator 804-782-4887
Questions
UNOS Celebrating 30 Years