SCARCE: Stewardship Curriculum and Audit for Residents to Cultivate Efficiency The development and assessment of a novel CanMEDS Manager curriculum to improve resident test-ordering efficiency Andrew Burke, Chris Hillis, Jason Cheung, John Neary, Leah Hillier, Shuoyan Ning, Carol Mantle, John You, Parveen Wasi
Disclosures I do not have an affiliation (financial or otherwise) with a pharmaceutical, medical device or communications organization.
Introduction to SCARCE CanMEDS Manager Role 1 : Allocate finite health resources appropriately Consider efficiency alongside effectiveness Need for Teaching Waste Reduction to Residents: 1/3 of all healthcare costs non-contributory to care 2 Diagnostic testing is most wasteful area: >54%! 3 Current Resident Test-Ordering Practices: Efficient practice is not highly valued Little formal instruction on appropriate practices 4 1. Frank JR, et al. The CanMEDS 2005 physician competency framework. Better standards. Better physicians. Better care. Ottawa: The Royal College of Physicians and Surgeons of Canada; 2005 2. Institute of Medicine. The Healthcare Imperative: Lowering Costs and Improving Outcomes. Washington, DC: National Academies Pr; 2010. 3. Miyakis S et al. Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy. Postgrad Med J 2006; 82:823 829. 4. Dine CJ, et al. Educating physicians-in-training about resource utilization and their own outcomes of care in the inpatient setting. J Grad Med Educ 2010; Jun;2(2): 175-80.
Unnecessary Investigations at HHS 100 90 80 70 Percentage of Unnecessary Investigations (%) 60 50 40 30 20 10 0
Goal of Study Clinical Question Will the SCARCE Curriculum and Audit reduce unnecessary investigations ordered by housestaff on the Clinical Teaching Unit? Key Points Explain the SCARCE Curriculum Methodology of double-blind prospective controlled trial Impact of SCARCE on practice patterns and healthcare costs
Methodology
Session 1: Appropriate Use of Investigations Prospective Cohort Trial Large Group Format 60 minutes Session 2: Investigation Practices Feedback Audit Small Group Format 60 minutes Pre (Wk 1) Intervention (Wk 2) Post (Wk 3) Hamilton General CTU Housestaff (CC, PGY1, PGY2) Monitor Investigations SCARCE Curriculum and Audit Monitor Investigations Juravinski Hospital CTU Housestaff (CC, PGY1, PGY2) Monitor Investigations Usual Teaching Monitor Investigations
SCARCE Curriculum Appropriate Use of Investigations Clinical reasoning to increase yield Mrs. EV: 68F w DVT on LMWH LOS 64 days = 71 Normal INRs Costs of investigations to system Case-based examples and quality improvement
SCARCE Audit Feedback on Investigation Practices Trained clinician reviews housestaff orders Assessment of whether investigations are necessary Peer-review and reflection
Outcome Assessment Investigation Adjudication Unnecessary Investigation 5,6 : Admission and Progress Notes Evidence-Based Criteria for Investigations Reviewers given all documentation available at time of order Highly unlikely to contribute to diagnosis Highly unlikely to change management Redundant or unreasonable frequency Unreasonably low pre-test probability Admission and Ward Investigations Risk of investigation outweighs benefit Duplicated assessment: Blinded Senior Resident and Staff Physician 5. van Walraven C & Naylor D. Do we know what inappropriate laboratory utilization is? JAMA. 1998;280:550-558. 6. Verstappen W et al. Effect of a pactice-based strategy on test ordering performance of primary care physicians: a randomized trial. JAMA. 2003; 289:2407-2412.
Results
Baseline Characteristics Housestaff Characteristics SCARCE Site Control Site Total no 18 18 Clinical Clerks no (%) 5 (29) 6 (35) PGY-1 no (%) 9 (53) 8 (47) PGY-2 no (%) 3 (18) 3 (18) Patient Characteristics Total number 151 132 Age - yr 68.1 74.2 Female (%) 43.5 51 Mean Day of admission 4.8 4.6
Investigations Reviewed 283 Admitted Patients 78 Pts < Day 3 of Admit 205 Eligible Patients 64 Patients with Investigations 264 Investigations Electrolytes 35 Creatinine 29 CBC 26 Urea 23 Calcium 20 Magnesium17 Phosphate16 Albumin16 AST/ALT 13 Chest X-ray 11 Misc 60
Housestaff Investigation Ordering Practices: SCARCE vs. Control 75 70 65 60 Percentage of Unnecessary Investigations 55 50 45 40 Week PRE 1 Week 3 35 30 25 Control SCARCE Curriculum Received by Housestaff
Housestaff Investigation Ordering Practices: SCARCE vs. Control 75 70 *Absolute decrease of 13.9% (4.2-19.5) 65 60 Percentage of Unnecessary Investigations 55 50 45 40 * Week PRE 1 Week POST 3 35 30 25 Control SCARCE Curriculum Received by Housestaff *p<0.01
Investigation Profile of SCARCE Housestaff 7 6 5 Investigations/ Patient-Day 4 3 * Week PRE 1 Week 3 POST 2 * 1 0 Total Unnecessary Necessary Investigation Type
Discussion Cost Implications of SCARCE: Direct savings of $24.51/patient-day Hamilton Health Sciences: >$2,000,000 annually Starting a Culture of Efficiency and Cost Control: Educating trainees to allocate finite resources Collaborating with a wide group of stakeholders Limitations to SCARCE Trial: Documentation-based reviews Length of follow-up (future study)
Acknowledgements SCARCE Collaborators: Dr. Parveen Wasi Dr. John You Dr. John Neary Carol Mantle Dr. Chris Hillis Dr. Jason Cheung Dr. Shuoyan Ning Leah Hillier Funding: MOHLTC Collaboration Initiative Statistical Support: CLARITY Research Group Dr. Eleanor Pullenayegum Dr. Stephen Walter
Thank you!
Classification of Unnecessary Investigations 5,6 Excessive Frequency/Redundancy eg. Repeat LFTs after normal day 1 & 2 Very Low Pre-test Probability eg. Urea if volume & kidneys normal No Change to Management eg. PTT if on warfarin No Change to Diagnosis eg. D-dimer in high suspicion PE
Agreement Between Independent Assessors Pair A Pair B Proportion Agreement 0.74 0.80 Kappa 0.43 0.60 Bias 0.01 <0.01
References 1. Institute of Medicine. The Healthcare Imperative: Lowering Costs and Improving Outcomes. Washington, DC: National Academies Pr; 2010. 2. Miyakis S et al. Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy. Postgrad Med J 2006; 82:823 829. 3. Reinhardt UE. Fees, volume and spending at Medicare. 24 December 2010. Accessed at http://economix.blogs.nytimes.com/2010/12/24/fees-volume-and-spending-atmedicare/ on May 13, 2012. 4. Dine CJ, et al. Educating physicians-in-training about resource utilization and their own outcomes of care in the inpatient setting. J Grad Med Educ 2010; Jun;2(2): 175-80. 5. van Walraven C & Naylor D. Do we know what inappropriate laboratory utilization is? JAMA. 1998;280:550-558. 6. Verstappen W et al. Effect of a practice-based strategy on test ordering performance of primary care physicians: a randomized trial. JAMA. 2003; 289:2407-2412.
Future Directions Longer Term: SCARCE Effectiveness at week 8 Crossover CTU Sites Implement SCARCE into Regular Curriculum Faculty Investigation Ordering Practices Faculty Development
Determining Unnecessary Investigations Highly unlikely to contribute to diagnosis Highly unlikely to change management Redundancy or unreasonable frequency Unreasonably low pre-test probability of abnormal result Risk of investigation greatly outweighs benefit
Results Investigation Profile Before and After SCARCE SCARCE Week 1 SCARCE Week 3 P value Control Week 1 Control Week 3 P value Investigations 90 63 57 59 Investigation/pt/day 5.63 3.32 0.02 5.18 4.46 0.89 Unnecessary inv/pt/day 3.44 1.36 0.01 3.18 2.54 0.80 Necessary inv/pt/day 2.19 1.96 0.42 2.00 1.92 0.92 SCARCE resulted in a significant decrease in total investigations per patient per day This change was driven by a decrease in unnecessary investigations