Mastery Learning of Clinical Skills

Size: px

Start display at page:

Download "Mastery Learning of Clinical Skills"

Louisa Newton
8 years ago
Views:

1 2012 ICRE Conference Northwestern University Feinberg School of Medicine Mastery Learning of Clinical Skills William C. McGaghie, PhD Diane B. Wayne, MD

2 No Conflicts of Interest

3 Objectives Recognize variation in clinical skill among experienced professionals Review principles of mastery learning and deliberate practice Describe advantages of mastery learning with deliberate practice over traditional educational strategies Design a mastery learning program for a sample clinical skill

4 How will we get there? Use clinical skills data to highlight problems with traditional training Discuss how mastery learning is better than traditional training Review steps to develop a mastery learning curriculum

5 Traditional Training Competence

6 Most Education Interventions Pretest Training Posttest

7 Typical Educational Outcomes P<.001

8 Mastery Learning: A Better Approach Pretest Lecture, video, demonstration and deliberate practice Posttest FAIL PASS

9 Mastery Learning Educational Outcomes

10 Mastery Learning Clinical Outcomes Central Venous Catheter Insertion

11 Mastery Learning Outcomes % Correct (5.1) 97.4 (3.5) Pre 40 Post (23.4) 48.3 (26.8) MPS 10 0 Internal Jugular p<.005 Subclavian Barsuk et al. Critical Care Medicine 2009

12 Clinical Outcomes: Infections Barsuk et al. Archives of Internal Medicine 2009

13 T3 OUTCOMES: BENEFITS TO SOCIETY Outcomes: Cost Effectiveness The total annual estimated savings were approximately $820,000, 139 patient hospital days, and 120 MICU days When compared with the cost of our intervention ($112,000), the net savings was approximately $708,000 (a 7:1 rate of return) Cohen et al. Simulation in Healthcare 2010

14 How will we get there? Use clinical skills data to highlight problems with traditional training Discuss how mastery learning is better than traditional training Review steps to develop a mastery learning curriculum a. Deliberate Practice b. Rigorous Assessment

15 Goal: Educate Superb Clinicians Effective & Safe Quality Patient Care Good Patient Outcomes

16 Key Medical Education Research Concepts Mastery Learning Deliberate Practice

17 Mastery Learning Features 1. Baseline, i.e., diagnostic testing; 2. Clear learning objectives, units ordered by difficulty; 3. Educational activities (e.g., deliberate skills practice) focused on objectives; 4. Minimum passing mastery standard (MPS) for each unit; 5. Formative testing mastery of each unit; 6. Advancement if performance MPS; or 7. Continued practice or study until MPS is reached 8. Time varies, outcomes are uniform McGaghie et al., Chest 2009

18 Deliberate Practice (DP) Features 1. Highly motivated learners with good concentration; 2. Engagement with a well-defined learning objective or task; at an 3. Appropriate level of difficulty; with 4. Focused, repetitive practice; that leads to 5. Rigorous, precise measurements; that yield 6. Informative feedback from educational sources (e.g., simulators, teachers); and where 7. Trainees also monitor their learning experiences and correct strategies, errors, and levels of understanding, engage in more DP; and continue with 8. Evaluation to reach a mastery standard; and then 9. Advance to another task or unit 10. Goal: constant improvement Ericsson Acad Med. 2004; McGaghie et al., Chest 2009

19 19

20 Checklist Development 1. Define task 2. Review literature 3. Draft checklist 4. Review by experts 5. Pilot test with feedback 6. Revise checklist 7. Calibrate checklist 8. Revise checklist periodically review and revise Stufflebeam D. Guidelines for Developing Evaluation Checklists

21 Angoff Setting an MPS Define a group of borderline trainees who have a 50% chance of passing Each judge estimates the proportion of borderline trainees who would correctly perform each item Hofstee Each panelist records judgments about: Minimum acceptable passing score Maximum acceptable passing score Minimum acceptable failure rate Maximum acceptable failure rate Avoid seat of the pants approaches Norcini Med Educ 2003, Downing et al.; Teach Learn Med. 2006

22 Rigorous Outcome Measurement Needs Reliable Data Valid Decisions or Judgments Uses Feedback to Learners Research Progress Norcini, Med Educ 2003

23 Six Lessons Learned 1. ML with DP is a key feature of SBME programs aimed at boosting skill and knowledge acquisition among medical learners at all levels strong, integrated, sustained 2. Robust measures yielding reliable data that permit valid decisions or inferences are a key feature of SBME 3. Rater training and constant calibration are essential 4. Self-assessments are biased, poor relation to performance measured objectively 5. ML with DP cannot be done on the cheap 6. Follow-up testing and remediation are needed to assure retention McGaghie (2008)

24 Design your own Mastery Learning curriculum Learning objectives Teaching method (i.e. simulators) Skills checklist Rater training Lecture, videos, demonstrations Deliberate Practice Standard Setting Educational and Clinical Outcomes

25 Potential topics IV insertion Foley catheter insertion Phlebotomy Incision and drainage of wound Suturing Code blue team response Patient handoffs Code status discussion Patient presentation skills Come up with your own!

26 Presentations

27 Putting it all together: Intern boot camp Interns report 3 days early for orientation Participate in teaching and assessment Must meet minimum standard to start training - Cardiac auscultation - Paracentesis - Lumbar puncture - ICU clinical skills - Code status discussion - Handoffs

28 Historical Controls Boot Camp-Trained Interns MPS 100% 90% 80% Clinical Skills Examination Scores 70% 60% 50% 40% 30% 20% 10% 0% Cardiac Auscultation Paracentesis Lumbar Puncture ICU Clinical Skills Code Status Discussions Cohen et al Acad Med 2013, In press

29 References Barsuk JH, McGaghie WC, Cohen ER, O'Leary KJ, Wayne DB: Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009;37: Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB: Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med. 2009;169: Cohen ER, Feinglass J, Barsuk JH, et al: Cost savings from reduced catheterrelated bloodstream infection after simulation-based education for residents in a medical intensive care unit. Simul Healthc. 2010;5: Downing SM, Tekian A, Yudkowsky R. Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teach Learn Med. 2006;18:50-57 Stufflebeam D: Guidelines for developing evaluations checklists: the checklist development checklist (CDC). Available at:

Ralph P. Leischner, Jr. MD Institute for Medical Education Competency-Based Medical Education: Origins and Prospects

Ralph P. Leischner, Jr. MD Institute for Medical Education Competency-Based Medical Education: Origins and Prospects William C. McGaghie, PhD No Conflict of Interest Competency-Based Medical Education: