EPAs Making Sense of the Milestones Michael S. Beeson, MD, MBA; Jonathan Fisher, MD; Danielle Hart, MD

Transcription

1 EPAs Making Sense of the Milestones Michael S. Beeson, MD, MBA; Jonathan Fisher, MD; Danielle Hart, MD The Milestones May Not Make Sense? Every program is struggling Milestones were intentionally written at a non granular level The issue: Translating resident and medical student actions in patient care to general levels of subcompetency proficiency Methods of Milestone Assessment End of rotation, end of month, end of six months End of shift Experiential End of Shift Evaluations Benefit is avoiding more subjective assessment not directly related to patient care Pretty Pictures But what of validity and reliability? We know the SDOT had acceptable reliability (LaMantia et al, 2009) Used as a model for other specialties Flashback CORD Academic Assembly, 2013 A video was shown of a resident evaluating an ankle injury End of shift forms were distributed, and each conference participant was asked to complete the evaluation based on observing the video How was the reliability? AWEFUL! Each of the eight end of shift forms, with 9 12 Milestones each, had poor inter rater reliability The expectation was that the conference participants were well versed in Milestones, and could complete an evaluation in a similar way Not! Why Not? Olle Ten Cate Nuts and Bolts of Entrustable Professional Activities March, 2013 issue of JGME EPAs are not an alternative for competencies, but a means to translate competencies into clinical practice. Competencies are descriptors of physicians, EPAs are descriptors of work. EPAs usually require multiple competencies in an integrative, holistic nature. How is This Significant?

2 Competencies are descriptors of physicians, EPAs are descriptors of work End of Shift Evaluations using actual Milestones are attempting to translate patient care into the Milestone subcompetency descriptor of the physician As supervising physicians we are used to evaluating a specific patient presentation of a specific clinical condition! Rather than evaluating the specific patient care as presented, we are trying to translate that into subcompetency proficiencies, or descriptors of the physician rather than the specific patient care given Requires significant faculty training Enter EPAs Entrustable Professional Activities Examples Pediatrics Well child checkup at 12 months Where are they on the growth charts Milestones met Home situation Smoking environment Vaccinations Diet Can include direct observation with patient presentation, documentation of visit Emergency Medicine EPAs Granularity of EPAs Example sore throat EPA vs manages an unstable patient EPA Potential Sore Throat EPA checkoff items: Introduces self to patient and verifies patient Obtains focused history and physical examination Ascertains allergies Applies Centor criteria or other scoring system to determine need for testing and treatment Presents the history and physical examination in an organized manner Can list a differential diagnosis for pharyngitis, including those with potential morbidity and mortality If indicated, is able to choose an appropriate antibiotic for the treatment of streptococcal pharyngitis. Can choose an alternative antibiotic based on allergy history Designs an appropriate disposition and follow up If other patients being evaluated, is able to manage their needs as well Demonstrates empathy towards the patient Ascertains patient s expectations of ED visit and whether they were fulfilled

3 Can identify areas of weakness in the evaluation and care of the patient with pharyngitis Based on the patient s presentation of pharyngitis provides cost effective care Documents the patient s visit completely Checkoff Items Can Be Mapped to Milestones Pharyngitis EPA KSA Milestone Subcompetency Milestone(s) affected Proficiency Level Introduces self to patient and verifies patient Obtains focused history and physical examination Ascertains allergies Applies Centor criteria or other scoring system to determine need for testing and treatment Patient Centered Communication (ICS1) Performance of Focused History and Physical Exam (PC2) Pharmacotherapy (PC5) Emergency Stabilization (PC1) Establishes rapport with and demonstrates empathy toward patients and their families Performs and communicates a focused history and physical exam which effectively addresses the chief complaint and urgent patient issues Consistently asks patients for drug allergies Discerns relevant data to formulate a impression and plan

4 Diagnostic Studies (PC3) Determines the necessity of studies; or Orders appropriate studies 1 or 2 Beauty of EPAs As supervising physicians we are used to evaluating clinical care People can keep doing what they are doing. Databases can translate the checkoff items into specific Milestone subcompetency proficiency levels Faculty do not need to! Faculty do not need to have a working knowledge of Milestones to evaluate residents! They can continue what they are doing, supervising residents and medical students in terms of clinical care being provided with that assessment How Many Successful Sore Throat EPAs? When can you entrust the resident to care for patients who present with sore throat? In other words, they will consider Ludwig s angina, peritonsillar abscess, retropharyngeal abscess, stomatitis, thrush, etc.? How Do You Decide When to Trust Your Residents To Do Something Independently? Can Depend on Many Factors Supervisor Trainee Supervisor Trainee Relationship Context Task You are taking a calculated risk OTC To Incorporate EPAs into Emergency Medicine A library is needed Ten Cate advised no more than 20 or so EPAs Mapping of checkoff items to Subcompetency Milestones and proficiency levels Incorporation into databases, RMS

5 Provide useful information to CCC for informing Milestone proficiency placement Task Force within CORD is working on these EM EPAs Nuts and Bolts of EPAs (OTC) Title Description Required Competencies Required KSAs How assess progress Expected progression Formal decisions Granularity of EPAs Broad v Narrow Lump v Split How many EPAs Generalizability Ease of Assessment Example EPA from Task Force EPA KSAs Milestone Map KSAs in this EPA evaluates and manages the undifferentiated unstable patient are grouped into Data gathering, data synthesis, initial management, etc. Any EPA with a KSA list longer than 10 was split up into subsections for ease of evaluation Each EPA will be evaluated YES/NO If answer is Yes, no further evaluation required (aside from L5 items or areas of future improvement) this enables expert educators and assessors to still use their pattern recognition skills

6 If answer is No, must unpack (aka. evaluate) ONE section of the KSAs (ie. if you felt the data synthesis was the issue, you d pick up that evaluation and fill it out) o Level 1: Not allowed to do the EPA = NOT ASSESSING THIS o Level 2: Allowed to do with full close supervision o Level 3: Allowed to do with distant reactive supervision o Level 4: Able to do EPA independently = YES DEFAULTS TO THIS Future Plans of the Task Force Finalize Current EPA List Feedback from Reactor Panel Feedback from CORD members Final Revisions Release Product to CORD members & Public Pilot / Study Assessment Tools Using EPAs