Effective Feedback & Meaningful Evaluation: Focus on Procedures

Transcription

1 Effective Feedback & Meaningful Evaluation: Focus on Procedures Inis Jane Bardella, M.D., FAAFP Associate Dean for Faculty Development and Global Health Initiatives Professor Department of Family and Preventive Medicine Chicago Medical School Rosalind Franklin University of Medicine and Science North Chicago, IL USA 1

2 Objectives Discuss the relationship between feedback, evaluation and learning procedures. Discuss a framework [RIME, Reporter Interpreter Manager Educator] for feedback and evaluation. Discuss the process of feedback and evaluation for procedural competency. Apply the RIME framework to achieve effective [behavior changing] feedback and meaningful [accurate, useful, efficient] evaluation of students and residents. Apply a model for procedural evaluation in RIME. 2

3 Vision Effectively prepare medical students and residents to meet the health needs of the people of the United States [especially the poor] with compassion, integrity, high ethical standards and a high level of competence. 3

4 Responsibility Patients, always first Students and Residents Faculty Clinical teachers you Clerkship director Program director Education system Advocate CMS 4

5 Education Process PIE Cycle Evaluation Planning Instruction 5

6 Background Adult Learning and Teaching What is different about adults? Self-directed Goal-directed Application-oriented Competency concerns Shared responsibility Moral responsibility Teacher is: Resource, Facilitator, Mentor 6

7 Background Adult Learning and Teaching How should this influence feedback? How should this influence evaluation? 7

8 Definition - Evaluation Process of making judgments based on factual information and observations in order to rate, rank or assess an individual s status at a given point in time. 8

9 Bottom Line of Evaluation Evaluation drives learning. Objectives determine evaluation. Evaluation is the second greatest fear of preceptors/clinical faculty. Evaluation is absolutely necessary to determine progression toward independent practice. 9

10 So, why formal evaluation? Summarize performance Determine competencies (behaviors) Guide future learning Communicate summary information Determine a grade Determine ability to practice independently How do we get there? Observation Feedback 10

11 Direct Observation Benefits Communicates you are interested. Establish baseline of learner s abilities. Confirm learner importance to the patient. Identify learner strengths and weaknesses. Reinforces active teaching role, that evaluation is not your only role. Enables evaluation. 11

12 Procedures Basic Reminders Identify the procedures that need to be observed and demonstrated. Essential vs non-essential Technique for independent practice Before procedure state purpose, describe procedure. During procedure point out landmarks, fine points. After demonstration discuss complications, other applications. After the demonstration summarize relevant points. 12

13 Procedures Learning Process Describe the procedure. Demonstrate and re-describe the procedure. Learner describes the procedure. Learner demonstrates the procedure. Summarize relevant points. 13

14 Definition - Feedback Provision of information to keep one on target. Provision of information to the performer about the performance. 14

15 Must be: Timely Specific Bottom Line of Feedback Descriptive Must include an action plan. Feedback is the greatest fear of preceptors / clinical faculty. Feedback is absolutely necessary to prevent failure and promote progress toward independent practice. 15

16 Feedback Techniques Video recorder what you see You palpated the liver edge gently, yet accurately. Personal reaction your emotions, tempered I feel frustrated when you are not prepared. Prediction of outcomes describe consequences If you begin to palpate the liver edge from the right costal margin, you may miss the inferior aspect of an enlarged liver. 16

17 Feedback Techniques Solicit learner self-assessment first. Maintain eye contact. Share information. Focus on what you (or others) have observed. Identify positives and negatives. Limit the amount (1-2 items, especially if negative). 17

18 Feedback Techniques Summarize Develop an action plan for change. Shared responsibility Follow-up Reassess Revise action plan 18

19 Feedback Examples Specific Description Lack of undermining, after excision of a skin lesion, can lead to a number of complications. Action Plan The tips of your iris scissors are used to blunt and sharp dissect the skin from the subcutaneous tissues. This mobilizes the skin layers to permit accurate approximation of the wound edges. What is/are the feedback techniques? 19

20 Baylor Family Medicine Kelsey-Seybold Clinic Comprehensive Procedure Education and Evaluation Workshops Logs Minimum of number of each procedure Evaluation based on ACGME competencies Elliott T. Development of a competency-based family medicine residency ambulatory procedural skills training program. Program Directors Workshop. August accessed July 30,

21 Baylor Family Medicine Kelsey-Seybold Clinic 1. Patient Care Performing a history Performing appropriate physical examination Appropriate gathering and interpretation of information Patient counseling Discusses with patient the prior therapies and alternatives to the procedure Informed consent discussed and obtained Proper execution /technique [Bardella] Proper, accurate technical skill [Bardella] 21

22 Baylor Family Medicine Kelsey-Seybold Clinic 2. Medical Knowledge Understands and verbalizes indications and contraindications for procedure, along with the risks and benefits Outlines steps of the procedure Appropriate equipment selection Appropriate patient preparation Appropriate anesthesia selection Proper execution /technique [Elliot] Recognition of pathology Understands how to manage complications Appropriate outcome Aftercare and Follow up 22

23 Baylor Family Medicine Kelsey-Seybold Clinic 3. Practice-Based Learning & Improvement Evaluates performance and follows through with needed improvements Develops a team approach with nurses/medical assistants 23

24 Baylor Family Medicine Kelsey-Seybold Clinic 4. Interpersonal & Communication Skills Demonstrates therapeutic relationship with patient and family Provides thorough explanations to patient throughout procedure Considers patient comfort throughout procedure Positions patient appropriately and respectfully Answers patient s questions effectively Communicates with team/assistant effectively 24

25 Baylor Family Medicine Kelsey-Seybold Clinic 5. Professionalism Accepts responsibility Demonstrates sensitivity to cultural issues, age, and disability Acts in best interest of the patient 6. Systems-based Practice Demonstrates understanding of practice Advocates for best quality care and most cost-effective approach 25

26 RESIDENT TO COMPLETE: PROCEDURE EVALUATION GENERAL PROCEDURE Resident: PGY: Date: Procedure: Patient name and chart number: ATTENDING PRECEPTOR TO COMPLETE: Poor Excellent Presents concisely but completely the relevant history and appropriate indications for procedure: Obtains informed consent including indications, details of the procedure, potential risks and associated complications: Demonstrates adequate comfort and knowledge of instruments: Used with permission. Patterson D. Implementation of a 3-year competency-based procedures curriculum STFM Annual Conference. accessed July 30, Demonstrates the following skills: : : : : Sterile technique: Personal protective measures: Recognized patient discomfort Adequate information exchanged with patient during and after procedure: Appropriate treatment/surveillance plan made: Resident able to perform this procedure independently? Yes No The resident could use more work on the following: Comments: Preceptor signature Resident signature 26

27 Framework RIME Reporter Interpreter Manager Educator 27

28 Framework RIME Reporter Consistently professional, good interpersonal skills Reliably obtains and communicates clinical findings Ownership of clinical findings Interpreter Consistently able to analyze and prioritize patient problems Ownership of explaining things 28

29 Manager Framework RIME Consistently proposes reasonable options incorporating patient preferences Ownership of developing action plan with patient Educator Consistent level of knowledge of current medical evidence Can critically apply knowledge to specific patients Ownership of evidence for action and sharing 29

30 Application RIME What is the RIME? How did you determine the RIME? 30

31 Framework RIME Synthetic model synthesis What we do all day long in patient care! Focus is patient performance in relationship to patient care Professionalism required High level of interpersonal skills required Descriptive Progressive, building model Competencies (KSA, behaviors) integral Not relative Expectations clear 31

32 RIME Caveats Students must be reasonable, residents must be right. Program must determine common / core / essential content and uncommon/non- essential content. Not focused on technical skills. Framework for assessing procedure skills. Decision making is foundational. Technical skill is essential. 32

33 Application What is the RIME? How will you determine the technical competency? What is the procedural competency? What are your feedback and comments? 33

34 Summary Evaluation is a continuous process. Evaluation drives learning. Goals & objectives determine evaluation. Feedback keeps the learner on target to achieve goals and objectives. Observation is required for feedback. Procedural competency includes technical and decision making skills. 34

35 RIME it. Summary Framework to assess behavior, e.g. competency. Procedure competence assessed within RIME framework. Develop a plan for progression. Include comments. End goal is independent practice. 35

36 References Bardella IJ, ed. Essentials of precepting , revised Bardella IJ, ed. Giving Bad News. August 1999; revised September 2000; revised November 2001; revised December Elliott T. Development of a competency-based family medicine residency ambulatory procedural skills training program. Program Directors Workshop. August accessed July 30, Ende, J. Feedback in clinical medical education. JAMA. 1983;250: Massachusetts statewide area health education center program. Faculty development workbook Pangaro LN. A new vocabulary and other innovations for improving descriptive in-training evaluations. Acad Med. 1999;74:

37 References Pangaro LN. Evaluating trainees in the clinical setting: what does competency in patient care look like? Presented 25 Jan 2012 at RFUMS, North Chicago, IL. Patterson D. Implementation of a 3-year competency-based procedures curriculum STFM Annual Conference. accessed July 30, Sepdham D. Julka M. Hofmann L. Dobbie A. Using the RIME model for learner assessment and feedback. Fam Med. 2007;39(3): Society of teachers of family medicine pep2 committee. Preceptor education project, second edition: facilitators guide. Society of teachers of family medicine. Kansas City Stephens MB. Gimbel RW. Pangaro L. The RIME/EMR scheme: an education approach to clinical documentation in electronic medical records. Acad Med. 2011;86:

38 Thank you evelopment/facultydevelopment.aspx 38