Workplace Learning The Evidence for Teaching, Learning and Assessing in the Workplace Karen V. Mann PhD 1
Overview 1. What are the assumptions of WL? 2. What are the advantages and limitations? 3. What is the role of the teacher in WL? 4. What is the role of the learner in WL? 5. Future directions 2
Workplace learning Broad term, covers a range of topics Young and old Evolving our understanding of apprenticeship 3
Workplace Learning Deliberate learning Physicians need to go beyond the mere accumulation of experiences (van de Wiel et al., 2011, p. 83) Often overlooked as a valuable tool Essential for the improvement of trainees performance Life long practice 4
Assumptions 1. Experience 2. Supervision 5
Experience Empowering and encouraging environment Trainee must feel supported to engage in experiential opportunities (van der Zwet et al., 2010) Opportunities to participate (e.g., clerkships) in environments that challenge the trainee and provide quality encounters (Eraut, 2011; Pearson & Lucas, 2011; van der Zwet et al., 2010) Participation is motivating and encourages feeling like a doctor (Dornan et al., 2007) 6
Motivation in the Learning Environment: An Example Experience-based learning in undergraduate students Participation was central to students learning Supported participation Affective Pedagogic Organizational Participation at all levels: Boosted confidence Increased motivation Enhanced professional identity Increased confidence (Dornan, et al, 2007) 7
Experience We have definitely learned a lot from it that will be useful in our final exams and beyond. The downside of being here...it would be nice to have a little bit more clinical experience on our own, rather than just sitting in with the GPs. (Fifth-year medical student) (Pearson & Lucas, 2011) 8
Experience Learning from the problems encountered (van de Wiel et al., 2011) Clinical learning occurs through engagement with many professional groups and levels of experience through a respectful manner (Pearson & Lucas, 2011) 9
Experience Several factors influence experience in workplace learning Enthusiasm and commitment of teacher Presence of clear objectives and guidance Patient cases / clinical work Immediate problems vs. gaps in skills/knowledge Cooperation of colleagues, supervisors and other specialists Recognition and respect when contributing to the team The quality of supervision (r = 0.73, p <.01), patient mix (r = 0.65, p <.01), and independence (r = 0.48, p <.01) impact instructional quality (van der Zwet et al., 2010) 10
Adequate supervision a. Assessment -Supervisory assessment and self-assessment b. Observation -Mini-clinical evaluation and direction observation of skills c. Discussion -Clinical cases, case based discussion d. Feedback -Peers, coworkers & patients; multi-source feedback; objective tools- mini-pat, TAB & PSQ 11
Assessment Evaluation of skills and practices undertaken in day-to-day work Assessment supports development of competencies Competency does not always predict performance Trainees concern about meeting expectations of teachers (Eraut, 2011) 12
Assessment How does assessment change learning? Credibility and Relevancy Multiple sources of feedback for trainees to reflect upon Certainly the more respect I had for them the more negative feedback and the more straightforward feedback I could tolerate from them. (P6) after I told the patient the bad news, then the family told me, You seem cold or kind of not compassionate. I took that very seriously (P3) (Watling et al., 2012) 13
Observation Direct observation provides the data for feedback (Ryan et al., 2010) Faculty, Student and System issues can influence the ability to observe or be observed (Kogan et al, 2011) Faculty need support to develop skill in observation and feedback (Fromme et al., 2009) Observation and feedback requires planning and intention 14
Discussion Consulting with colleagues/peers on a case Incidental learning Who to approach? Coming to a common ground on differences of opinions Opportunities to learn to talk, and learn from talk (Lave and Wenger,1991) 15
Discussion Difficult cases discussed in daily or special review meetings help solve medical problems and provides alternative solutions for trainees I see many routine cases. Occasionally you have to look things up. Occasionally it s good to test your own opinion against those of others. That s what the Friday afternoon patient review meetings are for, our strength is that we re a team; by talking to each other, we improve our level. (van de Wiel et al., 2011) 16
Feedback Providing Feedback Critical in guiding/adjusting the trainee to desired outcomes and positive impact on doctors learning & performance (Eraut, 2011; Miller, 2010; Norcini & Burch, 2007; van de Wiel et al., 2011) Consistent feedback from a credible source can change clinical performance (Veloski et al., 2006; Watling et al., 2010) 17
Feedback Receiving feedback Multisource feedback is the most evaluated form of feedback, but there is conflicting evidence on its effectiveness (Miller, 2010) Effectiveness of feedback for the learner depends on credibility of advice and frequency of negative feedback (Eraut, 2011) Integrating/evaluating is dependent on who is providing the feedback (e.g. Patient vs. Peer) Feedback that is incongruent with self-assessment may be rejected and evoke strong emotional response (Sargeant et al, 2009) Patients rated doctors significantly higher on performance than their colleagues (4.34 vs. 3.69, t(df, 66)=7.7, p < 0.001) (Archer & McAvoy, 2011) 18
Feedback Seeking & incorporating feedback Learning and change can occur when feedback indicates a need for change Positive interpretation of feedback and believing change is possible improves learning and growth Most often sought when immediate care needs to be provided (van de Wiel et al., 2011) 19
Tensions in seeking and using feedback Between people Wishing to learn and improve vs. wishing to appear knowledgeable and confident among peers and superiors Tensions in learning environment Providing genuine assessment feedback vs. playing the evaluation game Tensions within self Wanting feedback yet fearing disconfirming information (Mann et al, 2011) 20
Advantages of Workplace Learning Contributes to professional and personal growth, and increases confidence in ability (Lester & Costley, 2010) Benefits the organization Allows for learning at both the individual and the collective level (Billett 2004) 21
Limitations of Work-base learning Reciprocal relationship between environment and learner Often no definitive guide for facilitating WBL Assessment and feedback may be influenced by interpersonal relationships (Norcini & Burch, 2007; Watling et al., 2010) Faculty development, and learner support are key to success 22
Role of the Teacher Offer comfortable and safe working and learning environments multiple opportunities to engage in practice meaningful and emotional encounters with patients observation purposeful and relevant feedback trust and value in trainees competencies and skills care, commitment and enthusiasm for teaching 23
Role of the teacher [The GPs] will say you need to know that, go away and read that and it is nice because you do not often get that. People just go you should know this. Well there is a lot of things that I should know but to actually be given a title and certain key things to go away and learn is extraordinarily helpful. (Fifth-year medical student, Pearson & Lucas, 2011)
Role of the Teacher Encourage: critical thinking questioning and seeking answers knowledge sharing engagement in cases and learning opportunities checking/discussing mutual understandings (e.g., coming to a common solution) 25
Motivation in the Learning Environment: An Example How residents learn Feedback is essential to motivation Learning and performance goal orientation Instrumental and supportive leadership (Teunissen et al., 2009) 26
Preceptors Influence on Motivation Instrumental Leadership Supportive Leadership Clear Goals Structured Work Guidelines Friendly, Approachable Considerate More Feedback Seeking Increased Feedback Provision and Value More Feedback Seeking Increased Feedback Provision and Value (Teunissen, et al., 2009) 27
Informal Learning and the Hidden Curriculum Loss of idealism Adoption of a ritualized professional identity Emotional neutralization Change of ethical integrity Acceptance of hierarchy Learning of less formal aspects of good doctoring (Lempp & Seale, 2004) 28
The Hidden Curriculum Power and hierarchy Patient dehumanization Hidden assessments Emotional suppression Limits of medicine Emerging accountability Balance and sacrifice Faking it Human connection (Gaufberg et al., 2010) 29
How is Caring Learned? Professional and pedagogical caring A vehicle for integrating scientific and human aspects of professional practice Interactions between teachers and students Long term effects of caring (Cavanaugh, 2002; Haidet et al., 2006) 30
Faculty s Role in Teaching Humanistic Behaviour Faculty selected by residents as role models & teachers of humanistic care Observations revealed non-verbal behaviours: Demonstration of respect, overtly and frequently Building a personal connection Eliciting patients emotional responses to illness Demonstration of self-awareness and reflection (Weissman et al., 2006) 31
The Student-Supervisor Relationship The student-supervisor relationship functions to: Facilitate direct transmission of patient-centred knowledge, skills and attitudes Provide social support for the student s patient-centred behaviour Provide support of the student as person Mirror patient-centered behaviour by being learnercentred Address supervisor vulnerability (Bombeke et al., 2010) 32
Role of the learner Engage in WL to learn and to improve patient care Unsure of solution and/or gap in skills Practical experience triggers reflections and problem solving Self-assessment (Eraut, 2011; van de Wiel et al, 2011) Seeking feedback is related to trainees ability to adequately assess their own performance 33
Role of the learner Provide support to peers and encourage discussion and exchange of information Informal peer learning was strong and important in practice you get to compare your level of knowledge with your peers and also...you know, you can teach them and they can teach you as well... (Pearson & Lucas, 2011,) 34
Where Is the Learner s Voice? Residents in Family Medicine viewed receiving specific, timely, and frequent feedback as key to making the CBAS a worthwhile investment for them. Feedback encouraged their own self-assessment and motivation to fill gaps. Faculty development is critical Learner development and orientation are critical (Ross et al, 2012) 35
Future Direction Little work has been done to empirically assess the effectiveness of work-based learning on doctors performance and learning Need to understand what environments facilitate and support WBL Future work should look at incorporating mixed methodologies and focus on the quality of the data Literature is dominated by theoretical studies (van de Wiel et al., 2011) What they should do vs. what they actually do 36
Summary 1. What are the assumptions of WL? 2. What are the advantages and limitations? 3. What is the role of the teacher in WL? 4. What is the role of the learner in WL? 5. Future directions 37
Thank you! Your comments and questions?
References Archer, J. (2010). State of the science in health professional education: effective feedback. Med Teach, 29: 855-71. Archer, J. & McAvoy, P. (2011). Factors that might undermine the validity of patient and multi-source feedback. Med Edu, 45: 856-93. Billett S. (2004). Workplace participatory practices: conceptualising workplaces as learning environments. J Workplace Lear, 16:312 24. Bombeke K, Symons L, Debaene L, et al. (2010). Help: I m losing patient centredness! Experiences of medical students and their teachers. Medical Education, 44: 662-673. Cavanaugh SH. (2002). Professional caring in the curriculum. In Norman G, van der Vleuten CPM & Newble D (eds.) International Handbook of Research in Medical Education. Dordecht: Kluwer, pp. 981-996. Dornan T, Boshuizen H, King N, Scherpier A (2007). Experience-based learning: a model linking the processess and outcomes of medical students wokplace learning. Med Educ 41:84-91 Eraut, M. (2011). Informal learning in the workplace: evidence on the real value of work-based learning (WBL). Development and learning in Organizations, 25(5): 8-12. Fromme, H. B., Karani, R., & Downing, S. (2009). Direct observation in medical education: Review of the literature and evidence for validity. Mount Sinai School of Medicine, 76: 365 0 371. Lave J, Wenger E (1991). Situated Learning. Legitimate Peripheral Participation. Cambridge, UK: Cambridge University Press. 39
References Gaufberg L, Batalden M, Sands R & Bell SK. (2010). The hidden curriculum: what can we learn from student narrative reflections? Academic Medicine, 85: 1709-1716. Haidet P, Kelly PA, Bentley S, et al. (2006). Not the same everywhere: patient-centered learning environment of nine medical schools. Journal of General Internal Medicine, 21: 405-409. Lester, S. & Costley, C. (2010). Work-base learning at higher education level: value, practice and critique. Studies in Higher Education Lempp H, Seale C. (2004). The hidden curriculum in undergraduate medical education: qualitative study of medical students perception of teaching. British Medical Journal, 329: 770-773. Kogan, J., Conforti, L., Bernabeo, E., Iobst, W & Holmboe, E. (2011). Opening the black box of clinical skills assessment via observation: a conceptual model. Med Edu, 45L 1048-1060. Mann K, van der Vleuten C, Eva K, Armson H, Chesluk B et al. (2011). Tensions in informed selfassessment: how the desire for feedback and reticence to collect and use it can conflict. Acad Med 86:1120-1127. Miller, A. (2010). Impact of workplace based assessment on doctors education and performance: a systematic review. BMJ, 341(c5064). Norcini, J., & Burch, V. (2007). Workplace-based assessment as an educational tool: AMEE Guide No 31. Med Teach, 29: 855-71. Pearson, D., & Lucas, B. (2011). Engagement and opportunity in clinical learning: Findings from a case study in primary care. Medical Teacher, 33: e670-77. 40
References Ross, S., Poth, C., Donoff, M., Papile, C., Humphries, P. et al. (2012). Involving users in the refinement of the competency-based achievement system: An innovative approach to competency-based assessment. Med Teach, 34(2): e143-147. Ryan, J., Barlas, D., & Sharma, M. (2010). Direct observation evaluations by emergency medicine faculty do not provide data that enhance resident assessment when compared to summative quarterly evaluations. Academic Emergency Medicine, 17: S72-77. Teunissen P, Stapel D, van der Vleuten C, Scherpbier A, Boor K, et al. (2009). Who wants feedback? An investigation of the variables influencing residents feedback seeking behaviour in relation to night shifts. Acad Med 84:910-917. van der Vleuten, C (1996). The assessment of professional competence: development, research and practical implications. Adv Health Sci Educ, 1: 41-67. van de Wiel, M., van den Bossche, P., Janssen, S. & Jossberger, H. (2011). Exploring deliberate practice in medicine: how do physicians learn in the workplace? Adv in Heal Sci Educ, 16: 81-95. van der Zwet, J., Zwietering, P., Teunissen, P., van der Vleuten, C. & Scherpbier, A. (2010). Workplace learning in general practice: Supervision, patient mix and independence emerge from the black box once again. Med Teach, 32: e294-99. Veloski, J., Boex, J., Grasberger, M., Evans, A., & Wolfson, D. (2006). Systematic review of the literature on assessment, feedback and physicians clinical performance: BEME Guide No 7. Med Teach, 28: 117-28. Watling, C., Driesen, E., van der Vleuten, C., & Lingard, L. (2012). Learning from clinical work: the roles of learning cues and credibility judgments. Medical education, 46: 192-200. Weissman P, Branch W, et al. (2006). Role modeling humanistic behavior: leaning bedside manner from 41 the experts. Academic Medicine 81: 661-667.