Developing a qualitative approach to 360-degree feedback to aid understanding and development of clinical expertise

Transcription

1 Journal of Nursing Management, 2007, 15, Developing a qualitative approach to 360-degree feedback to aid understanding and development of clinical expertise ROBERT GARBETT MSc BN (Hons) RN PG Cert LL 1, SALLY HARDY EdD MSc RGN RMN 2, KIM MANLEY PhD MN BA RGN DipN (Lond) RCNT PGCEA CBE 3, ANGIE TITCHEN DPhil (Oxon) MSc MCSP 4 and BRENDAN MCCORMACK DPhil (Oxon) BSc (Hons) PGCEA RGN RMN 5 1 Research Fellow, Nursing Development Centre, University of Ulster and Royal Hospitals Trust, Royal Victoria Hospital, Belfast, 2 Associate Research Fellow, Royal College of Nursing Institute, London, 3 Director of Practice Development (paper guarantor), Royal College of Nursing Institute, London, 4 Senior Research and Practice Development Fellow, Royal College of Nursing Institute, London and 5 Director of Nursing Research and Practice Development, Nursing Development Centre, University of Ulster and Royal Hospitals Trust, Royal Victoria Hospital, Belfast, UK Correspondence Sally Hardy Deputy Director of NAMRU Institute of Health School of Nursing and Midwifery Edith Cavell Building UEA Norwich NR4 7TJ UK S.Hardy@uea.ac.uk GARBETT R, HARDY S, MANLEY K, TITCHEN A, McCORMACK B. (2007) Journal of Nursing Management 15, Developing a qualitative approach to 360-degree feedback to aid understanding and development of clinical expertise Aim This paper presents one aspect of a 5-year multicentre action research study to develop an accreditation process for clinical nursing expertise. Part of the process consisted of the exploration, critique and refinement of qualitative 360-degree feedback as a tool for peer review. Background Three hundred and sixty-degree feedback is widely used as a personal and professional development strategy. This part of the overall study challenged assumptions about the necessity for anonymity and structured questionnaires to collect data.the study involved 32 experienced clinical nurses drawn from a range of clinical settings supported by Ôcritical companionsõ (colleagues from clinical practice, education, management and research, recruited to provide supervision and support). Method(s) Study participants, facilitated by the project team (the authors), engaged in critiquing and refining 360-degree feedback as a process to help them examine and develop their practice. Conclusion(s) On the basis of our findings this approach to gathering 360-degree feedback facilitates the collection of evidence that aids professional development. There are indications that it may also contribute to improved working relationships. Keywords: 360-degree feedback, adult learning, expertise, practice development Accepted for publication: 20 January 2006 To optimize the usefulness of the process we recommend: Preparation is key; being clear with all involved about the purpose of the exercise and being explicit about the criteria used to identify role set members. Approaches to gathering information need to reflect what is convenient and appropriate to colleagues. Anonymity is not a necessary precondition to useful feedback. Introduction This paper describes a qualitative approach to 360- degree feedback developed as part of the Royal College 342 ª 2007 Royal College of Nursing. Journal compilation ª 2007 Blackwell Publishing Ltd

2 Commentary of Nursing s Expertise in Practice project (EPP) (Manley et al. 2005). There is a global concern with monitoring the impact of health care. This has given rise to a range of governance structures in both commercial and social healthcare systems. Within this climate of increased accountability, nursing has developed professionalization strategies that in turn aim to address chronic workforce shortages. For example, the emergence of new roles such as that of the consultant nurse (Manley 2000) has become a part of the far-reaching reform of career structure for allied health professions in the form of Agenda for Change (Department of Health 2005). The importance of accountability for performance over time is, in turn, apparent in the increasing importance of structures that help healthcare professionals demonstrate lifelong learning throughout their careers (Department of Health 2003, Nursing and Midwifery Council 2004). Strategies such as maintaining a portfolio of evidence of professional and personal development are an important aspect of a trend (Royal College of Paediatrics and Child Health 2000, Department of Health 2003, Nursing and Midwifery Council 2004) that increasingly draws on concepts from adult educational theory that emphasize the importance of learning from experience(higgs & Titchen 2001, Newman & Peile 2002). The Expertise in Practice project emerged as a contribution to these agendas. In particular, through its study aims (see Box 1), it has helped to develop strategies by which practitioners can critically examine their practice in such a way as to address both governance and developmental agendas. The project took the form of an emancipatory action research study (Grundy 1982). Over a 16-month period 32 nurses from six geographical areas across the UK recruited from a broad range of clinical specialities and settings worked with colleagues known as critical companions (Box 2) and in action learning sets meeting at monthly intervals in six centres across the UK. These mechanisms were used to help the nurses develop a portfolio of evidence to be accredited by the RCN based on plural sources of evidence: Box 1 Project aims To identify, recognize and value expertise in UK nursing practice To develop a recognition process for expertise in practice To develop further understanding of the concept of expertise in different specialities To explore the links between expertise and outcomes for service users and healthcare providers Box 2 Critical companions The term Ôcritical companionõ has been developed by Titchen (2001) to describe a form of helping relationship between colleagues in which one colleague with more expertise and/or experience in a particular aspect of practice uses a range of relationship and facilitative strategies to foster growth and learning in the other Self-assessment based on critical reflection using an evidence-based framework of nursing expertise (Manley & McCormack 1997) (Box 3); Feedback on observation of practice; Feedback from service users (this complex aspect of the project is reported elsewhere) (Manley et al. 2005, Hardy et al. 2006); Three hundred and sixty-degree feedback from colleagues using qualitative data collection methods. It is this last aspect of evidence collection that is dealt with here. Responses from colleagues in a range of disciplines lead us to believe that the approach is potentially useful to all practice-based professionals as a means of critically examining practice and fostering deeper and more productive professional relationships. The concept of peer review is now a part of medical career development (Rodgers & Manifold 2002, Department of Health 2003). While in the business world, the utility of traditional paper-based approaches is being questioned and the value of face-to-face qualitative approaches explored (Peiperl 2001). In this paper we report on the development of a qualitative approach to gathering feedback that contributes to these developments. Objectives An expert panel in the field of accreditation critiqued the study during its development. They argued that corroboration was necessary so that practitioners could reflect on discrepancies between their own views and those of others and that such views could be used to affirm or challenge the practitioner s own views. To this end, the concept of 360-degree feedback, used extensively in business and increasingly in health care (Alimo-Metcalfe 1998, Swain et al. 2004) as a developmental tool, was examined and adapted. Ward defines 360-degree feedback as, ÔThe systematic collection and feedback of performance data on an individual or group, derived from a number of the stakeholders in their performanceõ (Ward 1997, p. 4). Data collected from stakeholders in an individual s work provide feedback that the individual compares with his own self-assessment. The process consists of: ª 2007 Royal College of Nursing. Journal compilation ª 2007 Blackwell Publishing Ltd, 15,

3 Robert Garbett et al. Box 3 Attributes of expertise a concept analysis (Manley & McCormack 1997) These attributes were developed using a research method known as concept analysis (Morse 1995). It is an approach that looks at the ways that a particular term (or assemblage of terms) is used. It enables the combination of evidence gathered from plural sources (e.g. various empirical approaches, policy, academic review, etc.) and has been used extensively in the development of theory in nursing 1. Holistic practice knowledge is concerned with: Ongoing learning and evaluation from new situations Using various forms of knowledge in practice (e.g. propositional, experiential, professional craft/phonetic, etc.) Drawing from the range of knowledge bases (alongside experiential learning) to assess situations and inform appropriate action with consideration of consequences Embedding new knowledge and accessing this in similar situations as they occur. 2. Saliency is related to Observation of non-verbal cues to understand the person's individual situation Listening and responding to verbal cues Regarding the patient as a whole (i.e. recognizing their uniqueness) to inform treatment process Ability to recognize the needs of the patient, colleagues and others in the actions taken Picking up cues that can be missed or dismissed by others, to inform the situation 3. Knowing the patient is concerned with Respect for people and their own view/understanding/way of being, of the world Respecting patientsõ unique perspective on their illnesses/situation Willingness to promote and maintain a person's dignity at all times Conscious use of self to promote a helping relationship Promoting the patient's own decision-making Willingness to relinquish ÔcontrolÕ to the patient Recognizing the person's own expertise 4. Moral agency is concerned with Providing information that will enhance people's ability to problem solve and make decisions for themselves Working at a level of consciousness that promotes another person's dignity, respect and individuality A conscientious awareness of integrity in one's own work Working and living one's values and beliefs, whilst not enforcing them on others 5. Skilled know-how refers to Enabling others through a willingness to share knowledge and skills Adapting and responding with consideration to each individual situation Mobilizing and using all available resources Envisioning a path through a problem/situation and inviting others on that journey Role set selection (a group of colleagues selected to provide feedback on practice); Data collection usually a validated psychometric tool; Score calculation of a feedback report. By these means, the anonymity of feedback is assured. The literature asserts that this affords a degree of ÕsafetyÕ (Lepsinger & Lucia 1997, Tornow & London 1997, Ward 1997) for those involved. The literature suggests two principal benefits: Actively seeking feedback from a range of sources exemplifies responsiveness in an era when organizations change rapidly and service users expect action in response to their comments (Ward 1997); Teams stand to benefit from developing cultures where there are frameworks that allow people to offer feedback on each othersõ performance (Tornow & London 1997). The approaches traditionally used are predicated on two assumptions (Tornow & London 1997): anonymized feedback is more likely to result in honest feedback; and, to ensure that feedback can be aggregated and anonymized; an adequate number of raters need to be invited to participate in the process (estimates recommended range from at least 12 to 30 or more role set members). The use of measurement scales facilitates the preservation of anonymity more readily than verbatim comments. In this project, we questioned the assumption that randomization would deliver the kind of feedback that would be of use to our participants. Similarly we questioned whether anonymization was necessary for individuals to provide authentic feedback and whether overt provision of feedback could be ultimately more beneficial in promoting Ôimproved organizational communication and teamworkõ (Tornow & London 1997, Day 2000). A traditional randomized and anonymized approach was hampered by the lack of relevant tools for this area of enquiry. Validated tools tend to address leadership and/or managerial behaviour. There are no relevant tools that look at practice expertise in nursing. On a practical level, models of 360 feedback appear to have been developed in organizations where workers 344 ª 2007 Royal College of Nursing. Journal compilation ª 2007 Blackwell Publishing Ltd, 15,

4 Commentary can anticipate contact with large numbers of coworkers. We could not assume that all nurses in the study would have contact with a large enough role set to mean that anonymization and randomization could be undertaken in any meaningful way. Developing a qualitative approach to 360-degree feedback We provided participants with guidelines for the collection of 360-degree feedback based on principles of qualitative data collection (see Box 4). The six groups then refined these processes in practice, an approach congruent with the project s status as an action research study. Participants provided an account of the approach that they used, together with their analysis of the evidence that they collected and its impact on their understanding of their own practice. We supported the participants to work with their critical companions to identify a role set composed of a range of stakeholders in their practice, for example peers, junior and senior colleagues, colleagues from other disciplines and service users. Participants developed a Ôperson specificationõ for a role set member. The attributes identified included: honesty and integrity; openness; a capacity to understand practice within the context of a bigger picture; familiarity with an aspect (or all) of the practitioner s clinical practice; and people with both similar and different values and beliefs (see Table 1). The role sets ranged in size from three to 10 with an average of 5.4 (and a median of 5). They broadly represented the ÔmixÕ suggested above. However, individual selections varied. One participant asked for feedback from four colleagues who were of similar experience to herself (although she was relatively more ÔseniorÕ to them), stressing that, for her, the most important aspect in role set selection was finding people who had sufficient experience of working with her. We provided the participants with a range of possible approaches to gathering feedback in terms of their potential advantages and drawbacks. The participants adapted to the methods suggested, sometimes using the same approach with all role set members, sometimes using different approaches as dictated by the time and resources available to them. Broadly speaking, the approaches used fell into three categories (Box 5). A variety of approaches was used to analyse the data collected; these can be summarized as: The critical companion analysing the themes in the feedback and presenting it to the expert participant; The critical companion and expert participant analysing the data together. For approaches where the anonymization of feedback was necessary, the first approach tended to be used as the process for developing themes from the data. This approach provided an opportunity to generalize the points made. However, this then made it difficult to present detailed and specific feedback. Outcomes The findings confirm that this form of data is useful to practitioners seeking to understand their practice. The data also provide some support to our belief that ano- Box 4 How much feedback is Ôenough'? The guiding principle for role set selection emphasized identifying a group that would help the participants understand their practice expertise. To draw a parallel with qualitative research we suggested that participants identified a Ôtheoretical sampleõ (Silverman 2000) rather than a random sample. In other words, the sample is selected on the basis of relevance to the questions being posed and of peoplesõ capacity to provide the information needed for the activity at hand (Mason 1996). Table 1 A matrix for ensuring coverage of the scope of practice and different types of feedback A role set member should have experience of my: Clinical nurse specialist A Doctor B Occupational therapist C Nursing auxiliary D Manager E Client group Counselling skills Clinical knowledge Able to provide challenging feedback Will provide supportive feedback ª 2007 Royal College of Nursing. Journal compilation ª 2007 Blackwell Publishing Ltd, 15,

5 Robert Garbett et al. Box 5 Approaches to gathering evidence from role set members Feedback gathered in interviews by the critical companions used most frequently (nine cases). The conceptual framework of expertise was used to explore responses to broad questions, for example, ÔHow would you describe X's work in this departmentõ Both critical companion and expert participant involved in gathering data used in seven cases Data gathered using an anonymized questionnaire to collect written comments from role set members used in four cases nymity is not a prerequisite to the provision of open, honest feedback. To ensure that feedback was sufficiently detailed and explicit, those collecting evidence from role set members had to actively probe with their questioning. One critical companion put it this way: Ô(I have) done five interviews so far and got on with the analysis, people are being really positive but (we are) not getting the critique, not getting areas of development, there is some stuff, but its so subtle so we have had to move to more interrogative interview style to get at the actual experiences.õ However, participants reported that even relatively gentle challenges were enough to stimulate reflection and self-critique. Another general comment about the feedback was that it tended to be consistent despite the degree of detail being raised. For example, all members of a role set identified the nurse in question as being thorough and methodical. There were nonetheless examples of role set members who spoke about the same person quite differently. For example, one role set member suggested that the participant did not see the person s experience from all angles and did not fully appreciate the contribution of other approaches to treatment, while another respondent felt that their appreciation of the wider context was clear. This disagreement provided the opportunity for the participants to reflect on their practice more deeply. We set out to challenge the assumption that anonymity has an effect on the openness of the feedback offered. On the basis of this, admittedly small, set of examples we argue that the lack of anonymity does not mean that a role set member will not provide both supportive and challenging feedback, although it would seem that both candour and the capacity to provide detailed feedback require time and effort to develop. For example, in one case the opportunity to provide feedback was the catalyst for initiating difficult discussions about the extent to which the expert participant could be seen to dominate her colleagues through her own enthusiasm and dynamism. This situation had been hidden until the 360-degree feedback was gathered. The activity provoked a degree of anxiety amongst the participants. However, they also reported feeling affirmed by the nature of the feedback that they received. One talked about how useful it was to know that she was valued as Ôthey had never made it explicit beforeõ. Another said: The most significant experience that I experienced was the 360 feedback exercise. It was at that point in my journey that I experienced a Ôdifficult phase of angst Õ before my transformation. The feedback was in itself extremely positive. It was the analysis of the content of the feedback that really challenged my understanding and ability Õ Although challenging, this participant found that the exercise helped her to explore her practice more deeply and ensure that skills evident in her practice with patients were applied equally to managerial and educational aspects of her role. Conclusions Gathering evidence to demonstrate professional and personal effectiveness is increasingly important in health care. The qualitative approaches to 360-degree feedback developed as part of this project contributed corroboration and critique, as well helping participants to develop more open relationships with colleagues and service users. To optimize the usefulness of the process we would suggest the following recommendations: Thorough preparation. It is necessary to be clear with all involved about the purpose of the exercise and to be clear about the criteria used to identify role set members. Suitability. Approaches to gathering information need to reflect what is convenient and appropriate to the colleagues one is working with. They also need to be manageable for the person gathering the information; for example, interviews provide rich data but are time consuming to organize and produce large quantities of data that needs to be processed and analysed (which may mean the acquisition of new skills and knowledge). Producing and distributing a questionnaire may be more manageable 346 ª 2007 Royal College of Nursing. Journal compilation ª 2007 Blackwell Publishing Ltd, 15,

6 Commentary Box 6 For further information about copies of the full project report please contact: RCN Institute Practice Development Team, 20 Cavendish Square, London WC1G ORN, UK. Tel.: + 44 (0) Resources developed through the project can be found at: and less threatening but the detail of feedback may be less. Anonymity. Within the project there were examples of both anonymized and open approaches to gathering feedback. Both were found to be useful. Open approaches provided more detailed information for the development of expertise and workforce effectiveness. The world of business has started challenging the impersonal approaches to gathering feedback suggesting that customized, qualitative feedback, though more difficult and time consuming to generate, is more helpful in improving performance (Peiperl 2001). Similar logic is apparent in tools such as the ÔMini Peer Assessment ToolÕ developed as part of the Modernising medical careers framework (Department of Health 2003). On the basis of our research we would concur with this view and recommend the use and testing of this approach as part of a strategy for professional development. Ethics approval The London Multicentre Research Ethics Committee (Central Middlesex Hospital ref. MREC/OO/2/82) granted full ethical approval in March Role of the funder in the research process The study was funded through the Royal College of Nursing Institute. RG, SH, AT and KM were employees of the RCN Institute during the conduct of the study. Guidance from a wide range of stakeholder groups guided various phases of the project (Box 6). References Alimo-Metcalfe B. (1998) 360 degree feedback and leadership development. International Journal of Selection and Assessment 6 (1), Day D.V. (2000) Leadership development: a review in context. Leadership Quarterly 11 (4), Department of Health (2003) Choice and Opportunity Modernising Medical Careers for Non-consultant Career Grade doctors. Department of Health, London. Department of Health (2005) Agenda for Change: NHS Terms and Conditions of Service Handbook. Department of Health, London, pp Grundy S. (1982) Three modes of action research. Curriculum Perspectives 2 (1), Hardy S.E., Titchen A. & Manley K. (2007) Patient narratives in the investigation and development of nursing practice expertise: a potential for transformation. Nursing Inquiry 14 (1), Higgs J. & Titchen A. (2001) Practice Knowledge and Expertise in the Health Professions. Butterworth Heinemann, Oxford. Lepsinger R. & Lucia A.D. (1997) The Art and Science of 360-degree Feedback. Jossey-Bass Pfeiffer, San Francisco, CA. Manley K. (2000) Organisational culture and consultant nurse outcomes: part 2 consultant nurse outcomes. Nursing in Critical Care 5 (5), Manley K. & McCormack B. (1997) Exploring Expert Practice (NUM65U). Royal College of Nursing, London. Manley K., Hardy S., Titchen A., Garbett R. & McCormack B. (2005) Changing PatientsÕ Worlds Through Nursing Practice Expertise: Exploring Nursing Practice Expertise Through Emancipatory Action Research and Fourth Generation Evaluation. RCN Institute, London. Mason J. (1996) Qualitative Researching. Sage, London. Morse J. (1995) Exploring the theoretical basis of nursing using advanced techniques of concept analysis. Advances in Nursing Science 17 (3), Newman P. & Peile E. (2002) Valuing learnersõ experience and supporting further growth: educational models to help experienced adult learners in medicine. British Medical Journal 325, Nursing and Midwifery Council (2004) The PREP Handbook. NMC, London. Peiperl M. (2001) Getting 360 degree feedback right. Harvard Business Review 79 (1), Rodgers K.G. & Manifold C. (2002) 360-degree feedback: possibilities for assessment of the ACGME core competencies for emergency medicine residents presented at the Council of Emergency Medicine Residency Directors (CORD) Consensus Conference on the ACGME Core Competencies: ÔGetting Ahead of the Curve,Õ March 2002, Washington, DC. Academic Emergency Medicine 9 (11), Royal College of Paediatrics and Child Health (2000) Portfolio of Achievement for Senior House Officers. Royal College of Paediatrics and Child Health, London, pp. 68. Silverman D. (2000) Doing Qualitative Research: A Practical Handbook. Sage, London. Swain G.R., Schubot D.B., Thomas V. et al. (2004) Three hundred sixty degree feedback: program implementation in a local health department. Journal of Public Health Management and Practice 10 (3), Titchen A. (2001) Critical companionship: a conceptual framework for developing expertise. In Practice Knowledge and Expertise in the Health Professions (J. Higgs & A. Titchen eds), pp Butterworth-Heinemann, Oxford. Tornow W.W. & London M. (1997) Maximizing the Value of 360-Degree Feedback. Jossey-Bass, San Francisco, CA. Ward P. (1997) 360 Degree Feedback. Institute of Personnel and Development, London. ª 2007 Royal College of Nursing. Journal compilation ª 2007 Blackwell Publishing Ltd, 15,