Competency-based 360 Multi-Source Feedback

Competency-based 360 Multi-Source Feedback

Contents INTRODUCTION 3 ESTABLISHING FEEDBACK GOALS 3 PROCESS AND RESOURCES 4 DELIVERING THE PROJECT 7 SELECTING A MULTI-SOURCE FEEDBACK SOFTWARE SOLUTION 9 360 DEGREE FEEDBACK CHECKLIST 13 SOURCES 14 2

Introduction 360 multi-source feedback is increasingly being used in employment settings to monitor and measure performance and provide meaningful feedback to employees on their strengths and areas for development. When used properly, this experience can be enriching for employees and managers alike; however, when improperly developed or implemented, 360 multi-source processes can be damaging for both the employee and the organization. This guide reviews the best practices organizations need to follow in order to ensure a positive experience for all involved. The Checklist at the end of the guide provides a series of key questions to help shape the implementation of a 360 Multi-source Feedback process. Establishing Feedback Goals 360 Multi-Source Feedback Defined 360 or multi-source feedback is a process for collecting information about an individual s performance from a range of sources. In addition to the individual being reviewed, these sources include supervisors, peers, subordinates, and in some cases customers. Multi-source systems are primarily used for developmental purposes, promoting individual selfawareness and highlighting key strengths and areas for development. Multiple sources provide a variety of perspectives about an individual s skills and competencies, helping to build a richer and more accurate picture than could be obtained from any one source. An effective 360 degree feedback process leads to more reliable performance ratings, better performance information, and greater performance improvements than feedback processes relying on a single source of information. Properly constructed, they promote an increased understanding of the behaviors required to improve both individual and organizational effectiveness and focus development on those competencies required for successful performance. Goals of Multi-Source Feedback Clearly defined goals and objectives will help guide the adoption of a multi-source feedback process and enable you to determine whether it is successful. Organizations typically use 360 degree feedback for one of two reasons: To support learning and development activities To enhance the performance appraisal process Before deciding on the objectives for multi-source feedback in your organization, it is vital to consider your organization s culture. 360 multi-source feedback is more likely to be successful when openness, mutual trust and honesty are an integral part of the organizational culture and there is a genuine desire for performance improvement. This is especially true if the results are used to support the performance appraisal process. 3

The purpose of feedback, whether it is for development or as part of a performance appraisal process, will influence the way the process is implemented. For example, if used for performance appraisal purposes: Participation in the process is likely to be mandatory It is likely to be carried out annually The individual s manager is likely to be involved in follow-up action Decisions on how the feedback links to reward (e.g., bonuses; salary increases; etc.) need to be clear Having defined the goals of your 360 multi-source feedback process, it then becomes possible to define and develop a system and process for achieving these goals. The next section reviews best practices in defining your process and the resources needed to ensure effective implementation. Process and Resources Essential Criteria to Consider in Designing Your Process How participants view the process is critical. If participants do not think that the system is fair, the feedback accurate, or the sources credible, they are more likely to ignore the feedback they receive. 1. Commitment Commitment from senior management plays a key role in establishing the credibility of a multi-source feedback process. Senior management commitment can be gained through witnessing the success of the system in one part of the organization, if their direct involvement is not possible at the outset. It is important to seek employee input in the development of the process to clarify employee expectations and perceptions of fairness. 2. The raters A number of factors need to be considered when choosing raters. Identifying the most appropriate people to rate each individual s performance The recipient must consider the raters to be credible in order to act on the resulting feedback. Identifying an appropriate number of raters If too few raters are used, one person s feedback can have a disproportionate impact on the overall results. With a small number of raters it is also difficult to ensure the anonymity of feedback sources. We recommend a minimum of 3 to 5 people per feedback group. If fewer are available, then combine groups for example, combining direct reports and peers into a single group. 4

Concern that the person being rated may respond negatively to raters who provide negative feedback To minimize this concern, feedback should be delivered anonymously for those groups for which there is concern or retaliation could be an issue. 3. The questionnaire Best practice suggests the method of assessment used in a multi-source feedback process should follow these guidelines. Describe behaviors related to actual job performance Competencies define the behaviors employees need to display for the organization to be successful; therefore, measuring the competencies at the target proficiency levels required in the job is an essential part of the feedback process. An example of how competencies are measured in a 360 Feedback process is shown in the diagram below. In this case, the competency being assessed is Teamwork. The competency is defined as Working collaboratively with others to achieve organizational goals and the specific behaviour that is being assessed is Seeks input from other team members on matters that affect them. Those individuals providing feedback rate how effective the employee is based on the employee s observed behaviour on the job. Align with other HR processes within the organization The competencies that are incorporated within the feedback process will depend on the goal of the process. If it is aimed at supporting employee development within their current jobs or roles, then the competency profile for the target employee s job would be used as the standard for providing feedback. If, however, the 360 Feedback process is being used to support development for advancement within the organization (e.g., Career Development; Succession Management), then the competency profile for the next level, or another more advanced job, would be the standard used to measure and provide feedback. 5

It is, therefore, important to define the goal of the 360 feedback process and then to pick the competency profile most suited to support this goal. These competencies and their associated behavioral indicators will serve as the measurement standards in the assessment process. Reflect the organization s culture and values Job profiles often incorporate core competencies that describe in behavioral terms the key values of your organization. Allow respondents to indicate when they have not had the opportunity to observe a behavior This prevents respondents from being inadvertently forced to give feedback that is based on guesses. 4. The structure of feedback Consistent with best practice, feedback should be broken out for each question by presenting the average ratings from each feedback group so that differences in perspectives are easy to identify. If there are enough raters involved, this should not compromise anonymity. If there are only a few raters, group averages can be combined to protect anonymity. The option to add observations or comments should be provided. This can help to throw more light on the ratings, but the person giving the feedback needs to be sensitive in providing this information. It is important, therefore, to provide an orientation to those giving feedback on best ways to do this, both in terms of the providing accurate rating as well as providing comments and examples that validate the rating in both a respectful and honest manner. Once a decision is made on who has access to the ratings, this needs to be followed consistently. A change in who has access to the information is one of the most commonly cited reasons for a lack of trust in the process. If there are good reasons to change, it is critical to seek the permission of the individuals involved before making that change. Time and Resources Required When planning a multi-source feedback process, it is important to have an accurate view of the time and resources needed to roll it out effectively. This includes the time needed to set up and manage the program, provide the feedback from the different groups, gather the feedback and compile reports, and finally give that feedback to the individual and support subsequent actions to develop and improve performance. When multi-source feedback is being used to encourage and enhance development, it is important to consider in advance the resources needed to support such development. Gathering feedback information is just the starting point in the development cycle. The next step is to create individual learning plans that target specific developmental needs. 6

Having defined the process and resources needed for competency-based multi-source feedback in your organization, the next steps are to pilot, implement and finally evaluate your program to ensure that it is meeting your intended goals. The next section addresses these topics. Delivering the Project Piloting A pilot can generate a realistic picture of the resources required to manage the process throughout the rest of the organization. Valuable insights can be gained about the time required to provide ratings and feedback, as well as how soon the feedback can be given to participants. Piloting also helps reduce uncertainties by allowing a test group to experience the process. It provides useful information for further planning and communication and allows for a review of the multi-source feedback instrument. An initial review allows consideration of such questions as whether the questionnaire is user-friendly, and whether appropriate development actions have been identified. Lessons learned through the pilot should be considered. Any alterations and adaptations that will make implementation smoother should be made. Implementation The most critical part of the implementation process is ensuring that all participants are clear about what is involved. To ensure this occurs: Establish an individual or team to take responsibility for administering the system this helps ensure that the procedure is running smoothly and any issues are resolved swiftly. Provide a point of contact for participants with questions and concerns. Establish timeframes for providing ratings and feedback. Send automated email invitations and reminders to individuals who are late completing their feedback. This reduces the administrator s workload and maintains momentum. Brief raters on the objectives of the scheme and provide instructions for completing questionnaires. Provide clear and positive communication throughout the process. 7

Providing Feedback Effective feedback is the springboard for subsequent development and is integral to the success of the process. HOW WILL THE FEEDBACK BE COMMUNICATED? Given that an individual is receiving sensitive information about how their colleagues, direct reports and manager view their performance, sensitivity is required. Best practice would be to make someone available to help interpret the results with that person. The people giving feedback will need to have the skills to support this process. The facilitators need a good understanding of the organization s policies on the process, the instrument and report, an awareness of the range of reactions individuals have to feedback, and interpersonal skills in conducting a feedback session. Facilitators must also be seen as trustworthy and credible. When being done for development, discussion of the results with the facilitator can help focus the discussion on future development planning rather than on the feedback itself. Skilled facilitators will help the individual to draw out evidence and make connections across different people and situations. It is this process that stimulates self-awareness and makes multi-source feedback such a powerful process. WHEN WILL THE FEEDBACK BE COMMUNICATED? Ideally, individuals should receive feedback as soon as possible after the feedback was given. This maintains the momentum of the process and the motivation of the individual. Given the pace of change in many organizations, shorter turn-around times ensure that the feedback is still relevant for the role. It is important to ensure that people receive feedback when there is support available to interpret the results. Providing a report without support, particularly prior to a weekend or going on holidays, is far from ideal, and can have negative consequences. Review Reviewing and evaluating the success of the process is widely overlooked. The key question to consider is whether the program met its original purpose. If the original purpose was to improve performance, have relevant development needs been identified? If it was to support the performance review process, has the process supplied the required information in a fair and credible way? QUALITATIVE REVIEW A qualitative review with the key people involved can provide invaluable information on whether the process has achieved its goals. This review should include individuals receiving feedback, doing the rating, facilitating the feedback and the line managers of those involved. The timing of the review will depend on the original purpose, with more time needed when the purpose was development. 8

THE QUESTIONNAIRE How effective is the questionnaire? Was it consistent with and link to other relevant indicators of performance in the organization? Did individuals gather development information? Did raters use the rating system effectively? Was it reliable? Did it look right? Use a system that aggregates data from the questionnaires in order to identify patterns of strengths and development needs across the participating group. This information can be used to feed into development planning at a strategic level, to ensure that the organization has people with the relevant skills to meet its objectives. Selecting a Multi-Source Feedback Software Solution Why a competency management software system is important Designing, developing, implementing and maintaining a competency framework is difficult to do in a paper-based format. It can quickly become unwieldy and out of control if not managed through a competency management software system. Without such a system it is difficult to build and maintain processes like 360 multi-source feedback based on the most current competency information. What to look for in a system 1. A system with existing well-researched competency content This includes a library of the general competencies as well as technical / professional competencies that are suited to your organization. These days, it is not necessary or even advisable to develop your competencies from scratch. It can take years to develop highquality competencies. Vendors often also have standard job competency profiles available that reflect the job duties / tasks typically required in jobs within specific functional areas as well as industry sectors. These can then become the starting point for use within your organization, editing and adjusting them to fit the unique requirements of your organization. 9

2. A system that supports standardized implementation Organizations are increasingly experiencing distributed workplaces, with employees operating out of multiple locations. As a result, it is becoming more difficult to ensure that the human resource processes are implemented in a uniform and standardized way. If you have a system that supports the standardized adoption of competency content and competency-based HR processes, it becomes easier to ensure that HR professionals, managers and employees are accessing and implementing the correct competency content in an online 360 multi-source feedback process. 3. A flexible system configurable to your needs In many 360 multi-source systems, the software delivery and content are inextricably linked. Organizations therefore have to buy-in to the content and underlying model being delivered in the software. So, for example, if you wish to implement a 360 process to assess leadership competencies, you effectively have to adopt the leadership competency model that is part of the feedback tool. But the model being delivered in the software may not meet your organizational needs, reflect the values and culture of your organization, or incorporate the competencies you are attempting to reinforce and develop within your various employee groups. A more appropriate and valid approach is to have a system that links to the competency content and models you have designed and developed for your organization. As such, the system should allow you to pick from a list job competency profiles or models, and then implement this competency information within the 360 multi-source feedback tool that is part of the same system. In addition, the tool should allow you to select the individuals and groups who will be part of the feedback process. In some cases, for example, you may wish to collect feedback from work colleagues; in other cases you may wish to collect feedback from work colleagues and clients of the target participants. In each case, the groups providing feedback should be based on the job being performed and the purpose of the assessment. The system should also be flexible with regard to the rating scale being implemented, both in terms of the number of levels (rating scales can run anywhere from 3 to 7 levels) as well as the scale type (e.g., effectiveness scale; observed frequency of the behaviour; etc.). Research on the number of levels and type of rating scale is extensive and subject to a great deal of debate as to what is best practice. You should have the flexibility to be able to choose or design a rating scale that works best for the feedback process and type of work being performed. Finally, you should have the flexibility to identify what type of competency information is being assessed. Most systems take the assessment process down to the level of the 10

behaviour / performance indicators for each competency (e.g., for Client Focus proficiency level 3 Looks for ways to add value beyond the client s immediate request ), but in some cases the assessment may be performed at the level of the competency. You should be able to choose what is being assessed according to the goal of the assessment. In CompetencyCore, for example, you can assess competencies at the individual behavioural indicator and / or at the competency level. 4. Reporting of feedback results The 360 multi-source feedback tool should also provide good graphical information that allows the comparison of results across the different types of people providing the feedback. For example, the image below, taken from a CompetencyCore 360 report, clearly demonstrate how the ratings given by each of the participants differ. The breakdown of information should not only be provided at the competency level, but also at the level of the individual behavioral indicators for each competency. This allows the target participant to gain different perspectives on his / her performance. It also provides a more diagnostic perspective on how each competency should be developed. For example, although the average performance on a particular competency might meet performance expectations, individual behaviours may require improvement within the competency. Finally, organizations can engage in a 360 multi-source feedback process to review performance at an organizational, regional, and / or functional level (e.g., all financial jobs). The reporting tools should therefore allow for the aggregation of data to determine key themes across selected groups. Plans and programs can then be identified to address highpriority development or training needs. 5. Security, confidentiality and anonymity As noted previously, it is important to protect the anonymity of certain types of raters in particular, when using direct reports to the target participant in the feedback process. As well, with a small number of raters, one person s feedback can have a disproportionate 11

impact on the overall ratings. It is therefore important to be able to define the rules in the software for combining certain types of raters scores to ensure feedback confidentiality and anonymity. Finally, when raters are asked to provide comments to substantiate their ratings, it is important that they are instructed to do this in a positive and helpful way, and that when doing so, they abide by whatever confidentiality and anonymity rules your organization establishes. Make sure that your software allows you to incorporate these kinds of instructions. 6. Integration and alignment within the talent management process 360 multi-source feedback is not a stand-alone process. It is done to accomplish a particular goal, for example to address gaps in competency through learning and development. Therefore, the software should allow the user to link to other HR processes in the system. For example, in CompetencyCore, any competency gaps identified through the assessment process can feed directly into a learning plan tool that provides targeted learning resources (e.g., on-job activities, books, courses, etc.), organized by competency, to help address those gaps. This is only one example of how the 360 feedback process can be integrated with, and feed into, other talent management processes within the organization. 12

360 Degree Feedback Checklist 1. Planning 2. Piloting Does the organizational culture support the introduction of 360 feedback? Are the purpose and objectives clear? Is there senior management commitment? Have those taking part been involved at the planning stage? Has a clear process for identifying the raters been agreed? Does the questionnaire relate to job performance and describe relevant behaviors? Is it a reliable and valid instrument? Is the feedback presented in a useful and sensitive way? Have the resources for subsequent development been considered at the planning stage? Is the questionnaire meeting its purpose? If it has not been used before, is it reliable and valid? Are the resource estimates for the rollout realistic? 3. Implementation 4. Feedback 5. Review Is there clear communication to all involved? Is there a clear owner responsible for administering the process? Is there a helpline or clear point of contact? Are the deadlines clear and who monitors whether they are being met? How are the raters being briefed? Have those giving the feedback been trained in the relevant skills? Will the feedback be communicated face-to-face? Is there appropriate support for those receiving the feedback? Has appropriate and sensitive timing been considered as part of the process? Has the review process been considered at the planning stage? Has the 360 feedback met its purpose? Has it resulted in improvement in performance? 6. Choosing a Software System Does the system come with well-researched competency content? Is the system secure and confidential? Does the system provide informative reports? Does the system help feed assessment results into other talent management processes? 13

Sources Alimo-Metcalfe, B. (1998). 360 degree feedback and leadership development. International Journal of Selection and Assessment, 6, 35-44. Cawley, B., Keeping, L., & Levy, P. E. (1998). Participation in the performance appraisal process and employee reactions: A meta-analytic review of field investigations. Journal of Applied Psychology. Craig, S. B., & Hannum, K. (2006). Research update: 360-degree performance assessment. Consulting Psychology Journal: Practice and Research, 58(2), 117 124. DTI. (2001). 360 degree feedback: Best practice guidelines. Downloaded from: www.dti.gov.uk/ mbp/360feedback/360bestprgdlns.pdf Fletcher, C., Baldrey, C., & Cunningham-Snell, N. (1998). The psychometric properties of 360 degree feedback; an empirical study and a cautionary tale. International Journal of Selection and Assessment, 6, 19-34. Levy, P. E., & William, J. R. (2004). The social context of performance appraisal: A review and framework for the future. Journal of Management, 30(6), 881 905. Maylett, T. (2009). 360-degree feedback revisited: The transition from development to appraisal. Compensation & Benefits Review, 41(5), 52 59. Morgeson, F. P., Mumford, T. V., & Campion, M. A. (2005). Coming full circle: Using research and practice to address 27 questions about 360-degree feedback programs. Consulting Psychology Journal: Practice and Research, 57(3), 196 209. For more than 25 years Organizational excellence through competency-based talent management Since 1989, HRSG has focused on competency-based talent management using best-practice, multilevel competency content. We provide a complete range of competency tools, including 550+ multi-level competencies, CompetencyCore software, consulting, training, and fast-track support packages that help organizations develop and deploy competencies in a fraction of the time. To discuss your competency initiative and see the HRSG multi-level competencies in action, call us at 1-866-574-7041 or email info@hrsg.ca. Human Resource Systems Group. All rights reserved. Tel: 1-866-574-7041 www.hrsg.ca info@hrsg.ca