360 Performance Evaluations An Assessment of the attitudes of Human Resource directors in the

Transcription

1 360 Performance Evaluations An Assessment of the attitudes of Human Resource directors in the state of Texas Melissa C. Garza An applied research project submitted to the Department of Political Science Southwest Texas State University in partial fufiillment for the requirements for the degree of Masters of Public Administration Fall 2000 FACULTY APPROVAL:

2 Abstract Performance evaluations are an integral part of most Human Resource systems. For this reason, performance evaluation systems command an inordinate amount of research attention. This study assesses attitudes of Human Resource directors in Texas state agencies with regard to key aspects of the 360 performance evaluation process. First, an examination of literature identifies recurring themes, issues and characteristics associated with the implementation and use of 360 systems including organizational culture, anonymity, utilization, accountability, resources, training and rater variance. To assess attitudes regarding key characteristics of the 360 evaluation process, data were collected from Human Resource directors in Texas state agencies via survey questionnaires. Next, the study reports attitudiial findings from the survey with regard to key characteristics derived from the literature review. Results indicate that Human Resource directors generally agree with the information pertaining to the conceptual categories revealed through the literature review. The findings indicate, for example, that organizational culture is a strong predictor of the success or failure of the implementation of 360 feedback systems. There is also support for the incorporation of accountability mechanisms, training and resources. Interestingly, it is found that utilization of feedback data for purely developmental purposes does not necessarily preclude sharing data with supervisors even though the data will be included in performance appraisals. This is important because experts in the field argue that feedback ceases to remain purely developmental when it is shared with supervisors and included in the performance appraisal process.

3 Finally, recommendations for future research are identified. One suggestion is to study organizations actually utilizing the 360 performance evaluation process. As agencies could potentially employ some, but not all, elements of 360, future research may focus on elements or portions of the process state agencies use. Subsequent studies may also include a population representative of all state agencies and may compare and contrast 360 systems in public and private sectors.

4 Table of Contents Abstract Chapter 1: Introduction Research Purpose Chapter Summaries Chapter 2: Literature Review Introduction Rationale Information Sources for Performance Appraisals Figure 1.1 Support for Organizational Effectiveness... Key Characteristics... Organizational culture... Anonymity... Utilization... Developmental Feedback Feedback for Performance.. Accountabdity Resources.. Trammg... Rater variance... Conceptual Framework Chapter 3: Research Setting Introduction Organizational culture... Anonymity... Utilization..... Accountabdity.. Resources/Training Rater variance Individual Leaders Performance Evaluation Systems in the Public Sector... Examples of 360 Performance Evaluation Systems Chapter 4: Methodology Introduction... Research Design Population... List of 36 State Agencies Surveyed... Operationalization of the Conceptual Framework... Statistics...

5 Chapter 5: Results Introduction Organizational culture Anonymity Utilization.. Accountability Resources.. Training Rater variance... Chapter 6: Conclusion and Recommendations Introduction Summary of Results by Survey Question Summary of Results by Category... Recommendations for future research Concluding Remarks Bibliography Appendices Original transcript of interview with Kay Betz, PhD Organizational climate survey Survey Instrument Index of Tables Conceptual Framework Linked to Literature Operationalization of the Conceptual Framework Percent responses regarding Organizational Culture Percent responses regarding Anonymity Percent responses regarding Utilization: Development Only Percent responses regarding Utilization: Performance Percent responses regarding Accountability Percent responses regarding Resources Percent responses regarding Training Percent responses regarding Rater Variance Summary of Survey Results for questions within each Category... Summary of Survey Results for each Category Table Table Table Table Table Table Table Table Table Table Table Table

6 Chapter 1 Introduction The work environment is changing. In an era where the nature of work is evolving into team-based approaches and where more work is completed away from traditional supervisory control, supervisors may not have all the information needed to provide complete, accurate feedback making conventional employee assessments less meaningful. Traditional performance evaluation systems often require supervisors and subordinates to develop a work plan at the beginning of a performance cycle. The supervisor subsequently provides feedback during annual or semi-annual reviews based solely on his or her assessment of subordinates. As a result, hierarchical performance evaluation systems ignore assessments from the full circle of people who are knowledgeable about employees' performance. An alternative to traditional, supervisory-controlled performance evaluation systems is three hundred sixty degree feedback, which assesses employee performance from several points of view: self, peers, superiors, subordinates and customers. The basic process consists of performance rated by people in an employee's entire circle of influence, ratings fed back to the employee and compared to self-assessment and developmental goals set in connection with a plan for achieving the goals. The logic of 360 feedback systems, in part, is that people at various levels within an organization may have the most accurate information on different aspects of an employee's performance. For example, subordinates may be the best sources of information on delegation and leadership skills white peers may have the best information on interpersonal skills and technical capability. Three

7 hundred sixty degree performance evaluation systems attempt to incorporate the most accurate information from those who have actually observed each dimension of an employee's performance creating an increased perception of fairness and credibility in the assessment process. By expanding the information available, multirater systems potentially improve the quality of information utilized during performance appraisals. As a result, the use of 360-degree feedback has increased dramatically in recent years. This study reflects a desire to ascertain the attitudes of Human Resource directors in the public sector about the 360 feedback process. Research Purpose The utilization of TQM and team-based work structures changed the nature of work in that employees began interacting and working with co-workers outside of conventional work groups and away from traditional supervisory control. As a result, traditional performance evaluation systems, aeated for work controlled by supervisors, often ignore a large pomon of an employee's performance. Three hundred sixty-degree performance evaluation systems have the potential to provide performance data on virtually every aspect of an employee's performance by including data from people within an employee's entire circle of influence. Including data from all of an employee's constituents improves the amount and quality of performance data. The impetus of this study was aeated, in part, by the increasing popularity and use of multirater systems.

8 The purpose of this study is to identify and describe the attitudes of Texas state Human Resource directors about key aspects of the 360 performance evaluation process. The purpose is achieved through: 1. A review of the literature to identlfy relevant attitudes and conceptual categories 2. A survey of Texas State Human Resource directors. Chapter Summaries This study is divided into six chapters, a bibliography and related appendices. Chapter One, which introduces the subject matter, contains the research purpose and a summary of chapters. Chapter Two provides a literature review with rationales and key characteristics of the 360 performance evaluation process. Included in Chapter Three, is a description of the 360 process in Texas state agencies and an interview with Dr. Kay Betz, Director of Employee Learning and Organizational Development at the University of Texas at Austin. An explanation of the research methodology, an assessment of surveys as a method for performing this research and a description of the research population are presented in Chapter Four. Chapter Five contains survey results and an analysis of survey data. Chapter Six is comprised of recommendations for future research and conclusions gathered through the literature review and survey research. The Appendices contain the original transcript of the interview with Dr. Betz, an organizational climate survey and the survey instrument.

9 Chapter Two Literature Review Introduction The purpose of this chapter is to describe the implementation and utilization of three hundred sixty-degree (360 ) feedback systems 1. The chapter begins by defining 360 systems and considering various rationales and applications of the 360-degree performance evaluation process. Next, seven conceptual categories of 360 systems are considered including organizational culture, anonymity, utilization, accountability, resources, training and rater variance. The chapter concludes with the development of a conceptual framework, which provides the organizing tool for the empirical component of this study. Three hundred sixty-degree performance evaluation systems are comprised of behavioral observations collected from many organizational layers that take into account evaluations from multiple raters in multiple levels of the organization (Mount et al., 1998). For example, people in an individual's circle of influence provide feedback about job performance including information coming downward from superiors, upward from subordinates, laterally from peers and co-workers, and inward from the individual (Edwards and Ewen, 1996, p. 46, Waldman et al., 1998). (See Figure 1. I). The logic behind multirater systems is to collect information from those with the best knowledge of an individual's performance and from those who ' For the purposes of this paper, 360-degree feedback, 360-degree systems, multirater systems, 360 systems and 360-feedback proms are synonymous with 360-degree performance evaluations.

10 have had the best opportunity to observe work behaviors, which provides a more accurate perspective of how others view performance such as management practices, interpersonal style and effectiveness (Edwards and Ewen, 1996, p. 19; Wimer and Nowack, 1998). In short, the basic idea of 360 is to gather input on employees' performance from the full circle of people with whom they frequently interact and who are knowledgeable about their performance (Edwards and Ewen, 1996, p. 19 and pp ). Three hundred sixty-degree performance evaluations are characterized as nontraditional performance management systems. Traditional performance evaluation systems focus on supervisory input, data collection and control of the performance evaluation process. Multirater systems are considered an alternative to traditional performance management systems because they focus on performance input from sources other than the supervisor. Edwards and Ewen and Antonioni maintain that performance information collected from multiple perspectives is more "honest, reliable, valid and credible" than information obtained through traditional performance evaluation systems (Edwards and Ewen, 1996, p. 4; Antonioni, 1996). It is interesting to note that 360 evaluations incorporate the research norm of triangulation. A performance evaluation is analogous to a research project with a particular working hypothesis. For example, a particular employee's performance may be satisfactory, less than satisfactory or exceptional. The 360 evaluation process provides a system to collect multiple types of evidence to support any given hypothesis. Although a hypothesis cannot be proved, multiple types of evidence increase the confidence that the evidence (if consistent) is accurate.

11 Rationale By increasing sources and types of data included in appraisals, 360 systems attempt to improve organizational performance by expanding the information available (Jackson and Greller, 1998). Because employees work with a wide range of people, managers alone cannot accurately observe and assess all dimensions of an employee's performance (Prewitt, 1999). Therefore, multirater systems provide unique perspectives on job performance and offer insight into behaviors that supervisors may otherwise miss, such as interactions with peers and performance on teams (Edwards and Ewen; Pollack and Pollack, 1996). Edward and Ewen argue that evaluations from a number of sources (versus traditional top-down evaluations) provide a more balanced and comprehensive view of an employee's performance. The literature suggests that multirater systems are more reliable, credible and predictably valid than single-rater systems. In addition, assessments conducted by multiple co-workers are usually more dependable and objective than information gained from a single person (Edwards and Ewen, 1996, p. 19). Multirater systems integrate input from various constituencies in the assessment of an employee's performance. While single-source assessments reinforce an employee's accountability to the supervisor, multirater systems link accountability to all stakeholders and help employees understand critical work-related constituencies other than the supervisor (Edwards and Ewen, 1996; Tornow et al., 1998, pp ). By comparing multirater feedback to self-evaluation, individuals obtain a more realistic picture of their strengths and weaknesses (Tornow, et al., 1998; Pollack and Pollack, 1996). When employees compare self, subordinate and peer ratings, they are more likely to become aware of specific areas in need of improvement. Also, 6

12 hearing the same message from several sources serves as a strong impetus for change (Jackson and Greller, 1998). The 360 process can increase employees' perceptions of fairness and credibility in the evaluation process. Edwards and Ewen point out that as performance measures potentially impact an employee's career in terms of selection, salary, training and development, organizations should employ the most fair and accurate method available. Perceived fairness is one of the key attributes of 360 performance evaluation systems (Edwards and Ewen, 196, p. 195). Also, multirater systems can reduce discrimination and bias (Hoflinan, 1995). Single source systems are often biased against older employees, women and people of color. Performance evaluations from single raters often result in lower scores for these groups because most supervisors are white males (Edwards and Ewen, 196, p. 195). Multirater systems, in contrast, can moderate adverse discrimination because the diversity of raters should reduce the likelihood of discrimination (Edwards and Ewen, 1996). For this reason, 360 systems promote diversity management?. ' Diversity management attempts to open organizational systems to all employees by building employee-specific skills and creating appropriate policies. Behaviors and policies, evaluated by their contribution to organizational goals and objectives, are tied to rewards and results. Diversity management could lead to increased productivity because it assumes that diverse groups will create new ways to work together effectively.

13 Figure 1.1 Information sources for 360 Performance Evaluations Managers Subordinates

14 Support for Organizational Effectiveness Improving each employee's performance potentially increases overall organizational effectiveness. Three hundred sixty-degree evaluation systems link organizational strategies, goals, core values and expectations to individual behavior and performance. According to goal setting theory, goals provide employees with information pertaining to tasks and the amount of effort required to ensure completion. Specific goals increase performance. Furthermore, difficult goals combined with feedback result in higher performance than goals which are easily attainable. Three hundred sixty degree evaluations are consistent with goal setting theory because employees understand that performance in the areas measured through feedback is important and valued. A study by Reilly et al. found that exposure to valued behaviors (e.g. encourages and facilitates teamwork and collaboration) is as important as receiving feedback. The authors discovered that simply receiving performance feedback not tied to behaviors valued by the organization made very little difference in performance improvement over time. On the other hand, exposure to valued behaviors created the impetus for change. In short, awareness of valued behaviors and knowledge that measurement would occur again resulted in setting specific behavioral goals after exposure to critical values (Reilly et al., 1996). Aspects of performance used to rate an individual during a 360 evaluation often inculcate behaviors important to the organization (Tornow, et al., 1998, p. 90). The 360 system is an effective way to promulgate organizational standards and expectations. Identifying and using core values and expected behaviors as ratings

15 communicates the importance of performance dimensions to both raters and feedback recipients and, in theory, also focuses attention on similar performance standards (Edwards and Ewen, 1996, p. 13). For example, utilizing behavioral statements supports core values through specific definitions, which allows the operationalization of the organization's expectations and values (Tornow et al., 1998, p. 80). Behavioral statements allow raters to evaluate how well ratees adhere to values and model desired behaviors (Edwards and Ewen, pg. 80). At its best, 360 systems translate desired core values into behavioral dimensions by creating categories of performance with specific illustrative behaviors delineated on the evaluation instrument. Subsequently, individuals receive feedback on 360 dimensions that, in turn, reinforce organizational values (Tornow et al., 1998, p. 14). For example, a manager can rate a peer with items related to leadership such as providing direction, influencing others, fostering teamwork and motivating others. By evaluating peers on these specific dimensions, the manager will perceive that the organization is serious about leadership. Also, when managers receive feedback on specific leadership dimensions, they recognize that these are behaviors the organization values. In addition to reinforcing organizational values, developmental needs are revealed via 360-degree feedback. Individual developmental needs are easily assessed through feedback provided by peers, subordinates and superiors (Hoffman, 1995). Aggregating data across individuals creates an organizational needs assessment. Organization-wide aggregated data brings trends and behavioral patterns to light, which provides insight and guidance into developmental needs at the organization or systemic level (Tornow et al., 1998, p ). 10

16 Key Characteristics Associated with 360 performance evaluation systems Three hundred sixty-degree evaluation systems offer the potential for substantial benefits, including improved evaluation systems in terms of quality, amount of information available and increased perceptions of fairness in the evaluation process. The literature suggests that the characteristics of the organization and its environment influence the effectiveness of a 360 system. Further, an effective 360 system requires attention to various facets of the implementation and assessment processes. The next section discusses factors identified in the literature such as organizational culture, anonymity, utilization, accountability, resources, training and rater variance, that create and support 360 performance evaluation systems. Organizational culture Organizational culture is a strong predictor of the success or failure of multirater processes. Regardless of the potential benefits made possible by the 360 process, multirating is not always appropriate (Bookman, 1999). The 360 process requires honest feedback from all raters. Consequently, organizational cultures promoting honesty, open communication and trust are more likely to successfully implement a 360 process and realize the associated benefits (Tornow et a]., 1998 p. 96). Conversely, organizations with a history of distrust, poor communication and fear may find 360-degree feedback difficult to use (Tornow et al., 1998, p. 99, Prewitt, 1999). Hayworth maintains that a "safe" environment is required for 360 performance evaluation systems to be effective (1998). In fact, "[o]rganization readiness is critical to assess before engaging in any 360-degree

17 feedback processes" (Tornow et al., 1998, p. 143). Certain values and skills within the organization facilitate the success of the 360 method and include, freedom to speak openly without fear of punishment or coercion, the ability to listen to others, respect their ways or understanding, and work to understand their perspectives, willingness to reflect on and challenge one's own thinking and actions and equality of ideas - that is, no individual's ideas are more correct than anyone else's simply because of the individual's position or status (Tornow et al., 1998, p ). Surveys gauging organizational climate and culture are available to assess an organization's readiness for the 360 process. (For an example of an organizational climate questionnaire, see Appendix A- 1). Anonymity The rater's sense that anonymity would be respected seems to rest upon an honest organizational culture. Traditional top-down evaluations have not been anonymous. Supervisors are often encouraged to have face-to-face meetings to discuss an employee's performance. This structure fits traditional systems because the supervisor or manager is providing feedback to subordinates. Moreover, traditional evaluation systems do not solicit subordinates' assessments of supervisors' performance. Three hundred sixty-degree systems change the role of supervisors and subordinates in the evaluation process and, as a result, create concerns about anonymity, which are particularly high among employees who evaluate their supervisors. Thus, successful 360 systems depend on guarantees of anonymity for key participants in the process. Assuring anonymity allows participants to feel safe

18 in providing honest feedback. Accordingly, 360 systems must guarantee absolute anonymity to respondents (Wimer and Nowack, 1998). The goal of anonymity is achieved if the individual receiving performance feedback information cannot determine the identity of those who provided the information (Edwards and Ewen, 1996, p. 60). The absence of anonymity distorts feedback scores employees provide. Antonioni found that appraisers who remained anonymous gave lower ratings than those who could be identified due to concerns over managers' reactions to the evaluations (Antonioni, 1996). Anonymous feedback lessens employees' fears of retribution from peers and superiors for low ratings (Westerman and Rosse, 1997). Antonioni also found that employees want their identity withheld and want to volunteer to rate managers (Antonioni, 1996). Anonymity allows raters the choice between participation and non-participation. Rater participation may be negatively affected if employees perceive that the ratee can determine the identity of rater. Low rater participation impacts the reliability, validity and user acceptance of ratings and limits the effectiveness of the system (Westerman and Rosse, 1997). Thus, the very credibility of the process rests on protecting the identity of those providing feedback (Edwards and Ewen, 1996). Experts in the field recommend that ratings remain anonymous. In fact, "360 systems must guarantee absolute anonymity to respondents only works when anonymity is assured (Edwards and Ewen, 1996, p ). Antonioni suggests that organizations take steps to safeguard the identity of raters, thereby alleviating concerns of retaliation, by providing data in an aggregated format and by including feedback from five or more direct contributors (Antonioni, 13

19 1996; Pollack and Pollack, 1996). In the absence of five direct contributors, the combination of three direct contributors and three peers may be utilized to preserve anonymity (Antonioni, 1996). Utilization: Development versus Performance Traditional performance evaluation systems typically focus on performance. Even systems that contain developmental goals often base evaluations on performance dimensions of the job. Exceptions occur when a supervisor requires an employee to attend sexual harassment prevention or diversity training. Yet courses requiring the participation of all employees in an organization are not typically targeted to individual development. On the other hand, behavioral statements tied to organizational expectations in many 360 systems allow employees to be evaluated for purely developmental purposes, but also allow evaluation on performance dimensions which are used to make administrative decisions, such as pay and promotion. When 360 feedback data is purely developmental, the employee neither receives nor suffers any employment decision as a result of the evaluations. The feedback data only informs the employee of areas in need of improvement. On the other hand, data used to evaluate performance potentially impacts salary, promotions, job assignments and other monetary incentives. The use of 360-degree feedback, either for performance, development or both, is a highly debated topic and is impacted by two factors: those who control feedback data and the nature and role of those who have access to feedback reports.

20 Developmental Feedback Developmental feedback raises awareness about areas in need of improvement because it is derived from credible sources with the power to motivate change. According to Edwards and Even, developmental feedback "facilitates the employee's growth on the job" (1996). In pure developmental systems, employees control access to their feedback. Supervisors do not see developmental feedback unless the employee shares the results with them (Edwards and Ewen, 1996, p ). Hence, feedback ratings are not included in performance appraisals (Atwater and Waldman, 1998). Pollack and Pollack found that managers were more accepting of feedback from subordinates and peers when ratings were used solely for development (1996). Moreover, recipients of feedback may be less defensive and more apt to regard feedback as credible if the information is utilized solely for developmental purposes. Therefore, developmental systems have greater potential to elicit honest feedback because administrative decisions surrounding pay, promotion, selection and training are not at risk (Edwards and Ewen, 1996, p. 47). Feedback for Performance Feedback used for performance is similar to feedback used for development except that the employee's ratings are shared with the supervisor who uses the information when making judgments about performance (Edwards and Ewen, p. 50). If the feedback is shared with the supervisor, the feedback becomes performance information. Although many development advocates recommend sharing feedback information with supervisors, Edwards and Ewen maintain that "supervisors cannot 15

21 use the information for development and then ignore that information during performance appraisals" (1996, p. 12). Hence, the requirement to share feedback with superiors has great potential to influence appraisals (Atwater and Waldman, 1998). Feedback used only for development may not prompt employees to set performance improvement goals (London et al., 1997). Prewitt maintains that getting information is a "waste of money if it does not lead to improved outcomes" (1999). Walker and Smither found that ratee accountability is low when feedback is used only for developmental purposes (1999). In other words, ratees may be more likely to use feedback when it is used to make administrative decisions (London et al., 1997). In addition, raters are "more observant and evaluate performance behaviors more carefully if they know that their ratings will have important consequences" (London et al., 1997). Basing performance evaluations and administrative decisions on 360 feedback has the potential to improve the quality of participation in the evaluation process for both raters and ratees. Although utilizing 360 feedback for performance evaluations and administrative decisions may increase accountability for raters and ratees, it may also produce negative consequences. First, employees may concentrate on how to achieve better evaluations rather than on how to improve performance (Pollack and Pollack, 1996). Second, when feedback data are utilized for performance, feedback recipients may become defensive, which potentially prevents changes in behavior (Tornow et al., 1998). Third, there are concerns regarding the accuracy of information when feedback data are collected for evaluation purposes because employees may fear reprisal from supervisors and peers if the feedback data impact promotions and 16

22 bonuses (Pollack and Pollack, 1996). Subordinates may inflate ratings if they know ratings will be used to determine managers' merit increases (Antonioni, 1996 and Atwater and Waldman, 1998). Because employees fear retaliation, Pollack and Pollack and Edwards and Ewen found that ratings collected for evaluation purposes were more lenient, less reliable, less valid and contained more halo ratings 3 (1996). Accountability The previous section discussed increased accountability for both raters and ratees when 360 feedback is used for administrative decisions. This section defines accountability with regard to the 360-degree process, and offers mechanisms to enhance accountability within organizations such as goals, action plans, repeated measures and rewards. The definition of accountability and the means by which to include it in a 360 degree process developed in this section draw upon two highly regarded theories: goal setting and expectancy. Within any evaluation system, employees are expected to use feedback from performance evaluations to improve work performance. But often incentives, consequences or mechanisms to ensure employees use feedback to improve are absent. In most evaluation systems, employees meet with a supervisor annually to discuss performance and, after leaving the meeting, fail to revisit or attempt to correct areas needing improvement. Hence, employees do not feel responsible for, or accountable to, utilize feedback obtained through the evaluation process. ' Halo ratings are a type of rater error occurring when the rater gives a positively skewed evaluation bared on one performance indicator the employee performs well.

23 Additionally, mechanisms to check supervisors' ratings of employees for accuracy and fairness are customarily absent in traditional systems. Evaluations are conducted and decisions are made unilaterally with little input from others even though supervisors may not observe all dimensions of employees' performance. In short, supervisors are not accountable for providing valid evaluations. In that 360 systems incorporate multiple raters, the problem of accountability extends to all raters. Each evaluator may observe and evaluate a particular portion of the employee's performance but, in the interest of fairness, raters should bear responsibility for providing objective and factual evaluations. Accountability for raters stems from the accuracy, honesty and integrity of the feedback provided. Therefore, mechanisms should be incorporated into the process to ensure accountability for ratees so that feedback data are utilized to improve performance and for raters to ensure that accurate and fair evaluations are provided. Accountability, previously mentioned as a reason to utilize 360-degree feedback information for performance and administrative decisions, is one of the major drawbacks associated with 360 systems. Multisource rating systems often do not hold ratees accountable for utilizing the information they receive (London et al., 1997). As a result, multisource feedback has limited impact because feedback alone will not result in behavior change (Walker and Smither, 1999). To cause change, some subsequent action must be taken (Jackson and Greller, 1998). In fact, Jackson and Greller found that "360 feedback is useful but follow-up action was the most critical factor in improving performance" (Jackson and Greller, 1998). According to goal setting theory, goals, motivation and performance are related. Goals are the source of motivation and direct behavior if two conditions are present. 18

24 First, the person has a clear understanding of the goals. Second, the person believes that the goals are worth achieving. Goal setting theory also maintains that the best goals, those that motivate and direct behaviors, are specific and difficult, yet attainable (Tornow et al., 1998, p. 26). Moreover, difficult goals result in higher performance than simple goals. Goal setting theory emphasizes that performance improvement requires specific goals and specific feedback (London et al., 1997). Feedback leads to higher performance than no feedback at all. As a result, feedback plus goal setting results in larger gains than feedback alone. The goals that people set in response to feedback elicit changes in behavior. Changes in behavior and performance improvements often depend on the extent to which employees use feedback to set performance improvement goals (Walker and Smither, 1999). Expectancy theory maintains that an individual will act in a certain way based on the strength and attractiveness of the expected outcome. In terms of workplace behavior, employees assess the effort required to complete tasks or perform adequately, the reward associated with the required performance and the attractiveness of the reward before deciding how to act. Thus, expectancy theory predicts that raters weigh participation in the 360 process in terms of the probability that participation will lead to desired outcomes such as enhanced communication, the revelation of neglected supervisory performance dimensions and the identification and resolution of conflict (Westerman and Rosse, 1997). As expectancy theory also predicts, raters will expect responses, or something in return, for their input. Responses to feedback provided by raters during the evaluation process are more likely to occur if ratees create action plans with areas for 19

25 improvement linked to 360 feedback (Walker and Smither, 1999). Consequently, ratees should participate in training that teaches them how to set specific, rather than general improvement goals, and to develop formal action plans outlining specific strengths and developmental opportunities, which will increase the probability of providing responses to ratees and utilizing feedback to improve performance (Antonioni, 1996). Action plans developed soon after feedback reports are received increase the likelihood that follow-through will occur. To avoid overwhelming employees, improvement goals should be limited to three to five areas. If more than five areas are included in the plan, employees will not have time to improve in all areas (Pollack and Pollack, 1996). In as much as multisource systems lack accountability for ratees, they often do not hold raters accountable for the accuracy of the information they provide (London et al., 1997). To remedy this deficiency, an accountable environment can be created through various mechanisms focusing on the system, rewards and the participants. Feedback systems with clear sets of objectives for both raters and ratees increase accountability by setting forth the expectations and responsibilities of the system. Another means of reinforcing accountability is to repeat the 360-degree process on a regular basis. "Knowing the feedback survey will be repeated increases salience for results" (London et al., 1997). Tracking changes over time promotes accountability for filling performance gaps identified in development plans (Tornow et al., 1998). In addition, comparison of ratings across years emphasizes self-improvement (Pollack and Pollack, 1996). Multirater systems should also provide rewards for continuous improvement (Walker and Smither, 1999). Organizations could implement a variety of rewards

26 for employees who continuously meet goals developed through action plans. Some examples include salary increases, conferences, banquets, plaques, or tickets to local events. Positive reinforcement helps shape new behaviors. By the same token, recognition is necessary to reinforce new behaviors. To connect organizational expectations and employee behaviors, rewards and positive reinforcement should be linked to desired behaviors (Antonioni, 1996). In addition, supervisors, who are expected to support employees' improvement efforts, should be rewarded for providing adequate resources for subordinate development (London et al., 1997). In terms of participants, accountability for raters will improve as responsibility for providing factual and meaningful information increases. When organizations increase accountability for ratees, the expectation is that feedback will be utilized to improve performance (Walker and Srnither, 1999). There are several means to increase culpability for ratees: clear measurements of improvement by comparing feedback data across administrations of evaluations, consequences for lack of improvement, the requirement to communicate goals and discuss results with a skilled facilitator, participating in training linked to feedback results, and creating individual development plans (London et al., 1997; Walker and Smither, 1999). Resources Positive reinforcement, discussed as one means to promote behavior change in the previous section, provides motivation for employees to adapt behavior to organizational goals and requirements. Although motivation can be enhanced through positive reinforcement, some goals require additional tools to assist employees in reaching performance goals. For example, a goal of arriving to work in 21

27 a timely manner may be easily sustained with positive reinforcement, but a goal of increasing knowledge in a particular area will not be met through simple encouragement. Developing new skills and behaviors requires adequate resources; otherwise change is unlikely to occur. To the point, behavior modification is more likely to occur if it is encouraged and supported by the organization (Waldman et al., 1998). By the same token, the benefits of multisource feedback are more apt to occur if follow-up activities support the process through training and mentoring programs. Yet many organizations do not require participation in developmental or training interventions in response to feedback (Walker and Smither, 1999). On the one hand organizations provide employees with evidence of developmental needs. On the other hand, organizations do not provide employees with the tools to improve. Pollack and Pollack, maintain that "[ilt is worse to provide feedback and no support than to provide no feedback at all" (1996). Feedback should be linked to developmental planning, goal setting and organizational support. When organizations devote resources to support feedback, ratees are more inclined to use the feedback to improve performance (Walker and Smither, 1999). Organizational encouragement and support include resources for improvement, which target specific developmental areas, and support for interpreting feedback. Resources include guidelines, books, training programs, self-study, formal job opportunities, job rotation, on-the-job training, or changes in work conditions to

28 develop needed skills.4 Assistance with the interpretation of feedback is especially important because "[p]eople don't know what to do with the feedback they receive" (Wimer and Nowack, 1998). Support may be derived from coaches or colleagues who can offer assistance in setting goals and linking the process to developmental activities (Tornow et al., 1998, p. 27). Mentoring programs, shadowing, individual or one-on-one coaching with a trained facilitator or internal support groups are also helpful 5. Training Training, briefly discussed as a resource for ratees, warrants more detailed attention because the utility of the 360-degree process may suffer if inexperienced employees participate in the provision and receipt of feedback. As a result, employees need help in learning how to provide, receive, understand and use 360 degree feedback (Edwards and Ewen, 1996, p. 157). This section focuses on training for all participants and discusses participation rates in the 360 performance evaluation process in light of expectancy theory. According to expectancy theory, an individual will act in a particular situation and will base the effort exerted on a particular task on the probability of a desired outcome and on the attractiveness of that outcome. People assess the task, the ability to successfully complete the task and the outcome of the task. In other for example, Walker and Smither, 1999, Wimer and Nowack, 1998; and Edwards and Ewen, See for example, Wimer and Nowack, 1998; Edwards and Ewen, 1996; Hayworth, 1998; Pollack and Pollack,

29 words, raters consider their ability to rate effectively as one factor in deciding whether or not to participate in the 360-degree evaluation process (Westerman and Rosse, 1997). Consequently, rater training is important because, according to expectancy theory, participation may hinge on raters' comfort with conducting performance appraisals. When participation rates decline, feedback becomes less useful for both developmental and evaluative purposes (Atwater and Waldman, 1998). Expectancy theory also predicts that an individual's effort on a specific task is directly correlated to his or her ability to perform the task effectively. Since people will do what is within their abilities, training is imperative (Westerman and Rosse, 1997). Initial training for raters should introduce employees to the 360-feedback process, outline steps and instruct respondents on how to complete behavior feedback surveys and provide feedback (Edwards and Ewen, 1996, pp ). Information pertaining to anonymity and confidentiality should be included to ease participants' concerns about retaliation (Edwards and Ewen, 1996). In addition, an explanation of the purpose of the ratings as well as roles, responsibilities and accountabilities should also be included (Westerman and Rosse, 1997; Waldman et al, 1998). The more information raters have about their roles and responsibilities, the more comfortable they will be with the process as well as their participation in the process. Training also facilitates the development of a common frame of reference for all raters (Antonioni, 1996). As such, raters should develop an understanding of the intricacies of the rating process. In addition, the potential for inconsistent ratings is always present therefore training cannot be overemphasized (Tornow et al., 1998).

30 While raters are providing 360 feedback, ratees are receiving feedback. Ratees should also participate in training so that they will understand and use feedback data to improve performance. For ratees, training should review the process, instruct employees on how to interpret multirater feedback and help them develop an understanding of the information they are receiving. In addition, ratees should know how to receive the information and how to use it constructively (Edwards and Ewen, 1996, p. 121). Human Resources staff may facilitate the receipt of appraisees' first feedback and may also conduct training pertaining to analyzing data, selecting improvement targets and setting specific goals and action plans. In fact, ratees may practice setting goals with hypothetical feedback data (Antonioni, 1996). Training for both raters and ratees should include an understanding of the system, discussion on instrument development, how information is collected and used, administration method and time lines, confidentiality, how to complete the questionnaire and examples of rating errors (Westerman and Rosse, 1997). Finally, to increase accountability for feedback results, training programs should be linked to performance measures (London et al., 1997). Antonioni recommends providing training based on action plans within four months. "Immediate training improves the chance of making improvements salient enough for appraisers to notice and provide positive recognition" (Antonioni, 1996). By looking at trends of feedback data across the organization, the need for new training programs may become apparent. It is possible to tie both training and development programs to individual needs and specific key organizational competencies (Walker and Smither, 1999).

31 Rater Variance The final issue associated with 360 feedback is rater variance. The 360-degree process derives much of its strength through the incorporation of feedback data from many sources. Collectively participants possess knowledge about most aspects of the ratee's performance. With multiple raters participating in the process, the potential for variance among raters within and between groups exists. For example, an employee's superiors may find his performance exceptionally commendable. On the other hand, an employee's peers may believe that his performance is abominable. The employee will potentially receive conflicting views of his performance based on the variance in ratings between groups. Raters within a rating group may also produce opposing evaluations creating rater variance within a rating group. Jackson and Greller believe that since the purpose of 360 is to capture differing information, "differences may reflect legitimate differences in expectations by parties in very different positions in the work environment" (1998). For this reason, rater variance among different groups of appraisers is to be expected. According to Jackson and Greller, variance among raters is acceptable as long as evaluations are utilized for purely developmental purposes. When evaluations are used for discipline, promotion, pay decisions, or other consequential action, rater variance becomes a greater concern. Thus, consequential action based on feedback data may create difficulties (Jackson and Greller, 1998). Conversely, Mount et al., found that rater variance was more strongly associated with individual raters in the same group than with raters between groups. These authors believe that "[b]ecause ratings from each rater appear to Capture unique rating variance, it is important to include multiple raters in the process rather than relying on the results of a single rater" 26

32 (Mount et al., 1998). Despite the potential difficulties noted above, rater variance can provide guidance in the interpretation and use of the findings. The solution to rater variance is to consistently apply rater standards, especially if the feedback will be used for administrative purposes (Jackson and Greller, 1998). Computer analysis can identify and eliminate invalid respondents through scientific principles that reveal "judgments that differ from all others beyond a reasonable chance" (Edwards and Ewen, 1996, p. 110). With the small number of raters in each group, one outlier can skew the average scores. If the outlier is not eliminated, the average score would not represent the collective intent of the multiple respondents within the group and employees will receive inaccurate results. Outliers have the largest affect on the highest and lowest performers, yet these are the employees who need the most accurate measurement for administrative purposes and motivation to change (Edwards and Ewen, 1996, p ). Organizations utilizing 360 feedback may consider utilizing trimmed mean scoring that discounts extreme high and low responses and removes nearly all outliers. Parameters for trimmed mean scoring can be set at any level but 20% - 30% is typical. The result is that responses more than 20-30% different from others are removed. Trimmed mean scoring provides a more stable mean for a small sample than a simple mean or average. It also provides better respondent anonymity. Since everyone's score could have been eliminated, no one person can be responsible for providing low ratings (Edwards and Ewen, 1996). Therefore, ratees receive critical feedback only when a number of people feel the same way.

33 Conceptual Framework This research utilizes a conceptual framework, a tool used to organize the empirical portion of this study. The framework emerged from the literature as recurring themes, issues and considerations associated with 360 systems. Through the literature review, key aspects of 360 performance evaluation systems were classified into seven conceptual categories: organizational culture, anonymity, utilization, accountability, resources, training and rater variance (See Table 2.1). Categories are utilized when the research purpose is to describe a particular subjectin this case, 360 performance evaluation processes. The categories provide a basis for identifying and describing the attitudes of Texas State Human Resource directors about key aspects of the 360-degree process. In essence, the categories derived from the literature provide the foundation for the conceptual framework, which is then linked to questions on the survey instrument. The next section briefly reviews and justifies the conceptual categories.