EFFECTIVENESS OF BELTON EMERGENCY SERVICES PERFORMANCE APPRAISAL SYSTEM EXECUTIVE DEVELOPMENT

Transcription

1 EFFECTIVENESS OF BELTON EMERGENCY SERVICES PERFORMANCE APPRAISAL SYSTEM EXECUTIVE DEVELOPMENT BY: Steven F. Holle Fire Chief Belton Emergency Services Belton, Missouri An applied research project submitted to the National Fire Academy as part of the Executive Fire Officer Program September 2001

2 2 ABSTRACT The problem was Belton Emergency Services utilizes a personnel evaluation system that has not changed in nearly 20 years. Raters, the author included, generally disliked the evaluation process, usually waiting until the deadline to complete it. Personnel being rated, ratees, generally disliked the process as negative and highly subjective. Bias, favoritism, and even falsification have been alleged of against some raters in the past. Belton Emergency Services personnel participating in the Campbell Organizational Survey (Campbell, 1988) instrument utilized in the Executive Development class at the National Fire Academy, indicated that information may not be communicated, and feedback on performance may not be timely, accurate, or constructive. Using an evaluative research method, the purpose of this research was to evaluate the effectiveness of the personnel evaluation system within Belton Emergency Services Department. The questions to be answered were: 1. What are the elements of an effective performance appraisal system? 2. Does Belton Emergency Services have an effective performance appraisal system? 3. How could Belton Emergency Services improve the effectiveness of its performance appraisal system? Literature review initially identified criteria commonly identified with effective performance appraisal systems. From the results of research question one, the performance appraisal system was analyzed and a survey conducted of the primary stakeholders in the performance review system, the raters and ratees. The fundamental questions being asked were, Are personnel receiving feedback on their performance? and, Has that feedback motivated behavioral change, and provided for positive

3 3 development? The answer received was affirmative, however, the process by which it is achieved was very weak. The results identified fundamental problems with the personnel evaluation system, including lack of a clearly defined purpose, nonalignment with the mission and vision of the department, lack of specific performance standards, among others. Improving the system involves addressing the problems identified with the following recommendations: 1. Review Belton Emergency Services mission statement and incorporate into the evaluation. 2. Develop a vision statement and incorporate into the evaluation. 3. Review job descriptions for each level to assure they are descriptive of the skills needed to accomplish the mission. 4. Conduct a job analysis to identify critical tasks and behaviors needed to accomplish the mission. 5. Decide what form of performance evaluation tool is desired. 6. Develop specific performance criteria. 7. Train supervisors how to administer the performance evaluation. 8. Quarterly evaluations are more effective. 9. Implement a performance diary or memory file system. 10. Develop policies outlining the purpose and function of the system.

4 4 TABLE OF CONTENTS Page ABSTRACT...2 TABLE OF CONTENTS...4 INTRODUCTION...5 BACKGROUND AND SIGNIFICANCE...5 LITERATURE REVIEW...8 PROCEDURES...19 RESULTS...22 DISCUSSION...46 RECOMMENDATIONS...48 REFERENCE LIST...50 APPENDIX A...53 APPENDIX B...58 APPENDIX B APPENDIX C...73 APPENDIX C APPENDIX D...85

5 5 INTRODUCTION The problem is Belton Emergency Services utilizes a personnel evaluation system that has not changed in nearly 20 years. Raters, the author included, generally dislike the evaluation process, usually waiting until the deadline to complete it. Personnel being rated, ratees, generally dislike the process as negative and highly subjective. Bias, favoritism, and even falsification have been alleged of against some raters in the past. Belton Emergency Services personnel participating in the Campbell Organizational Survey (Campbell, 1988) instrument utilized in the Executive Development class at the National Fire Academy, indicated that information may not be communicated, and feedback on performance may not be timely, accurate, or constructive. Using an evaluative research method, the purpose of this research is to evaluate the effectiveness of the personnel evaluation system within Belton Emergency Services Department. The questions to be answered are: 1. What are the elements of an effective performance appraisal system? 2. Does Belton Emergency Services have an effective performance appraisal system? 3. How could Belton Emergency Services improve the effectiveness of its performance appraisal system? BACKGROUND AND SIGNIFICANCE Belton Emergency Services is a department of the City of Belton (Belton), a political subdivision of the State of Missouri. The department provides fire protection and emergency medical services with 26 full-time uniformed personnel and one volunteer. Staff includes the Fire Chief, Deputy Chief, five Assistant Chiefs, three Captains, and 16 Firefighter/Paramedics.

6 6 Adopted in 1979, section 2-12 (c) of Belton s code of ordinances requires the evaluation of full-time staff annually. Performance evaluations shall include, but not be limited to, industriousness, work attendance, and responsiveness to directives (City of Belton, 2001a). Performance evaluations are required for advancement in Belton s merit pay system. Advancement in the merit system is automatic, however an employee must have received his annual evaluation before a pay increase is granted (City of Belton, 2001b, p. 8). A Department Head may recommend against the advancement of an employee within the merit system, after an unsatisfactory evaluation, placing the employee on probation for six months. New employees with tenure of less than 12 months are probationary employees as well. At the end of the probationary period, unsatisfactory evaluation is considered an automatic recommendation to release the employee, except in extenuating circumstances (City of Belton, 2001b, p. 9). Since the mid-1980s, Belton Emergency Services has used an appraisal instrument modified from that used by Belton (Appendix A). The modified form, based on the original, was intended to make the evaluation form more job related. Neither Belton, nor Belton Emergency Services have written policies or procedures with regard to how personnel evaluations are conducted. Supervisor s have not received formal training on conducting personnel evaluations. Personnel evaluations are conducted annually, the month preceding an employees anniversary date, administered by the employee s immediate supervisor. The City Clerk maintains this schedule and provides a deadline for completion of the appraisal. The evaluation period consists of the previous year. The immediate supervisor rates an employee according to a scale provided for a number of evaluation elements and provides

7 7 comments as desired. Evaluations are reviewed by additional supervisors, up the chain of command, and finally approved by the Fire Chief. The supervisor conducts an interview with the employee to present the evaluation and discuss and/or clarify the ratings provided on the various elements and answer any question the employee may have. The employee has the opportunity to make written comments or rebut information contained within the appraisal. Once the appraisal interview is completed, a Personnel Action Request containing the recommended action(s) of the Fire Chief is forwarded with the evaluation to the City Clerk. The Assistant City Administrator (who serves as the personnel officer of Belton) and City Administrator review the evaluation. After review, the City Administrator approves or disapproves the recommendations made by the Fire Chief. Evaluation of the effectiveness of the personnel evaluation is significant for several reasons. The evaluation has not been reviewed or modified for nearly twenty years. Raters who have not been trained and dislike the process may not spend an appropriate amount of time or seriously consider the purpose of the performance evaluation. This may lead to large differences in how people are evaluated among a given rater and between raters, affecting the credibility of raters, reducing the credibility and therefore the effectiveness of the system. Evaluating performance appraisal effectiveness relates to a number of subjects covered in Executive Development at the National Fire Academy. Effective teamwork, professional development, leadership, research, problem solving, ethics, creativity and innovation, employee relations, organizational change and development, and service quality were all addressed in the class and may be influenced by an effective performance evaluation.

8 8 This research will provide a measure of the effectiveness of Belton Emergency Services current personnel evaluation, identify areas of weakness, and provide recommendations for improvement. An improved evaluation will foster better communication among personnel, provide better alignment with department goals, improve objectivity and specificity, be less controversial, gain greater acceptance among supervisors and personnel, and therefore improve the effectiveness of the evaluation system. LITERATURE REVIEW There are few decisions made in modern organizations that do not, somehow depend upon performance appraisal (DeNisi, 1996, p. 1). Even though the role of evaluation may be uncomfortable for many, judgments of performance are needed if performance contingent decisions, ranging from termination to pay increase and promotion, are to have any sort of rational basis (Cardy, 1998, p. 132). Interest in the evaluation of performance probably goes back well over a thousand years (see Murphy & Cleveland, 1991), but published research on appraisal goes back at least to 1920 with Thorndike s (1920) paper on rating errors (DeNisi, 1996, p. 4). For decades it has been known to researchers that one the leading causes of organizational failure is "non-alignment of responsibility and accountability." Non-alignment occurs where employees are given responsibilities and duties, but are not held accountable for the way in which those responsibilities and duties are performed (Archer North & Associates, 2000).

9 9 The purpose of this literature review is to identify the elements characteristic of effective performance reviews on which Belton s performance review system s effectiveness can be measured, and identify how the system may be improved. You should not expect to find here the recipe for the perfect appraisal system. I don t think such a recipe exists and those who might try to convince you that they can sell you such a recipe are simply after your wallet. The important thing is to listen to the customers of the process and to work toward improvement (Cardy, 1998, p. 133). Matzer & Whisenand (1988, p. 274) write in Managing Fire Services that effective performance appraisal systems have most of the following characteristics: 1. The purpose of the system is clearly defined. 2. The system is based on a job analysis, job related behavior, and clearly defined performance standards. 3. Appraisals are conducted on an ongoing basis. 4. Appraisers receive extensive training in the use of appraisal techniques and in counseling employees. Provisions are made for appraisal discussion and positive feedback. Performance strengths and weaknesses are clearly spelled out along with a clear plan of action of what is needed to correct faults and improve. Bernardin, Hagan, Kane, & Villanova, (1998, p.6), in Performance Appraisal: State of the Art in Practice offer their prescriptions for effective performance management. 1. Strive for as much precision in defining and measuring performance dimensions as is feasible. Define performance with a focus on valued outcomes.

10 10 Outcome measures can be defined in terms of relative frequencies of behavior. Prescription applies regardless of corporate strategy. Define performance dimensions by combining functions with aspects of value (that is, quantity, quality, timeliness, cost-effectiveness, need for supervision, or interpersonal impact). Incorporate the measurement of contextual performance into PM system. 2. Link performance dimensions to meeting internal and external customer requirements. Internal customer definitions of performance should be linked to external customer satisfaction. 3. Incorporate the measurement of situational constraints into the PM system. Focus attention on perceived constraints on performance through self- and supervisory rating. Bernadin, et.al. (1988, p. 28) further states that organizations must clearly and consistently communicate the firm s mission or constancy of purpose. Watson (1998) agrees arguing that: Effective performance development must be linked to the goals and objective of the organization. That principle is called alignment and it also applies to measurement. Strong alignment is the genesis of all successful measurement. In practice, most failed measurement efforts lack a clear connection to the desired business outcomes (Watson, 1998).

11 11 In general, raters do not regard constraints on performance as a very serious problem hindering the attainment of desired outcomes while those performing the work believe constraints are a serious problem (Bernardin et al., 1998, p. 32). Constraints on performance are of greater interest because they tend to account for a great deal of the disagreement between raters and ratees, disagreements that often translate into costly difficulties for the organization (Dobins et al., 1993) (Bernardin et al., 1998, p. 33). Performance standards directly influence an individual s self-set goals; an individual s personal goals regulate actions; and those aspiring to achieve difficult, specific goals perform better than those who work toward easy, vague, or no goals. (Bernardin et al., 1998, p. 17). It has been shown in numerous studies that goal-setting is an important element in employee motivation. Goals can stimulate employee effort, focus attention, increase persistence, and encourage employees to find new and better ways to work. (e.g., Locke et al., 1981) The useful [sic] of goals as a stimulus to human motivation is one of the best supported theories in management. It is also quite clear that goals which are "specific, difficult and accepted by employees will lead to higher levels of performance than easy, vague goals (such as do your best) or no goals at all" (Harris & DiSimone, 1994) (Archer North & Associates, 2000). Roberts states that goal setting is a critical complement to the development of specific, objective performance standards. Effective goal setting requires two-way communication, even with assigned goals because understanding, clarification, revision and performance monitoring entails rater-ratee information exchange (Roberts, 1994).

12 12 Bernadin (1998, p. 28) includes the need to formally recognize contextual contributions to overall performance. Contextual performance includes work behaviors that have added value to the firm above and beyond the technical core of performance. These behaviors may include spontaneously offering suggestions for improving work processes, taking the initiative to repair broken machinery, maintaining good relationships, assisting coworkers, and providing mentoring for younger workers (Bernardin et al., 1998, pp. 29). Steven T. Edwards in, Fire Service Personnel Management, believes that an organization must include performance criteria, training, and an expectation-setting meeting in the performance review system. He also stated that it is critical that there be ongoing feedback of information to the employees being evaluated (Edwards, 2000, p. 153). An effective performance review should contain the following elements: job related, significant, accurate, reliable, and legally defensible (Czerwinski, 1998). Other elements that need to be considered would be: thoroughness, accuracy, meaningfulness, and satisfaction of needs (Carter & Rausch, 1999). Objectivity is also an important characteristic of effective performance review, although according to DeNisi (1996, p. 1) there are a limited number of jobs to obtain objective performance information. He further states, Organizations are forced to rely upon some type of subjective evaluation of a person s performance which then introduces error, bias, inaccuracy, and the inherent unfairness of most performance appraisals. If employees perceive that the information is inaccurate, biased and not comprehensive, a perceived state of unfairness will result. This sense of unfairness will be exacerbated if the

13 13 employee perceives no opportunity to rebut the information or present his or her views (Roberts, 1994). Leventhal and Greenberg discussed three broad principles that influence perceived fairness in performance appraisal systems. First, employees should possess the ability to correct or rebut inaccurate information. Second, that specific procedures should be introduced to increase the accuracy of performance information (such as performance diaries, comprehensive rater training, objective performance standards). Third, that procedural safeguards are present to discourage biased ratings (appeal processes, evaluating the rater on how well they administer the process). To fully operationalize these principles, a much greater voice and role for the employee is necessary (Roberts, 1994). Participation, goal setting and feedback are formal means for interjecting employee voice into the appraisal process. Participation enables the ratee to express opinions, correct misconceptions or inaccuracies, present additional information, develop more equitable performance standards and objectives, and increase the amount and quality of rater-ratee communication. Effective goal setting requires two-way communication, even with assigned goals because understanding, clarification, revision and performance monitoring entails rater-ratee information exchange. Effective feedback requires that employees possess the ability to clarify feedback, to point out external factors that influence performance, and the opportunity to suggest means for improving performance (Roberts, 1994). DeNisi (DeNisi, 1996, p. 127) describes rating accuracy as expressed in two forms. Classification accuracy, occurs when the best performers of a group of people are identified

14 14 and ranked. The other, behavioral accuracy, occurs when desired behaviors are differentiated from those that are undesired for each individual. Rating errors includes the following as offered by the University of Arizona, in Part 3 of their Human Resources Information Series (University of Arizona, n.d.): Halo and Horns: The tendency to generalize on factor ratings based on one or two specific traits; shortcomings of an employee who is especially cooperative may be overlooked as may strengths of an employee who frequently argues. Clones: Avoiding extremes to avoid problems; rating all employees high because "they're all good," or rating them all low because "they really need to be challenged." Similar-to-me: The tendency to give slightly higher ratings to employees who are similar to you in attitudes, education, income, interests, etc. than to employees who are different from you. Contrast: Rating an employee high (or low) due to an extremely high (or low) rating given to a previous employee. Recency:

15 15 Giving an undue amount of weight to what the employee has done in the last few weeks of a rating period over what the employee has done the rest of the period. Additional error described by DeNisi includes a distribution errors such as leniency error. Leniency error occurs when the number of high ratings exceeds the normal distribution (DeNisi, 1996, p. 184) such as too many A s given for a class (DeNisi, 1996, p. 168). DeNisi (DeNisi, 1996, p ) argues that while rating accuracy and reduction in error are important concepts for effective appraisals, that there may not be the cause and-effect relationship between accuracy and presence or absence of error that has been assumed. For example, with respect to leniency error: Unless we know the true distribution of performance in a work group, it makes no sense to discuss things such as leniency error. And, of course, we don t know the true underlying distribution, or we wouldn t need to do performance appraisals. So how many A s are too many? Perhaps my students are just very bright, or perhaps I am an excellent instructor. Furthermore, in organizational settings, the stated purpose of performance management systems is to improve employee performance. If such systems are effective, wouldn t an accurate distribution of ratings look like leniency (DeNisi, 1996, p. 168)? DeNisi argues that traditionally, efforts to remove error from the rating process have been assumed to improve accuracy. The problem is there are no clear conceptual and operational definitions of most forms of error, even halo error that has been widely researched. Therefore, no clear relationship between the presence of error and rating accuracy can be established (DeNisi, 1996, p ).

16 16 DeNisi then addresses the question of the proper criteria for evaluating appraisal effectiveness (DeNisi, 1996, p. 168). The goals of the appraisal system are more important than other criteria of appraisal effectiveness. At the simplest level, appraisals are done to aid in decision making (i.e., merit-pay increases) and to provide developmental feedback. We need to go back and reconsider the reasons for organizations conducting appraisals in the first place. One reason is to help in decision making. Here we might want accurate ratings because we hope that merit-pay increases and the like will motivate employees to work harder on their jobs. If the ultimate goal were performance improvement (and I believe this is the ultimate goal for all appraisals), then we should be able to relate ratings to some outcome measures indicating that performance has improved following an accurate appraisal. But notice that, even here, accuracy is only important because we assume that, at some level, everyone knows who is the best performer, and when that person gets rewarded, it motivates everyone else to try harder to perform at the same level. What s really important, then, is that the ratees perceive the decision as being fair. Accurate ratings may help ensure that, but we really don t know. What we do believe we know, however, is that employees will be more motivated working under systems that they perceive as fair (DeNisi, 1996, p. 128). Thus, in the field, rater and ratee attitudes towards the process and the outcomes (i.e. perceptions of distributive and procedural justice; see Folger, Konovksy, & Cropanzano, 1992) of appraisal may be more important to the ultimate effectiveness of any appraisal system (DeNisi, 1996, p. 129).

17 17 One might imagine, in other organizations, such goals as improved team performance, reduced costs, or even higher levels of job satisfaction. An effective appraisal system, then, would be one that maximized these goals rather than the goal of rating accuracy. Note again, it is not that these goals would be operating counter to the goal of trying to make ratings more accurate (whether we are interested in classification or behavioral accuracy), they would simply make that goal somewhat irrelevant (DeNisi, 1996, p. 129). A performance appraisal system is more likely to be accepted when the appraisal system is congruent or doesn t conflict with the employee s personal goals and values, and when the appraisal process does not exceed the bounds of the psychological contract between rarer [sic] and ratee (Roberts, 1994). Training of raters is an important part of any performance appraisal system. Helping raters to be more expert in their evaluations by providing clear standards and training raters how to use those standards is generally seen as more promising than training raters not to commit halo error, and so imply replacing one response tendency with another (DeNisi, 1996, p. 127). Every employee, including all manages and subordinates, must have clear understanding of the appraisal system. Everyone needs to understand the goals of the process; without understanding, the process will be fraught with confusion and mistrust. Consistent and ongoing training for managers and supervisors in the application of the system is essential for its success and to make it valuable to the employees and the organization. The appraisal process must be conducted skillfully and efficiently or the system will lose its credibility (Czerwinski, 1998, p ).

18 18 To be effective is defined as having an intended or expected effect ( Section 2-12 (c) of Belton s code of ordinances requires the evaluation of full-time staff annually. Performance evaluations shall include, but not be limited to, industriousness, work attendance, and responsiveness to directives (City of Belton, 2001a). Job analysis produces a description or specifications of the job, and can be used in performance review to identify or develop goals and objectives, performance standards, evaluation criteria, length of probationary periods, and the duties to be evaluated (Job- Analysis.Network, 1999). Employee attitudes influence the attitudes of raters. If employees are hostile and reject the system, raters may be unwilling or unable to effectively implement performance appraisal due to the costs attached t employee resistance. Raters that lack motivation to effectively implement the process will meet only the minimum requirements, and with performance appraisal, the minimum is rarely sufficient. This lack of motivation will lead to rating errors, general or ambiguous performance feedback, inadequate performance documentation, and a lack of confidence in the utility of performance appraisal information. This, in turn, can result in a system that is markedly less effective or totally ineffective (Roberts, 1994). There are no written goals, policies, or guidance provided for Belton s performance appraisal system at the departmental or city level. The literature review provided elements for a survey the author developed to obtain input from the personnel within Belton Emergency Services most affected by performance appraisals. All personnel who receive performance appraisals (Firefighter/Paramedic, Captain, Assistant Chief, and Deputy Chief) were requested to rate common goals, elements of effectiveness, and

19 19 perceptions as identified through the literature review. The fundamental question being asked is are personnel receiving feedback on their performance, has that feedback motivated behavioral change, and provided for positive development. The survey instrument and the results are contained in Appendix B. Another survey instrument was developed for supervisors that administer personnel evaluations (Deputy Chief, Assistant Chief, and Captains). The rating elements were the same as the survey in Appendix B, with the exception that the questions were to be answered from the supervisory point of view. In this case, the fundamental question to be answered was are personnel being provided with developmental feedback and responding with positive behavioral change. The survey instrument and the results are contained in Appendix C. PROCEDURES A literature search was conducted at the Learning Resource Center at the National Fire Academy in March Some resources were available at the Cass County Public Library, although most were found through the online card catalogues of the Learning Resource Center at the National Fire Academy, and the Mid-Continent Public Library ( Access to several online databases offering newspaper, magazine, and encyclopedic resources was very helpful from the Mid-Continent Public Library web site. There are extensive literary resources available concerning performance appraisal. Resources range across various texts, trade journals, research papers, articles, opinions, discussion groups, and the World Wide Web on management, human resources, performance management, performance appraisal, performance evaluation, etc. The literature review answered research question one defining components of performance evaluations that made them effective. The literature spanned many industries and complemented

20 20 one another. One author, DeNisi called into question the importance of accuracy and error avoidance, in lieu of designing the appraisal for personnel confidence and acceptance. DeNisi identified that the achievement of the goals of the appraisal system ultimately defined the effectiveness of the system. To answer the second question, the goals of Belton Emergency Services evaluation had to be determined. Belton Emergency Services has no written policies or goals for it personnel appraisal system. Elements to be discussed and the annual appraisal period were identified in the Belton Code of Ordinances (City of Belton, 2001a). At the simplest level, appraisals are done to aid in decision making (i.e., merit-pay increases) and to provide developmental feedback (DeNisi, 1996, p. 168) and performance improvement (DeNisi, 1996, p. 128). Other evaluation systems were also examined to see how their goals differed, and what information was shared between the supervisors and employees. The University of Arizona (U.A., n.d.) was used as a model providing elements of discussion during a performance appraisal interview. Many of these mirrored elements identified in research question 1 as important elements for effective performance appraisals. A survey was designed to determine if the fundamental goals identified by DeNisi were being achieved. Additional elements, from the University of Arizona mirrored were included to rate the thoroughness of discussion being conducted between Belton Emergency Services supervisors and staff. The rationale for the survey questions is included with the results of research question one. Because of the subjective nature of Belton s evaluation system, there was no quantitative means to determine the effectiveness by review of individual documents. Effectiveness is largely

21 21 a degree of perception of those participating in the process. The primary stakeholders in the appraisal process are Belton Emergency Services, the department supervisors administering the evaluations, and the employees themselves. The results of the survey identified that at a fundamental level identified by DeNisi, Belton Emergency Services performance appraisal is effective. The strength of effectiveness is undefined, however a number of elements identified in research question one are absent, or limited in scope leading one to characterize Belton Emergency Services appraisal system as weakly effective. The survey results, review of Belton Emergency Services appraisal instrument, policies and procedures, and employee comments as related to the results of research question one identified areas of weakness which then could be addressed in research question 3. Some employees provided direct suggestions for improvement and were included in the results of question 3. Assumptions and Limitations An assumption was made that respondents to the surveys understood each question and had the knowledge and ability to answer them accurately. It is assumed that staff tenure has little effect on the overall perception of the performance appraisal system. Because of the small number of personnel on the department, despite efforts to limit the influence of bias and assure respondents the ability to respond freely to the questions asked, some respondents may have felt constrained in their responses. Supervisors were included in both survey populations. It is assumed that they successfully understood the questions and responded to each from the proper point of view.

22 22 Of seven supervisor surveys distributed only five were returned, challenging the validity of the results. The statistical significance of the responses and/or differences has not been determined. The questions on the survey instruments are assumed to be valid. The construction of questions for redundancy, and different weighting are assumed to be irrelevant. The author, may have missed significant research material related to this applied research which could have changed the results of the research. The terms supervisor, rater, and employer are used interchangeably. The terms employee, ratee, and staff are used interchangeably. The terms personnel evaluation, personnel appraisal, and performance review are used interchangeably. RESULTS 1. What are the elements of an effective performance appraisal system? The literature review provided the following answer to question one and formed the basis for development of surveys to answer research question two. First, as Matzer & Whisenand (1988, p. 274) write, an effective performance appraisal system must have a clear defined purpose. DeNisi (1996) agrees while arguing the question of the proper criteria for evaluating appraisal effectiveness, that the goals of the appraisal system are going to determine the most important criteria for effectiveness. At the simplest level, appraisals are done to aid in decision making (i.e., merit-pay increases) and to provide developmental feedback (DeNisi, 1996, p. 168). For example, DeNisi (1996, p. 128) writes that the ultimate goal, performance improvement (and I believe this is the

23 23 ultimate goal for all appraisals) should be related to some outcome measures indicating that performance has improved. Bernardin et al. (1998, p. 6) share similar thoughts, defining performance with a focus on valued outcomes. Depending on the organization, additional goals may be addressed in performance review. The University of Arizona (U.A., n.d.) uses the following guide to define what they expect to communicate in a performance review: Department goals and objectives--how they affect the employee's job. Employee's duties/priorities--what's important to do; in what order. Expectations/standards for employee--required results (quality, quantity). Feedback on employee's performance--strengths and areas to improve. Feedback on supervisor's performance--effectiveness in supervisor role. Other factors--affect of unusual circumstances (personal, medical). Changes expected for department/job--resources, standards, methods. Suggestions for improvement--better methods, overcoming barriers. Career planning for employee--transfer, promotion, external interests. Employee training and development needs--professional and personal growth. From the list it can be seen that professional development, and training needs of the employee are also goals of the system. Second, performance criteria based on a job analysis, job related behavior, and clearly defined performance standards (Matzer & Whisenand, 1988, p. 274; Edwards, 2000, p. 153; Czerwinski, 1998) with as much precision in defining and measuring performance dimensions as is feasible Bernardin et al. (1998, p. 6). Job analysis produces a description or specifications of the job, and can be used in performance review to identify or develop goals and objectives, performance standards,

24 24 evaluation criteria, length of probationary periods, and the duties to be evaluated (Job- Analysis.Network, 1999). Bernadin et al. (1998, p. 28) further states that organizations must clearly and consistently communicate the firm s mission or constancy of purpose. Watson (1998) agrees arguing that Effective performance development must be linked to the goals and objective of the organization. A principle called alignment. For decades it has been known to researchers that one the leading causes of organizational failure is "non-alignment of responsibility and accountability." Non-alignment occurs where employees are given responsibilities and duties, but are not held accountable for the way in which those responsibilities and duties are performed (Archer North & Associates, 2000). Job related behavior includes: attendance, punctuality, dependability, relationships with co-workers and the public, acceptance of supervision, initiative/creativity, safety consciousness and the fulfillment of safety and loss control responsibilities, work habits (diligence, effective use of time, etc.), adherence to unit policies and procedures, cooperation/teamwork, customer service, communication/interpersonal skills, or other aspects of the employee s job-related behavior which affect overall work performance (both negatively or positively) (Oregon State University, 1999). Clearly defined performance standards must be specific, pertinent, attainable, measurable, and observable (SPAMO) (U.A., n.d.). Specific--spells out the who, what, when, where, and how of the desired level of performance. Written standards help prevent misunderstandings.

25 25 Pertinent--clearly related to the job. It's importance and relevance should be clear. Attainable--possible for the typical person in that position to perform at the required level. Unattainable goals frustrate, rather than motivate, most people. Measurable--defines, when possible, what will be measured to determine a successful level of performance. Measures usually involve one or more of the following: quantity, quality, time, or cost. Samples or spot-checks can simplify this process. Observable--should be able to see either the performance or the results of the performance (timely paperwork, resolved problems). These elements define, improve accuracy, and add objectivity and credibility to the performance appraisal. They also reduce subjectivity and bias, and address the thoroughness, accuracy, meaningfulness, and satisfaction of needs referred to by Carter & Rausch (1999). Bernadin et al. (1998, p. 28) defines performance dimensions by combining functions with aspects of value (that is, quantity, quality, timeliness, cost-effectiveness, need for supervision, or interpersonal impact) and link performance dimensions to meeting internal and external customer requirements. DeNisi (1996, p. 1) writes there are a limited number of jobs to obtain objective performance information and organizations are forced to rely upon some type of subjective evaluation of a person s performance. Bernardin et al. (1998, p. 6) suggests, incorporating the measurement of situational constraints into the PM [performance management] system. A number of factors beyond the ratees control can affect performance. For instance, a computer system could fail during a critical period before a deadline. Considering the constraint on performance adds fairness to the evaluation of the ratees performance.

26 26 Third, appraisals must be conducted on an ongoing basis (Matzer & Whisenand, 1988, p. 274; Edwards, 2000, p. 153). Reh (1998) argues annual performance reviews are a waste of time because they are too infrequent. More frequent, less formal feedback, is better. If you have to do formal reviews, do them quarterly. Most of us can actually remember what has happened over the last three months. Be sure to spend less than a quarter of the time doing them that you would have for an annual review. Then when you have to do an annual review you can just review the past three months and tack on the three previous quarterly reviews. The best performance review is the ongoing review. Work with your employees on how they are doing on a task by task basis - daily or weekly. Let them know right away what they are doing wrong so they can correct it now, not twelve months from now when it has become an ingrained habit (Reh, 1998). Fourth, appraisers receive extensive training in the use of appraisal techniques and in counseling employees (Matzer & Whisenand, 1988, p. 274). Czerwinski (1988, p. 32) agrees stating, Consistent and ongoing training for managers and supervisors in the application of the system is essential for its success and to make it valuable to the employees and the organization. Training raters to avoid common rating errors has been a focus of some training programs. DeNisi writes: Helping raters to be more expert in their evaluations by providing clear standards and training raters how to use those standards is generally seen as more promising than training raters not to commit halo error, and so imply replacing one response tendency with another (DeNisi, 1996, p. 167).

27 27 The fifth element of effective performance appraisals identified by Czerwinski (1998) is that it should be legally defensible. According to HRTools.com (1999), legally defensible performance appraisals have the following characteristics: 1. be in writing 2. contain specific procedures 3. specify documentation to be used and in what manner documentation is to be gathered 4. include specific instructions for supervisors, provide training for supervisors in how to evaluate and administer the system 5. use standardized forms for related groups of employees 6. establish objective and/or measurable job criteria 7. be thoroughly communicated to employees 8. be given formally at least on an annual basis 9. evaluate specific work behavior and not personal traits 10. and periodically review the system. The sixth element of effective performance appraisals is providing employees with procedural justice and a voice in the process. Employee acceptance is a critical component of an effective performance appraisal system. Ratee acceptance is maximized with the performance measurement process is perceived to be accurate, the system is administered fairly, the appraisal system is congruent or doesn t conflict with the employee s personal goals and values, and when the appraisal process does not exceed the bounds of the psychological contract between the rarer [sic] and ratee (Roberts, 1994).

28 28 The ability to correct or rebut inaccurate information, introducing specific procedures to increase the accuracy of performance information (such as performance diaries, comprehensive rater training, objective performance standards), and providing procedural safeguards to discourage biased ratings (appeal processes, evaluating the rater on how well they administer the process) (Roberts, 1994) improve the perceived fairness of the process. Participation, goal setting and feedback are formal means for interjecting employee voice into the appraisal process. Participation enables the ratee to express opinions, correct misconceptions or inaccuracies, present additional information, develop more equitable performance standards and objectives, and increase the amount and quality of rater-ratee communication. Effective goal setting requires two-way communication, even with assigned goals because understanding, clarification, revision and performance monitoring entails rater-ratee information exchange. Effective feedback requires that employees possess the ability to clarify feedback, to point out external factors that influence performance, and the opportunity to suggest means for improving performance (Roberts, 1994). In summary, the elements described above are characteristic of effective performance appraisals. Two surveys were developed for Belton Emergency Services personnel to complete. One survey was for all Belton Emergency Services personnel that receive performance appraisals by interdepartmental personnel. The second was for Belton Emergency Services supervisors who administer performance appraisals. The questions provided below contain the language of the personnel survey. The inflection of the supervisor survey language was changed to reflect the point of the person administering the evaluation.

29 29 1. I have been the subject of one or more performance evaluations at Belton Emergency Services. Intended only to include responses of personnel that have participated in a Belton Emergency Services performance review. 2. I have discussed the department s vision, mission, and goals and objectives during my performance evaluation along with my role in achieving the vision, mission, goals or objectives. Organizations must clearly and consistently communicate the firm s mission or constancy of purpose (Cardy & Dobbins, 1994; Deming, 1986) (Bernardin et al., 1998, p. 28). For decades it has been known to researchers that one the leading causes of organizational failure is "non-alignment of responsibility and accountability. Non-alignment occurs where employees are given responsibilities and duties, but are not held accountable for the way in which those responsibilities and duties are performed (Archer North & Associates, 2000). Watson (1998) said that effective performance development must be linked to the goals and objective of the organization. That principle is called alignment and it also applies to measurement. Strong alignment is the genesis of all successful measurement. In practice, most failed measurement efforts lack a clear connection to the desired business outcomes. Department goals and objectives--how they affect the employee's job (UA, n.d.). 3. My duties and the priority of my duties are discussed. Employee's duties/priorities--what's important to do; in what order (UA, n.d.).

30 30 4. Clear expectations and standards for completion of my duties are discussed. Expectations/standards for employee--required results (quality, quantity) (UA, n.d.). 5. I receive feedback on my performance (strengths and areas to improve) during my evaluation. Feedback on employee's performance--strengths and areas to improve (UA, n.d.). 6. Factors that affect my performance are discussed (personal, medical, departmental issues beyond my control, etc.). Other factors--affect of unusual circumstances (personal, medical) (UA, n.d.). In general, raters do not regard constraints on performance as a very serious problem hindering the attainment of desired outcomes while those performing the work believe constraints are a serious problem (Bernardin et al., 1998, p. 32). Constraints on performance are of greater interest because they tend to account for a great deal of the disagreement between raters and ratees, disagreements that often translate in to costly difficulties for the organization (Dobins et al., 1993) (Bernardin et al., 1998, p. 33). Effective feedback requires that employees possess the ability to clarify feedback, to point out external factors that influence performance, and the opportunity to suggest means for improving performance (Roberts, 1994).

31 31 7. Changes expected by the department that affect resources, standards, methods, etc. are discussed. Changes expected for department/job--resources, standards, methods (UA, n.d.). 8. Suggestions for improvement are discussed (changes in procedures, methods, etc. that will assist the department achieve it s vision, mission, goals and objectives). Suggestions for improvement--better methods, overcoming barriers (UA, n.d.). Effective feedback requires that employees possess the ability to clarify feedback, to point out external factors that influence performance, and the opportunity to suggest means for improving performance (Roberts, 1994). 9. Personal goals and objectives are discussed. I receive counseling to achieve those goals. Performance standards directly influence an individual s self-set goals; an individual s personal goals regulate actions; and those aspiring to achieve difficult, specific goals perform better than those who work toward easy, vague, or no goals. (Bernardin et al., 1998, p. 17). A performance appraisal system is more likely to be accepted when the appraisal system is congruent or doesn t conflict with the employee s personal goals and values, and when the appraisal process does not exceed the bounds of the psychological contract between rarer [sic] and ratee (Roberts, 1994).

32 My professional development is discussed with suggestions for professional and personal growth. Career planning for employee--transfer, promotion, external interests (UA, n.d.). Performance standards directly influence an individual s self-set goals; an individual s personal goals regulate actions; and those aspiring to achieve difficult, specific goals perform better than those who work toward easy, vague, or no goals. (Bernardin et al., 1998, p. 17). 11. The frequency of performance evaluations is adequate. Appraisals are conducted on an ongoing basis (Matzer & Whisenand, 1988, p. 274). And that is my point. ANNUAL performance reviews are a waste of time because they are too infrequent. So What s Better? Well, more frequent is better. And less formal is better. Better for both you and the employee. If you have to do formal reviews, do them quarterly. Most of us can actually remember what has happened over the last three months. Be sure to spend less than a quarter of the time doing them that you would have for an annual review. Then when you have to do an annual review you can just review the past three months and tack on the three previous quarterly reviews.

33 33 The best performance review is the ongoing review. Work with your employees on how they are doing on a task by task basis - daily or weekly. Let them know right away what they are doing wrong so they can correct it now, not twelve months from now when it has become an ingrained habit. Be sure to also point out the things they are doing well. Remember, we all like praise. Remember too that if you neglect to tell them that something they are doing is right, they may not realize it and they may change it. (Reh, 1998). 12. Performance evaluations are accurate. Here we might want accurate ratings because we hope that merit-pay increases and the like will motivate employees to work harder on their jobs (DeNisi, 1998, p. 128) Accurate feedback is more likely to result in behavioral changes in the desired direction (DeNisi, 1998, p. 169) An effective performance review should contain the following elements: job related, significant, accurate, reliable, and legally defensible (Czerwinski, 1998). In addition, other elements that need to be considered would be: thoroughness, accuracy, meaningfulness, and satisfaction of needs (Carter & Rausch, 1999). 13. Performance evaluations are fair. Given the subjective nature of these appraisals, it is not surprising that there have been volumes written about the errors, bias, inaccuracy, and inherent unfairness of most performance appraisals (DeNisi, 1998, p. 1).

34 34 What s really important, then, is that the ratees perceive the decision as being fair. that employees will be more motivated working under systems that they perceive as fair (DeNisi, 1998, p. 128). If employees perceive that the information is inaccurate, biased and not comprehensive, a perceived state of unfairness will result. This sense of unfairness will be exacerbated if the employee perceives no opportunity to rebut the information or present his or her views (Roberts, 1994). 14. Criteria to achieve a given rating are specific, pertinent, attainable, measurable, and observable. "SPAMO" Guidelines for Effective Standards Specific--spells out the who, what, when, where, and how of the desired level of performance. Written standards help prevent misunderstandings. Pertinent--clearly related to the job. It's importance and relevance should be clear. Attainable--possible for the typical person in that position to perform at the required level. Unattainable goals frustrate, rather than motivate, most people. Measurable--defines, when possible, what will be measured to determine a successful level of performance. Measures usually involve one or more of the following: quantity, quality, time, or cost. Samples or spot-checks can simplify this process. Observable--should be able to see either the performance or the results of the performance (timely paperwork, resolved problems) (U.A., n.d.).

35 I have changed my work habits as a result of a performance evaluation. Performance evaluations will allow the employee to understand what future action should be taken: to continue acceptable performance or to correct unacceptable behavior (Czerwinski, 1998, p 33). If the ultimate goal were performance improvement (and I believe this is the ultimate goal for all appraisals) (DeNisi, 1996, p. 128). 16. Performance evaluations have helped the department achieve its vision, mission, goals, or objectives. Organizations must clearly and consistently communicate the firm s mission or constancy of purpose (Cardy & Dobbins, 1994; Deming, 1986) (Bernardin, et.al., 1998, p. 28). For decades it has been known to researchers that one the leading causes of organizational failure is "non-alignment of responsibility and accountability." Non-alignment occurs where employees are given responsibilities and duties, but are not held accountable for the way in which those responsibilities and duties are performed (Archer North & Associates, 2000). Department goals and objectives--how they affect the employee's job (UA, n.d.). 17. My strengths and achievements are recognized. Provisions are made for appraisal discussion and positive feedback. Performance strengths and weaknesses are clearly spelled out along with a clear plan of action of what is needed to correct faults and improve performance (Matzer & Whisenand, 1988, p. 274).

36 36 Raters need to formally recognize contextual contributions to overall performance. Contextual performance includes work behaviors that have added value to the firm above and beyond the technical core of performance. These behaviors may include spontaneously offering suggestions for improving work processes, taking the initiative to repair broken machinery, maintaining good relationships, assisting coworkers, and providing mentoring for younger workers (Bernardin et al., 1998, pp ). 18. Performance evaluations identify functional areas of my performance needing improvement and establish agreement about how improvement is to be accomplished. 19. Our organization has a good performance appraisal system in place. Our organization has good performance appraisal systems in place (Campbell, 1988). 20. Feedback on performance for people at my level is timely, accurate, and constructive. Feedback on performance for people at my level is timely, accurate, and constructive (Campbell, 1988). 21. The effect of my performance upon our customers is discussed. Internal customer definitions of performance should be linked to external customer satisfaction (Bernardin et al., 1998, p. 6).

37 I am willing to assist in improving the performance appraisal system. To be effective, the performance appraisal system must have the support of the participants. To achieve and/or maintain support requires that the participants have input into its design. If very few personnel are willing to assist in improvement of the system, then building credibility among participants will be more difficult, maybe impossible. 23. I have the opportunity to correct or rebut inaccurate information. There are three broad principals that influence perceived fairness in peformance appraisal systems. First, employees should possess the ability to correct or rebut inaccurate information (Roberts, 1994). 24. Personnel evaluations are biased. If employees perceive that the information is inaccurate, biased and not comprehensive, a perceived state of unfairness will result. This sense of unfairness will be exacerbated if the employee perceives no opportunity to rebut the information or present his or her views (Roberts, 1994). Organizations are forced to rely upon some type of subjective evaluation of a person s performance which then introduces error, bias, inaccuracy, and the inherent unfairness of most performance appraisals (DeNisi, 1996). 25. My supervisor and I establish goals to improve my performance and I have the opportunity to clarify and revise how I may meet my goals.

38 38 It has been shown in numerous studies that goal-setting is an important element in employee motivation. Goals can stimulate employee effort, focus attention, increase persistence, and encourage employees to find new and better ways to work. (e.g., Locke,et al., 1981). The useful [sic] of goals as a stimulus to human motivation is one of the best supported theories in management. It is also quite clear that goals which are "...specific, difficult and accepted by employees will lead to higher levels of performance than easy, vague goals (such as do your best) or no goals at all." (Harris & DiSimone, 1994) (Archer North & Associates, 2000). Goal setting is a critical complement to the development of specific, objective performance standards. Effective goal setting requires two-way communication, even with assigned goals because understanding, clarification, revision and performance monitoring entails rater-ratee information exchange (Roberts, 1994). 26. I have been adequately trained to conduct performance appraisals (Supervisor survey only). Appraisers receive extensive training in the use of appraisal techniques and in counseling employees (Matzer & Whisenand, 1988, p. 274). Czerwinski (1988, p. 32) agrees stating, Consistent and ongoing training for managers and supervisors in the application of the system is essential for its success and to make it valuable to the employees and the organization. Helping raters to be more expert in their evaluations by providing clear standards and training raters how to use those standards is generally seen as more promising than training

39 39 raters not to commit halo error, and so imply replacing one response tendency with another (DeNisi, 1996, p. 167). 2. Does Belton Emergency Services have an effective performance appraisal system? Literature review, supervisor, and staff surveys provided the answers to this question. A clearly defined purpose is the first element required for an effective performance evaluation system. As previously stated, Belton has no written policies or procedures defining the purpose or goals of the performance evaluation system. Therefore, only the simplest goals of performance improvement and to aid in decision making and provide developmental feedback (DeNisi, 1996) are assumed. Question 5 asks whether feedback (strengths and needs for improvement) is communicated between supervisors and staff during an evaluation. The weighted scores of both surveys provide strong agreement that this takes place. Question 15 of the surveys asks the whether performance evaluations caused a change in work habits. All respondents of the staff survey answered in agreement with the statement, while a majority of supervisors answered in agreement. At this most fundamental level, the system is effective. In addition, the purpose should be linked to the goals and objectives of the organization, aligned with its mission and vision. Belton Emergency Services has a very simple, broad, mission statement that may not be specific enough from which to develop meaningful performance objectives. In addition, there is no vision statement on which to define the departments future direction. Each position has a job description, however, specific performance criteria are contained within the appraisal instrument

40 40 for a given position. The appraisal instrument does include additional evaluation elements for personnel with supervisory responsibility. Staff and supervisor survey questions 2, 8, 16, and participant comments generally deal with the link the goals of the performance evaluation with the vision and mission of the department. The general consensus among the participants is that the current system does not adequately communicate and/or link the mission, vision, goals and objectives of the department to performance appraisal. Questions 2, 3, 4, 5, 6, 7, 8, 10, 14, 16, and 17 incorporate the goals and subject matter of the University of Arizona as representative of a thorough performance appraisal. Weighted scores between supervisors and staff provide general agreement on questions 2, 3, 4, 5, 6, 16, and 17. Questions 7, 8, 9, 10 and 14 indicated disagreement between the raters and ratees. The statistical significance of this disagreement is questionable because of the small samples involved, however, it is interesting that where these differences occur, the supervisors indicate agreement with the statement that a given element is discussed, while the staff indicate disagreement. This indicates a lack of clear direction for supervisors in conducting performance evaluations. Some supervisors may regularly include a number of these elements; others may not include any. Knowing the tenure of the staff receiving evaluations and their relative scores would provide more insight. Do longer-term employees have a more positive or negative view of the performance evaluation system than newer employees do? Second, performance criteria based on a job analysis, job related behavior, and clearly defined performance standards (Matzer & Whisenand, 1988, p. 274; Edwards, 2000, p. 153;

41 41 Czerwinski, 1998) with as much precision in defining and measuring performance dimensions as is feasible Bernardin et al. (1998, p. 6). A look at the appraisal instrument in use (Appendix A) reveals a document lacking specific performance criteria. Each rating element utilizes the same five-level rating scale with no criteria to achieve any given rating. Each rating element is highly subjective, and the evaluator determines what behaviors are taken into consideration under a given element are included. This leads to a great deal of variability among raters and between employees rated. Questions 4 and 14 of the survey seem to offer conflicting results related to this element. Supervisors and staff generally agreed with question four that clear expectations and standards for completion of employee duties are discussed during performance evaluations. The response to question 14 states criteria to achieve a given rating are specific, pertinent, attainable, measurable, and observable was mixed. Supervisors weakly responded in agreement, while staff participants responded in disagreement. Employee and supervisor perceptions of the two questions may have been somewhat different because of the more specific definition provided in question 14. It also identifies weakness in the performance appraisal system; the absence of clear standards, a poorly structured appraisal instrument, and lack of employee and supervisor training. The third element identified was that performance appraisals must be ongoing. Annual appraisals are conducted. Question 11 asks if the frequency of performance evaluations is adequate. Supervisors and staff generally responded in agreement, however individual comments indicated otherwise, citing a lack of ongoing feedback. The literature review indicates that feedback must be ongoing and suggests performance appraisals be conducted quarterly. This element is minimally effective.

42 42 The fourth element relates to rater training which the literature review identifies as important. Question 26 of the supervisor survey inquires of supervisors whether they felt they were adequately trained to conduct performance evaluations. Supervisors responded with disagreement and is supported by elements considered earlier in these results. Fifth, is an appraisal system that is legally defensible. The elements were identified through literature review and are reflected the answer to research question one. Belton Emergency Services appraisal system is in writing, contains a limited number of specific procedures, uses a standardized form for all employees, and is formally administered on an annual basis. The system lacks specific documentation and how that documentation is to be gathered, specific instructions for supervisors and training in how to evaluate and administer the system, or objective and/or measurable job criteria. In addition, the performance appraisal system has not been thoroughly communicated to employees, the structure of the system does not guarantee the evaluation of specific work behavior, and the system has not been reviewed for nearly 20 years. The sixth element identified was the addition of procedural justice and a voice in the process for employees through participation, goal setting, and feedback (Roberts, 1994), to maximize communication and employee acceptance of the performance appraisal system. On question 23 of the surveys raters and ratees agree that ratees have the ability to rebut, correct, or respond to information contained in the Belton Emergency Services personnel evaluation. Participants in the surveys generally agree that the performance evaluations are accurate (question 12) and fair (question 13), and disagree that performance evaluations are biased (question 24). This perception is not universal, however, where one participant comments that

43 43 evaluations have been used to threaten and/or punish people. Employee acceptance, therefore, is an individual characteristic. In summary, Belton Emergency Services performance appraisal system is effective in the most fundamental way. The assumed goals of the system, performance improvement and to aid in decision making and provide developmental feedback (DeNisi, 1996) are being met. Several areas of weakness are identified through the results whereby the effectiveness of the performance appraisal system can be improved. 3. How could Belton Emergency Services improve the effectiveness of its performance appraisal system? Improvements of Belton Emergency Services performance appraisal system are derived from the weaknesses identified in research question 2. Fred Nichols (2000) article Don't Redesign Your Company's Performance Appraisal System, Scrap It, advocates eliminating performance appraisal systems because of the high costs involved in designing and administering appraisals, training raters, and handling postappraisal appeals, grievances, or lawsuits with relation to the benefits received. Section 2-12 (c) of Belton s code of ordinances requires the evaluation of full-time staff annually. Therefore, discarding the performance appraisal system is not an option without the action of the City of Belton. Having a clearly defined purpose aligned with the mission, vision, goals and objectives of the department has been identified through the literature review and surveys as an important, yet missing component of Belton Emergency Services performance evaluation system. Review of Belton Emergency Services mission statement, the development of a vision statement, and the setting of specific goals for the performance evaluation system must be defined and added to the policy manuals of the department.

44 44 A job analysis to review job descriptions in relation to the mission statement must be conducted. Supervisors and staff should develop specific performance criteria for each job description. The literature review indicates that different appraisal methods have advantages and disadvantages. Supervisors and staff should select a method that best meets the goals established for the evaluation system (Archer North & Associates, 2000). Modification or replacement of the existing appraisal instrument incorporating the performance criteria developed will provide a much more specific and consistent evaluation of performance. Ongoing feedback was identified as an important component of performance appraisal. While most survey participants indicated that annual appraisals were adequate, several significant comments were received regarding new probationary employees and the lack of feedback before receiving the evaluation. Reh (1998) suggests more frequent feedback, quarterly evaluations, and the University of Arizona (n.d.) suggests the use of a diary or memory file to assure a thorough and fair appraisal. It is especially useful with long periods between evaluations. This can be as simple as a file folder. It can include notes on exceptional (positive or negative) performance or behavior by the employee. It can also include copies of exceptional documents such as letters of appreciation [or] of warning. By keeping a memory file on each employee, a supervisor can track an employee's performance progress as well as what coaching has been provided. Depending on how accessible and complete that file is, the employee may want to set up his own memory file to ensure all key aspects of his performance are recorded (U.A., n.d.).

45 45 Captain Dana Marconett submitted a policy proposal (Appendix D) for probationary employees. The proposal has the characteristics of frequent feedback (positive and negative), some performance criteria, and a memory file. The concept could be expanded to all employees. Rater training must be conducted for improved performance evaluations. The survey results indicated a lack of consistency among raters, lack of direction by the instruments provided to them. Rater participation in goal setting, job analysis, and setting of performance criteria will provide additional information to the raters on the purpose and expectations of the performance evaluation system. Additional training, such as a section entitled Performance Review and Goal Setting Information for the Firefighter included in the Making A Difference: The Fire Officer s Role (OnGuard, 1999), will provide additional guidance for raters. To improve the systems legal defensibility, review of the evaluation system must take place periodically, the goals and objectives of the system and performance criteria need to be documented in the policy manuals of the department. The documentation must include specific instructions for completion of the appraisal instrument and how to administer it. Previously mentioned improvements in defining the mission and vision of the organization, development of specific performance criteria, and employee and supervisor training in the purpose and administration of the system will also address deficient issues of defensibility. Improving employee voice and participation into the system begins with the participation of supervisors and employees in the review and development process of the revised performance evaluation system. This will assist in developing a system that is perceived as fair. The current system allows for employee comments in order to clarify or rebut inaccurate or disputed information. Expansion of this concept to a formal appeal process may provide additional procedural justice for employees in the process.

46 46 DISCUSSION Belton Emergency Services performance appraisal system is not unlike the systems in use by many companies. Organizations are forced to rely upon some type of subjective evaluation of a person s performance (DeNisi, 1996, p. 1) creating the errors, bias, inaccuracy, and inherent unfairness of most performance appraisals complained about by some employees. The perfect system does not exist (Cardy, 1998). Therefore, the goal of this research is to improve the effectiveness of Belton Emergency Services personnel evaluation system. It is clear that Belton Emergency Services personnel evaluation system, although weakly effective, can use substantial improvement. Belton Emergency Services appraisal system needs a clearly defined purpose (Matzer & Whisenand, 1988, p. 274; DeNisi, 1996). Bernadin et al. (1998, p. 28) further states that organizations must clearly and consistently communicate the firm s mission or constancy of purpose. Watson (1998) agrees arguing that effective performance development must be linked to the goals and objective of the organization. A principle called alignment. For decades it has been known to researchers that one the leading causes of organizational failure is "non-alignment of responsibility and accountability." Non-alignment occurs where employees are given responsibilities and duties, but are not held accountable for the way in which those responsibilities and duties are performed (Archer North & Associates, 2000). The survey results show that the absence of a clearly defined purpose communicating Belton Emergency Services mission and vision is lacking and staff personnel lack alignment or focus with the mission/vision of the department. Without a defined mission, department

47 47 personnel create their own mission and vision and can create confusion, conflict, and dissent within the department. Clearly defined performance standards must be specific, pertinent, attainable, measurable, and observable (SPAMO) (U.A., n.d.). The results indicate that these elements are also lacking from the performance appraisal process. Specific performance standards allow for more uniform and consistent evaluation of personnel. Although some level of subjectivity will likely remain, employees will know what is expected and will more likely perceive performance appraisal as fair when conducted. Constraints on the performance of personnel should be formally considered in the evaluation process. Performance appraisals must be conducted on an ongoing basis (Matzer & Whisenand, 1988, p. 274; Edwards, 2000, p. 153). Reh (1998) argues annual performance reviews are a waste of time because they are too infrequent. Most of us can actually remember what has happened over the last three months. Be sure to spend less than a quarter of the time doing them that you would have for an annual review. Then when you have to do an annual review you can just review the past three months and tack on the three previous quarterly reviews (Reh 1998). The author agrees that many of the problems experienced with Belton Emergency Services performance review are related to the frequency of evaluation. Employees need feedback at the time a problem occurs. A diary system, or memory file documenting employee accomplishments as well as problems, reviewed quarterly would reduce a number of problems with the existing system.

48 48 Training for supervisors, on the performance appraisal system, the mission and vision of the department, goal setting, and time management would improve the thoroughness and consistency of performance appraisals provided employees. Many of the improvements already identified will make the Belton Emergency Services performance appraisal more legally defensible. Involving employees in the review of the personnel evaluation system will provide greater confidence and acceptance of the system. The results indicated that most employees felt they had the ability to correct or rebut inaccurate information. The appraisal instrument should be replaced. A look at the appraisal instrument in use (Appendix A) reveals a document lacking specific performance criteria. This leads to a great deal of variability among raters and between employees rated. At a minimum, specific performance criteria should be developed for each category of the existing appraisal instrument. Other appraisal methods, (i.e. 360 evaluations, etc.) should be considered. RECOMMENDATIONS While BES personnel evaluation is effective at the most basic level, much improvement is necessary. Professional development of personnel has been virtually non-existent and would be a natural extension of an evaluation. Therefore, the author recommends the formation of a committee of Chief Officers, supervisors, and staff representatives to conduct the following review of the Belton Emergency Services personnel evaluation system. 1. Review Belton Emergency Services mission statement and incorporate into the evaluation.

49 49 2. Develop a vision statement and incorporate into the evaluation. 3. Review job descriptions for each level to assure they are descriptive of the skills needed to accomplish the mission. 4. Conduct a job analysis to identify critical tasks and behaviors needed to accomplish the mission. 5. Decide what form of performance evaluation tool is desired. 6. Develop specific performance criteria. 7. Train supervisors how to administer the performance evaluation. 8. Quarterly evaluations are more effective. 9. Implement a performance diary or memory file system. 10. Develop policies outlining the purpose and function of the system.

50 50 REFERENCE LIST Archer North & Associates. (2000) Performance Appraisal: The Complete Online Guide. New York, NY: Author. Retrieved May 15, 2001 from the World Wide Web: (Original work published in 1998) Bernardin, H. J., Hagan, C. M., Kane, J. S., & Villanova P., (1998). Effective Performance Management: A Focus on Precision, Customers, and Situational Constraints. In J. W. Smither, (Ed.), Performance Appraisal: State of the Art in Practice. San Francisco, Jossey-Bass, Inc. Campbell, D. P. (1995). Campbell Organizational Survey (COS ). (Available from National Computer Systems, Inc., P.O. Box 1416, Minneapolis, MN 55440) Cardy, R. L. (1998). Performance Appraisal in a Quality Context: A New Look at an Old Problem. In J. W. Smither, (Ed.), Performance Appraisal: State of the Art in Practice. San Francisco, Jossey-Bass, Inc. Carter, H.R., & Rausch (1999). Management in the Fire Service (3 rd ed.). Quincy, MA. National Fire Protection Association, Inc. City of Belton. (2001). Code of Ordinances, City of Belton, Missouri. (Supplement 90). Talahasee, FL: Municipal Code Corporation. Retrieved July 12, 2001 from the World Wide Web: City of Belton. (2001). Personnel Code Book. (Original work published 1974, Available from City of Belton, 506 Main St., Belton, MO 64012). Czerwinski, R. E. (1998). Performance Appraisals What Can Be Done To Improve Them? ARP, National Fire Academy, Emmitsburg, MD.

51 51 DeNisi, Angelo S. (1996). A cognitive approach to performance appraisal: A program of research. London, UK: Routledge Edwards, S. T. (2000). Fire Service Personnel Management. Upper Saddle River, NJ: Prentice-Hall, Inc. Fink, L. S., & Longenecker, C. O. (1999, September/October). Creating Effective Performance Appraisals. In Industrial Management (5th ed., Vol. 41). Norcross, GA: Institute of Industrial Engineers. Retrieved June 9, 2001 from EBSCO host database (MasterFILE Premier, ) on the World Wide Web: HRTools.com. (1999). Legally Defensible Performance Appraisals. In Overview: Legally Defensible Performance Appraisals. Petaluma, CA: KnowledgePoint. Retrieved July 8, 2001 Job-Analysis.Network. (1999). HR Guide to the Internet: Job Analysis: Overview. In Job- Analysis.Network. Retrieved July 21, 2001 from the World Wide Web: Matzer, John Jr. and Whisenand, Paul M. (1988). Managing fire services. (2nd ed.) (p. 274). Washington, DC: International City Management Association. Nichols, F. (1997, May). Don t Redesign Your Company s Performance Appraisal System: Scrap it. In Corporate University Review. Retrieved April 10, 2001 from the World Wide Web: OnGuard, Inc. (1999). Making a Difference: The Fire Officer s Role. Loveland, CO: Author. (Original work published 1988) Oregon State University. (1999, June 23). Instructions for Completion of the Performance Evaluation Form. In HRIS Project. Retrieved August 2, 2001 from the World Wide Web:

52 52 Reh, F. J. (1998, March 24). Why Annual Performance Reviews Are A Waste Of Time. Retrieved July 1, 2001 from the World Wide Web: s&cob=home Roberts, G. E. (1994). Maximizing Performance Appraisal System Acceptance: Perspectives from Municipal Government Personnel Administrators. In Public Personnel Management (4th ed., Vol. 23, pp ). Alexandria, VA: International Personnel Management Association-USA. Retrieved June 9, 2001 from EBSCO host database (MasterFILE Elite) the World Wide Web: University of Arizona, Human Resources Department. (n.d.). Part 3: Managing Employee Performance & Performance Appraisal. In Human Resources Information Series. Retrieved June 15, 2001 from the World Wide Web: Watson, S. C. (1998, May). Five Easy Pieces to Performance Measurement. Training & Development, 52(5), Retrieved April 14, 2001 from the World Wide Web:

53 53 APPENDIX A BELTON MISSOURI EMERGENCY SERVICES DEPARTMENT PERSONNEL PERFORMANCE EVALUATION FORM Name Position (title and Grade) Social Security Number Date of Evaluation The numerical rating given to each performance category will be based upon the following standard predicated on the responsibility of all department employees to be knowledgeable of the departments Standard Operating Procedures and EMS Protocols and thereby have adequate understanding of what would be expected of them: 5. Consistently performs above expectation in most areas considered. 4. Performance in all areas considered meets or exceeds expectations. 3. Minimum acceptable performance. Some areas may need improvement. 2. Below minimum acceptable performance. Substantial improvement needed. 1. Unacceptable performance. Where no comments appear, refer to the wording corresponding to the numerical rating shown.

54 A. JOB KNOWLEDGE: 54 FIRE SERVICE: RATING: EMERGENCY MEDICAL SERVICES: RATING: B. INITIATIVE: RATING: C. ATTENDANCE: RATING: D. PUNCTUALITY: RATING: E. JOB ACCOMPLISHMENT: FIRE SERVICE: RATING: EMERGENCY MEDICAL SERVICES: RATING: F. ATTITUDE: RATING:

55 G. PLANNING AND ORGANIZING: RATING: 55 H. CONTACTS WITH PUBLIC: RATING: I. ABILITY TO COMMUNICATE EFFECTIVELY: ORAL: RATING: IN WRITING: RATING: J. WILLINGNESS TO FOLLOW INSTRUCTIONS AND CARRY OUT DECISIONS: RATING: K. WORKING RELATIONSHIPS WITH OTHER EMPLOYEES: RATING: L. ADAPTABILITY/FLEXIBILITY: RATING:

56 M. KNOWLEDGE OF OCCUPATIONAL HAZARDS AND SAFETY 56 PRECAUTIONS: FIRE SERVICE: RATING: EMERGENCY MEDICAL SERVICES: RATING: N. KNOWLEDGE OF PERTINENT CITY ORDINANCES, LAWS, RULES, REGULATIONS AND PROCEDURES: RATING:

57 57 O. OVERALL PERFORMANCE EVALUATION: RATING: Evaluator's Signature Date Personnel Officer's Signature Date City Administrator's Signature Date EMPLOYEE COMMENTS: Employee Signature Date

58 58 APPENDIX B BELTON EMERGENCY SERVICES Steven F. Holle Fire Chief TO: ALL PERSONNEL A requirement of my participation in the Executive Fire Officer Program of the National Fire Academy is to conduct an applied research project for each class. My project for the Executive Development class is to conduct an evaluation of the effectiveness of our personnel evaluation system. The purpose of the attached survey(s) is to evaluate various elements that affect the effectiveness of the system. The Performance Evaluation Survey is for all personnel. Those individuals involved in evaluating personnel should also complete an additional instrument called the Supervisors Survey of Performance Evaluation. Please return the completed survey(s) by Tuesday, July 17 in order for your survey to be included in the results. There are three components to the survey: 1. Circle the number on the rating scale that most closely matches your feelings about the element being rated. 2. Provide additional comments you wish to share about your experience with our personnel evaluation system, and 3. Suggest objectives for improvement of the system; how it will better meet the needs of the department and personnel. To protect anonymity for components two and three, do not make any identifying marks on the instrument. You may type your comments on a separate sheet of paper if you wish. Please be candid and give serious consideration to this part of the survey. Your comments and suggestions are greatly valued. Please return your survey upon completion to your supervisor. Compiled results will be available for review upon completion of the project. Thank you, Steven F. Holle Fire Chief 223 Main Street Belton, MO Phone (816) Fax (816)

59 59 Belton Emergency Services Performance Evaluation Survey Please return the completed survey by July 17, Please circle the number in the right hand column that corresponds with your feelings about the element described in the center column according to the rating scale below. In the space provided or on an additional sheet of paper, please comment on the element or offer a suggestion for how it may be improved. Strongly Disagree Disagree Somewhat Disagree Somewhat Agree Agree Strongly Agree I have been the subject of one or more performance evaluations at Belton Emergency Services I have discussed the department s vision, mission, and goals and objectives during my performance evaluation along with my role in achieving the vision, mission, goals or objectives My duties and the priority of my duties are discussed Clear expectations and standards for completion of my duties are discussed I receive feedback on my performance (strengths and areas to improve) during my evaluation Factors that affect my performance are discussed (personal, medical, departmental issues beyond my control, etc.)

60 60 7. Changes expected by the department that affect resources, standards, methods, etc. are discussed Suggestions for improvement are discussed (changes in procedures, methods, etc. that will assist the department achieve it s vision, mission, goals and objectives) Personal goals and objectives are discussed. I receive counseling to achieve those goals My professional development is discussed with suggestions for professional and personal growth The frequency of performance evaluations is adequate Performance evaluations are accurate Performance evaluations are fair Criteria to achieve a given rating are specific, pertinent, attainable, measurable, and observable I have changed my work habits as a result of a performance evaluation Performance evaluations have helped the department achieve it s vision, mission, goals, or objectives

61 My strengths and achievements are recognized Performance evaluations identify functional areas of my performance needing improvement and establish agreement about how improvement is to be accomplished Our organization has a good performance appraisal system in place Feedback on performance for people at my level is timely, accurate, and constructive The effect of my performance upon our customers is discussed I am willing to assist in improving the performance appraisal system I have the opportunity to correct or rebut inaccurate information Personnel evaluations are biased My supervisor and I establish goals to improve my performance and I have the opportunity to clarify and revise how I may meet my goals Please type your comments or suggestions for improvement on a separate sheet of paper if necessary.

62 62 APPENDIX B-1 Results Belton Emergency Services Performance Evaluation Survey 1. I have been the subject of one or more performance evaluations at Belton Emergency Services. Of 24 surveys distributed, 20 (83.33%) were returned. An employee who had not participated in a performance evaluation during his brief tenure submitted one survey. The total number of participant is this survey is I have discussed the department s vision, mission, and goals and objectives during my performance evaluation along with my role in achieving the vision, mission, goals or objectives I was allowed to express how I could do better to achieve department goals. However, it was not explained to me what goals the shift commanders had for me over the next year. 2. The overall vision, mission, goals, and objectives of the department can be inferred by the individual components of the evaluation, but are not specifically discussed as such. 3. My duties and the priority of my duties are discussed Duties Yes, Priorities, not as often. 2. I believe my duties are defined throughout the year, and discussed as such. They are not clearly defined during the evaluation process in any specific detail.

63 63 4. Clear expectations and standards for completion of my duties are discussed Could be clearer. 2. See #3. Even less attention is given to what is expected and what the standards are. 3. There are not clear standards and expectations for everyone. Standards for one shift/person may not be the same for another. Example: What constitutes/qualifies a trained pump operator? What constitutes/qualifies being about to work or train a new employee? Is it standard for everyone to sweep/clean the bay floor everyday? 5. I receive feedback on my performance (strengths and areas to improve) during my evaluation This is one of the strongest points of my officers. They always talk with you one on one if something needs improvement, and assist you in how to improve on it. But they also give praise if they see it's deserved. 2. I usually receive feedback is when a call went poorly or when I did something wrong. During my evaluation, we will discuss my areas of improvement. My areas of strength are not discussed as much. 6. Factors that affect my performance are discussed (personal, medical, departmental issues beyond my control, etc.)

64 64 1. Some are, some arent. Sometimes minor issues are beat to death while more important ones are left alone. 2. See #3. 3. I do not feel comfortable discussing any of those issues with anyone on this fire department due to lack of confidentiality with my peers. 7. Changes expected by the department that affect resources, standards, methods, etc. are discussed These are not discussed at all. 8. Suggestions for improvement are discussed (changes in procedures, methods, etc. that will assist the department achieve it s vision, mission, goals and objectives). Weighted Score Discussed yes, followed up on????? 2. These may or may not come up in discussion. See #7. 3. Suggestions are made frequently, however there is often little outcome, decisions made, implementation or progress shown.

65 65 9. Personal goals and objectives are discussed. I receive counseling to achieve those goals I feel that our officers have always encouraged us to set our own goals. They have always been willing to assist us in reaching these goals. 2. With the exception of areas needing improvement, no true goals are established or discussed. 3. I have expressed personal goals and they are often ignored. I know where I want to be in 10 years and BFD does not offer those steps for me to reach my long-range goals. Do officers know how to counsel someone in this area? 10. My professional development is discussed with suggestions for professional and personal growth They are sometimes discussed but rarely followed up on. 2. No discussion of professional development is involved. 11. The frequency of performance evaluations is adequate

66 66 1. It seems that the only time issues are addressed are annually. Some should be more often. 2. I feel that the first two years one should have an evaluation every six months. Then one/year for three years (+). This allows the "new guys" the opportunity to better understand the goals of the department. 3. A yearly evaluation of the purpose of raises is adequate; however, a semi-annual evaluation would benefit for the purpose of knowing where an employee needs to improve prior to the yearly evaluation. 4. One a year is adequate. 12. Performance evaluations are accurate Format doesn't allow. 2. The scale of 1-5 does not give an adequate evaluation of each area. Also there seems to be no consistency between supervisors of what is "acceptable" or "exceeds". There needs to be a written procedure for supervisors to follow. 13. Performance evaluations are fair Generally. 2. They are fair for the most part. I believe Captains should evaluate firefighters and Assistant Chiefs should evaluate Captains. On several occasions, the Assistant Chief

67 67 performed the evaluation of a firefighter, and should only happen in the long term absence of the Captain, or when a Captain has not worked with the firefighter long enough to perform an accurate evaluation. Also, on some shifts both the Captain and the Assistant Chief sit in on the firefighter's evaluation. This is unnecessary and intimidating. 3. Evaluations come from my immediate supervisors so I do believe they are fair and accurate. 14. Criteria to achieve a given rating are specific, pertinent, attainable, measurable, and observable Format doesn't allow. 2. Develop a Career Guide which sets criteria for each position within the department. 3. See # I have changed my work habits as a result of a performance evaluation Personally, yes. 2. Discussions during evaluations have been beneficial to my improvement in specific areas. 3. I have improved in areas of weakness that were mentioned in my evaluations.

68 Performance evaluations have helped the department achieve its vision, mission, goals, or objectives But there needs to be more than annual evaluations. 2. I don't know what BFD's visions or goals are. Why are they not posted so we are reminded of them every day? 17. My strengths and achievements are recognized If they're known. 2. This area could use some improvement. Though most strengths are discussed, consideration for achievements and additional training should play a part in the evaluation process. 18. Performance evaluations identify functional areas of my performance needing improvement and establish agreement about how improvement is to be accomplished

69 69 1. Pay increases should acknowledge those who achieve more than others. 2. How improvement is to be accomplished is up to the supervisor, and is not always discussed. 19. Our organization has a good performance appraisal system in place Not bad, but we shouldn't be afraid to tell people where they need improvement. 2. It needs a lot of work. See # We usually do not receive feedback unless it is telling us something we did wrong. Example: memo on time cards, problems with reports. 20. Feedback on performance for people at my level is timely, accurate, and constructive Timely-sometimes, accurate - sometimes, constructive - rarely. 2. See # Positive feedback for even the smallest things goes a long way. Be creative with it. 21. The effect of my performance upon our customers is discussed

70 70 1. This is not discussed at all. 22. I am willing to assist in improving the performance appraisal system This evaluation is a good start. 2. I'm always willing to assist in ways to improve the department however many members are not wanting my help. 23. I have the opportunity to correct or rebut inaccurate information The space is provided. 2. Not at all. See #11. If a supervisor is not giving constant feedback in a day-to-day direction, most of the evaluation can be a surprise. 24. Personnel evaluations are biased It's always going to happen.

71 71 2. For the most part they are unbiased. See #11&13. With only a yearly evaluation, the mood of the supervisor towards the subordinate during the week of evaluation can play a major factor in bias. 3. Personal evaluations will always be biased and subjective. If the two parties cooperate, "get-a-long, the evaluations will generally reflect a positive mark. However, if the two parties find it difficult to work together or communicate effectively, the evaluation will generally receive low marks, even if the performance is acceptable. 25. My supervisor and I establish goals to improve my performance and I have the opportunity to clarify and revise how I may meet my goals Not yet. Goals here are a personal thing. There is no direction set as of yet. 2. Not at all. See #11& I would like to do this, but I don't feel my officers know how to. Do not expect supervisors to know how to do this without some sort of training. General Comments: Historically the evaluations at this department have been used to threaten and/or punish people. Many of the supervisors neglect to say anything about your performance until evaluation time and then hit you with all the stuff you're doing wrong. There has historically been little praise or actual constructive evaluation done. Usually nothing at all gets said about the departments mission, goals, etc., and it's doubtful that the majority of the personnel even know what those things are. Evaluations need to be more frequent and more needs to be done to help the employee know where he stands and what he's doing right as well as what he needs improvement on. The numbering system needs to be replaced. The criteria are hard to understand and are too similar for each number category. A simplified system of: 1-below expectations, 2-meets expectations, 3-exceeds expectations; might be more effective. A narrative using the current outline form as a guideline for categories would also be an improvement. Chief Holle, I am pleased to see that you are assessing the current employee evaluation process. I feel that if any adjustments are needed, they are minor. The current process helps personnel realize the expectations and goals of our department, as well as achieve those goals. However, I do feel that the evaluation process is a small part of this department achieving its goals. If personnel are to achieve these goals, they need to know what your goals are for this department.

72 72 I feel that it would be a great benefit to this department if we had a quarterly meeting of all personnel. This would allow us to know what you feel needs to be done to make this department more productive. It would also give us the opportunity to give you our opinions as to what needs to be done to make Belton Fire more prosperous. In my opinion, the attendance would be purely voluntary. Also, it should be known that this should not be an opportunity for a "gripe session." If given ample notice, employees should have enough time to write their concerns down and come up with proposed solutions to be discussed at these meetings. I feel that this department is to grow positively for the public and its members, communication needs to be the highest priority. These meetings would provide a gel between the departments administration and work force. Together, with open communications, we can provide a more efficient service to our customers as well as making this a stronger department. Thank you for taking the time to consider this proposal. My biggest problem with our evaluation process is with new employees not receiving an evaluation for 1 year. There is no formal evaluation to help them improve their weaknesses. They could/should be done on a rotation at days, as an example. This system should also be used for promotions as well. I would like to see a system with a self-evaluation segment. This can help open the employee's eyes when he has to sit & think about his performance. For an effective personal or department evaluation system to take place there needs to be an improvement of communications between all levels. Personal evaluation involves effective and professional communication, which the department is lacking. I feel that currently, we are in a situation where past issues and feelings are keeping us from quality dialog with each other and achieving greatness. Suggestions * Have a quarterly meeting with everyone from the department. This meeting would be beneficial to establish/modify department goals; talk about department wide issues and other items. I also believe that this will bring team unity between the shifts. * Have a 360 evaluation where firefighters evaluate officers. This would not only help officers improve in areas of weakness but help future officers know how to evaluate effectively. * Let s train our supervisors on how to evaluate effectively and accurately. Why not have their input on designing a new personal evaluation form. * How about a patient satisfaction survey or a patient follow-up questionnaire? Let s ask our customers how we can improve as a department. This would also help in finding out information for question 21.

73 73 APPENDIX C BELTON EMERGENCY SERVICES Steven F. Holle Fire Chief TO: ALL PERSONNEL A requirement of my participation in the Executive Fire Officer Program of the National Fire Academy is to conduct an applied research project for each class. My project for the Executive Development class is to conduct an evaluation of the effectiveness of our personnel evaluation system. The purpose of the attached survey(s) is to evaluate various elements that affect the effectiveness of the system. The Performance Evaluation Survey is for all personnel. Those individuals involved in evaluating personnel should also complete an additional instrument called the Supervisors Survey of Performance Evaluation. Please return the completed survey(s) by Tuesday, July 17 in order for your survey to be included in the results. There are three components to the survey: 1. Circle the number on the rating scale that most closely matches your feelings about the element being rated. 2. Provide additional comments you wish to share about your experience with our personnel evaluation system, and 3. Suggest objectives for improvement of the system; how it will better meet the needs of the department and personnel. To protect anonymity for components two and three, do not make any identifying marks on the instrument. You may type your comments on a separate sheet of paper if you wish. Please be candid and give serious consideration to this part of the survey. Your comments and suggestions are greatly valued. Please return your survey upon completion to your supervisor. Compiled results will be available for review upon completion of the project. Thank you, Steven F. Holle Fire Chief 223 Main Street Belton, MO Phone (816) Fax (816)

74 74 Belton Emergency Services Supervisors Survey of Performance Evaluation Please return the completed survey by July 17, Please circle the number in the right hand column that corresponds with your feelings about the element described in the center column according to the rating scale below. In the space provided or on an additional sheet of paper, please comment on the element or offer a suggestion for how it may be improved. Strongly Disagree Disagree Somewhat Disagree Somewhat Agree Agree Strongly Agree I have conducted one or more performance evaluations of subordinate personnel The department s vision, mission, and goals and objectives are discussed during performance evaluation along with the employee s role in achieving the vision, mission, goals or objectives The employee s duties and the priority of the employee s duties are discussed Clear expectations and standards for completion of the employee s duties are discussed Employees receive feedback on their performance (strengths and areas to improve) during the evaluation Factors that affect employee performance are discussed (personal, medical, departmental issues beyond their control, etc.)

75 75 7. Changes expected by the department that affect resources, standards, methods, etc. are discussed Suggestions for improvement are discussed (changes in procedures, methods, etc. that will assist the department achieve it s vision, mission, goals and objectives) Personal goals and objectives are discussed. You provide counseling to assist the employee to achieve those goals Employee professional development is discussed with suggestions for professional and personal growth The frequency of performance evaluations is adequate Performance evaluations are accurate Performance evaluations are fair Criteria to achieve a given rating are specific, pertinent, attainable, measurable, and observable Performance evaluations have changed work habits of employees under my supervision Performance evaluations have helped the department achieve it s vision, mission, goals, or objectives

76 Employee strengths and achievements are recognized Performance evaluations identify functional areas of employee performance needing improvement and establish agreement about how improvement is to be accomplished Our organization has a good performance appraisal system in place Feedback on performance for people at the employee s level is timely, accurate, and constructive The effect of employee performance upon our customers is discussed I am willing to assist in improving the performance appraisal system I have the opportunity to correct or rebut inaccurate information Personnel evaluations are biased My supervisor and I establish goals to improve my performance and I have the opportunity to clarify and revise how I may meet my goals

77 I have been adequately trained to conduct performance appraisals Please continue your comments or suggestions for improvement on a separate sheet of paper if necessary.

78 78 APPENDIX C-1 Results Belton Emergency Services Performance Evaluation Survey 1. I have conducted one or more performance evaluations of subordinate personnel. Of seven surveys distributed, six (85.17%) were returned. An employee who had not participated in a performance evaluation during his brief tenure submitted one survey. The total number of participants included in this survey is five. 2. The department s vision, mission, and goals and objectives are discussed during performance evaluation along with the employee s role in achieving the vision, mission, goals or objectives The employee s duties and the priority of the employee s duties are discussed Clear expectations and standards for completion of the employee s duties are discussed

79 79 5. Employees receive feedback on their performance (strengths and areas to improve) during the evaluation Factors that affect employee performance are discussed (personal, medical, departmental issues beyond their control, etc.). Weighted Score Changes expected by the department that affect resources, standards, methods, etc. are discussed Suggestions for improvement are discussed (changes in procedures, methods, etc. that will assist the department achieve it s vision, mission, goals and objectives). Weighted Score

80 80 9. Personal goals and objectives are discussed. You provide counseling to assist the employee to achieve those goals Employee professional development is discussed with suggestions for professional and personal growth? The frequency of performance evaluations is adequate Performance evaluations are accurate

81 Performance evaluations are fair Criteria to achieve a given rating are specific, pertinent, attainable, measurable, and observable Performance evaluations have changed work habits of employees under my supervision

82 Performance evaluations have helped the department achieve its vision, mission, goals, or objectives Employee strengths and achievements are recognized Performance evaluations identify functional areas of employee performance needing improvement and establish agreement about how improvement is to be accomplished Our organization has a good performance appraisal system in place

83 Feedback on performance for people at the employee s level is timely, accurate, and constructive The effect of employee performance upon our customers is discussed I am willing to assist in improving the performance appraisal system

84 I have the opportunity to correct or rebut inaccurate information Personnel evaluations are biased My supervisor and I establish goals to improve my performance and I have the opportunity to clarify and revise how I may meet my goals I have been adequately trained to conduct performance appraisals

85 85 APPENDIX D Belton Emergency Services Performance Evaluation Survey Summary Personnel Weighted Score Summary Supervisor Weighted Score Summary 1 I have been the subject of one or more performance evaluations at Belton Emergency Services. 19* 5* 2 I have discussed the department s vision, mission, and goals and objectives during my performance evaluation along with my role in achieving the vision, mission, goals or objectives. 3 My duties and the priority of my duties are discussed Clear expectations and standards for completion of my duties are discussed I receive feedback on my performance (strengths and areas to improve) during my evaluation Factors that affect my performance are discussed (personal, medical, departmental issues beyond my control, etc.). Changes expected by the department that affect resources, standards, methods, etc. are discussed. Suggestions for improvement are discussed (changes in procedures, methods, etc. that will assist the department achieve it s vision, mission, goals and objectives) Personal goals and objectives are discussed. I receive counseling to achieve those goals My professional development is discussed with suggestions for professional and personal growth The frequency of performance evaluations is adequate Performance evaluations are accurate Performance evaluations are fair Criteria to achieve a given rating are specific, pertinent, attainable, measurable, and observable I have changed my work habits as a result of a performance evaluation. 28 5

86 86 16 Performance evaluations have helped the department achieve it s vision, mission, goals, or objectives My strengths and achievements are recognized Performance evaluations identify functional areas of my 18 performance needing improvement and establish agreement about how improvement is to be accomplished Our organization has a good performance appraisal system in place Feedback on performance for people at my level is timely, accurate, and constructive The effect of my performance upon our customers is discussed I am willing to assist in improving the performance appraisal system I have the opportunity to correct or rebut inaccurate information Personnel evaluations are biased My supervisor and I establish goals to improve my performance and I have the opportunity to clarify and -5-6 revise how I may meet my goals. 26 I have been adequately trained to conduct performance appraisals. -3