POLITICAL SCIENCE DEPARTMENT MERIT PROCESS Approved April 2011

Transcription

1 POLITICAL SCIENCE DEPARTMENT MERIT PROCESS Approved April 2011 All faculty members, with the exception of the department chair, are each placed into one of four categories: Unsatisfactory Satisfactory High Merit Exceptional Merit Categories need not have an equal number of professors. In fact, depending on the range of performance that year, one or more categories may be empty. The overall category placements are derived from a weighted average of the rankings compiled from each of the three criteria: scholarship, teaching, and service. Each step is explained below. SCHOLARSHIP Step 1: Quantification Each professor s scholarly output for the previous three calendar years, as documented on the annual reports, is assigned a single numeric value ( scholarly score ) based on a formula derived by taking the average of the input given by all 12 faculty members in a 2009 survey. Note: scholarly output is defined strictly as an item, already in print, with a copyright date during the three-year period of analysis. No credit is given for works accepted but not yet in print, or in progress. The scholarly-score formula is: i (q * v) Where, i = the different items (types of scholarly output) q = the quantity of a given item produced over the last three calendar years v = the value assigned to a given item by the 2009 department survey The scores needed to be adjusted to avoid conflict with the department s Tenure and Promotion document. Specifically, categories encompassing peer-reviewed

2 original research were rescaled such that the lowest score in that category was made equal to the highest score in any other category but that, otherwise, the scores within each subset (peer-reviewed original research, or not) maintained their proportion to each other. To scale it properly, the multiplier ended up being 2.29 for peer-reviewed original research. The adjusted v scores for 2011 are: Research book: (100*2.29) = 229 Journal research article: (35*2.29) = 80 Contribution to edited volume, research: (31*2.29) = 71 Edited volume: 67^ Textbook, first edition: 71 Textbook revisions: 36 Journal pedagogical article: 20 Review article in a journal: 23 Book review: 5 Contribution to edited volume, pedagogical: 20 Contribution to edited volume, review essay: 21 Encyclopedia article: 8 Magazine/newspaper article: 3 ^If the editor of an edited volume also contributed chapters to the volume, up to one chapter may be counted in addition to the points given for editing. So, for example, the maximum points an editor of a research volume could receive is 138 (67+71). The faculty members are rank ordered by their scholarly scores, and placed preliminarily into the following categories: unsatisfactory, satisfactory, high merit, exceptional merit. Some categories may be empty. The thresholds for each category are decided by committee consensus. It is recommended that the committee look for natural clustering among the scores, so as to maximize the between-group differences and minimize the within-group differences. For example in 2009, the threshold for being at least satisfactory was the equivalent of publishing two scholarly articles during the three-year period: 70 points. Step 2: Refinement After the initial category placement is complete, the committee takes a careful look at the annual reports, with an eye toward factors missed by the quantification that, taken together, might cause a faculty member to move into a higher category. The threshold for moving up is decided by committee consensus. Such factors may include, but are not limited to: A large proportion of solo-authored works

3 Articles in top journals Books at top academic presses Works that appear to have had a major impact on the field Research awards or recognition Projects in print that appear to have required extraordinary effort Successful grant activity After reviewing the annual reports and making upward movements by consensus, the committee finalizes the category placement for scholarship. TEACHING Step 1: Narrative Each faculty member will submit a teaching narrative (up to one page) to provide the framework for a holistic evaluation of teaching. Factors that may be included (but are not limited to): New teaching strategies/ technologies employed Efforts to improve teaching (workshops attended, materials reviewed, peer input sought, etc.) Innovations/challenges in classes taught Brief explanation of assessment tools/class materials New course preparations Information for interpreting SPOT scores Highlights of teaching awards/honors (especially student-initiated) Based upon the narrative, which largely reflects effort, each person will be placed in one of four categories: a. Exceptional merit b. High merit c. Satisfactory d. Unsatisfactory Some categories may be empty. During years in which the merit pool is exceptionally high, the committee is encouraged to solicit more detailed teaching portfolios from each faculty member. Step 2: Other Factors The committee will take a careful look at the annual reports, with an eye toward factors that may warrant special consideration, and therefore positively impact merit. Such factors may include, but are not limited to:

4 Lack of honors or limited-enrollment classes Service classes that tend to receive lower SPOT scores Factors in this category generally represent collective goods but can have individual costs. Consideration of other factors can move faculty into higher categories of merit. Step 3: SPOT Scores Student perception of teaching is often an indication of quality of teaching, though not always. Consequently, performance on SPOTs is relevant, though should not be the primary tool of evaluation. After careful inspection of the first two documents, the committee will examine SPOT scores to determine the need to further refine category placement. Because SPOT scores are highly sensitive to sampling error and other trivial or idiosyncratic factors, it is recommended that clusters be analyzed. If robust distances exist (based on standard errors), individual placement into categories can be altered. Also, the department average, found on the SPOT reports, may be used as a mechanism to evaluate relative performance and be used to further refine merit. Step 4: Placement into Categories After reviewing the narrative and annual reports and making movements by consensus based on SPOT performance, the committee finalizes the category placement for teaching. Each person will be placed into one of the following categories: Exceptional Merit High Merit Satisfactory Unsatisfactory The placement into categories reflects an effort to reward effort and success in teaching, allowing for individualistic and holistic evaluations of this critical, though difficult to measure, element of our profession. SERVICE The placement of faculty members into service categories is a one-step process. For each professor, the committee takes a careful look at the narrative on service (which is to be no more than one page in length) and the annual report relevant to department, college, university, and community service. Because service tasks vary

5 greatly, the committee should take particular notice of the narrative detailing the time commitment involved in the professor s service responsibilities. If deemed necessary, the committee may also seek verification from third parties as to the quality of a faculty member s performance in a given service task. It then decides, by consensus, which of the following categories is most appropriate: unsatisfactory, satisfactory, high merit, or exceptional merit. Because the political science department performs an extraordinary amount of service relative to other departments, it is recommended that the placements skew toward the high end. For example, it may be appropriate for as much as half of the department to be deemed exceptionally meritorious, with a majority of the rest deemed highly meritorious. Also, the committee should reserve unsatisfactory for the rare case in which a faculty member is demonstrably shirking his/her responsibilities. OVERALL CATEGORY PLACEMENT After the committee assigns categories to every professor for scholarship, teaching, and service, the categories are converted to numbers: 1 Unsatisfactory 2 Satisfactory 3 High Merit 4 Exceptional Merit An overall score for each professor is derived by calculating a weighted average, per the workload standard: (R *.4) + (T *.4) + (S *.2) Where, R = Scholarship T = Teaching S = Service The faculty members are rank ordered by overall score, and then assigned to one of four categories: unsatisfactory, satisfactory, high merit, exceptional merit. These are the final categories that determine the merit portion of faculty pay raises. It is recommended that each of the highest three categories contain at least one faculty member, unless a strong case can be made to the contrary. The numerical cut-points for each category are decided by committee consensus. It is recommended that the committee look for natural clustering among the scores.

6 Because the scores may cluster closer together, it is recommended that the committee agree to a priori cut-points, to be used if the clustering analysis is inconclusive. One way to do this is to treat the scores as a simulated GPA. In 2009, for example, higher than 3.0 was exceptional merit. Also, the committee should be aware that it might miss some nuance by averaging an ordinal indicator. For example, it might agree that the distance between unsatisfactory and satisfactory is larger than between satisfactory and high merit, and would thus decide to differentiate otherwise similar scores when one of them contains an unsatisfactory rating for one of the three criteria. Likewise, it may decide that a necessary condition for achieving overall exceptional merit is to have earned exceptional merit in at least one area. Regardless, it is recommended that these decisions be made before the overall scores are calculated. PAY RAISES First, the department chair reports the allotted merit pay amounts to the committee, in the form of 1) the percentage of the total salary pool set aside for merit raises, and 2) its translation into dollar amounts. Next, the department chair divides the pool into two portions: COLA and merit. The COLA portion must be allocated uniformly, either as a proportion of each faculty member s current salary, as a dollar amount, or as a combination of the two. The remaining merit portion is allocated based on overall merit category placement. Every professor within a given category must earn a higher-dollar raise from this portion than every professor in a lower category. The COLA/merit proportion is decided by the department each spring for the next raise cycle. The department views three options as acceptable: 50/50, 60/40, and 67/33. Calculation for merit pay for academic year: The allocation between COLA and merit will be 60% COLA, 40% merit. Merit will be determined based upon the following: Merit: Category 1 = Unsatisfactory Category 2 = Satisfactory Category 3 = High Merit Category 4 = Exceptional Merit Total pool = 40% of merit allocated to department X = Standard merit raise F1 = frequency unsatisfactory category

7 F2 = frequency satisfactory category F3 = frequency high merit category F4 = frequency exceptional merit 1.5X(F4) + X(F3) +.5X(F2) + 0X(F1) = Total pool The department chair will determine the merit recommendations based upon the approved formula, with the endorsement for the committee chair. APPEALS PROCESS If a faculty member wishes to appeal the committee s decision, the following procedure shall be followed: First, the faculty member will request a meeting with the personnel committee chair. The meeting may be requested no later than two weeks after the faculty member is notified of his/her category placement. The chair will be obligated to explain, as thoroughly as possible without violating the transparency principles articulated below, the quantitative and qualitative factors that went into the professor s category placement. If the faculty member is not satisfied with the explanation, the next step is to file a written appeal. The appeal will be considered by the personnel committee, which can either uphold or modify its original decision. If the faculty member is not satisfied with the committee s decision, the next step will be to take it up as business with the department. APPENDIX: PRINCIPLES Transparency of process vs. transparency of outcome. It is vitally important that every step in the process be as transparent as possible. Non-committee members are entitled to know, to the greatest extent practicable, exactly what criteria are employed in the evaluation, and how they are being employed. On the other hand, the committee believes that the specific outcomes should be kept confidential. The exception is that all faculty members are entitled to know in which overall category they were placed, as well as the dollar amount of their own merit raise.

8 On a related note, an individual committee member should never speak on his/her own behalf during a committee meeting. Faculty considerations The committee recognizes that professors are already burdened with considerable bureaucratic paperwork. Nonetheless, the evaluation of annual reports is a timeconsuming endeavor, made much worse by ambiguities and inconsistencies across faculty. It is therefore recommended that colleagues take care to craft their annual reports with an eye toward the criteria spelled out in this document. As examples: List the word count of encyclopedia articles Be thorough about service activities Include a one-page (maximum) appendix detailing the time commitments involved in service activities State unambiguously whether a scholarly work is in print or forthcoming, and give the accurate copyright date List scholarly work from the past three years on your most recent report. List any circumstances, perhaps in the teaching narrative, that might account for low SPOT scores Process continuity Because both faculty fortunes and merit-pool size vary by year, it is recommended that the merit process place a high value on continuity year-by-year. Marginal, necessary changes may occur by committee consensus, while larger changes must incorporate full department input, and must be justified by a clear case that the present system is unworkable or unfair. Given the complexity of the process and the importance of continuity, the committee membership should never contain fewer than two professors who were on the committee during the prior year. It is recommended that committee members serve staggered three-year terms, with one term expiring each year.