SYLLABUS MGMT551 Program Evaluation

SYLLABUS MGMT551 Program Evaluation Instructor: Email: Message Phone: Required Textbook Russ-Eft, D. & Preskill, H. (2001). Evaluation in organizations; A systematic approach to enhancing learning, performance, and change. Cambridge, MA: Perseus Publishing. Course Description The purposes, procedures, and techniques of effective evaluation processes are considered, with an emphasis on informing those involved in organizational change efforts. Students have the opportunity to prepare a detailed evaluation plan. Course Objectives 1. Recognize what evaluation means and the various formats it takes. 2. Be familiar with roles, general models, approaches and taxonomies of evaluation. 3. Be able to identify evaluation design elements. 4. Be able to determine appropriate evaluation application. 5. Communicate and report results of evaluation. 6. Engage in the evaluation of the evaluation. 7. Locate various sources of evaluation instruments. 8. Be able to complete an evaluation. Participation Policy Participation in the on-line class is mandatory. Students are responsible for contacting the course instructor if there is any difficulty in understanding the course materials or completing the course assignments. The purpose of participation in an on-line class is to generate thoughtful discussion and to share ideas and experiences with your cohorts. Each participant is expected to participate by posting responses to discussion questions and/or case studies and to offer evaluative responses to other assignments. Assignment of points for posts/responses is not automatic but earned. Points will be assigned to your posts based on completeness of answer, following format, following instruction, and proper writing. Points will be assigned to your responses based on evaluative content. Clearly, all responses must be well thought out, well organized and well written. I agree followed by a restatement of points from the post is not acceptable. The response must reflect serious consideration of the cohort s answer, add to it or question it, this shows evaluation. You may do this by incorporating outside material into the conversation (this does not mean you need to read numerous other books. Relate something from recent

reading, personal experience or go on line and see what other information is available). The time between posts is long enough to accommodate a posting, an evaluation of the post and on-going on-line discussion. This will allow cohort communication to clarify, question, etc. This activity is required and will be a component of the response grade. Please be interactive, you will have the entire week after posting your material to comment and create a conversation. You must do more than comment one time. If you read and comment once (on a case), even if the comment shows signs of genius, you will receive no more than 5 points. To receive the full 10 points you must be engaged in an on-going conversation. The same will be true for commenting on the assigned papers. Check your posts often and get involved in the discussion. We will be working on your final paper, section by section as we move through the class. You will be assigned incremental sections of your paper. You are to post them in your assigned group and to the discussion board. The objective is for you to use your experience and abilities to produce an excellent final paper and to mentor those with less experience to refine their papers. These will not be graded but you must complete them. Failure to submit these assignments and take part in these coaching opportunities will significantly affect your final grade. Assignments must be submitted by the published due dates unless special permission has been obtained. Twenty-five percent (25%) of the total assignment points will be deducted for each week an assignment is late. It is the faculty s responsibility to provide a structure and framework for learning and to give guidance and direction to all learning experiences. Each student is responsible for his/her own learning. Learning is ultimately the responsibility of the student and participation is expected. Student Responsibilities 1. Required assignments must be completed. 2. Involvement in class exercises and participation via the class site is expected. 3. If a student requires accommodation for a disability, he/she should contact the instructor at the beginning of the class session (in the first week of class). 4. Students are to maintain academic integrity (see policy below). Unless directed otherwise, students are to do their own work on all assignments and tests. Plagiarism is never acceptable. Academic dishonesty will result in actions in accordance with the college s disciplinary policy and may also result in loss of credit for the assignment or class.

Evaluation Methods Interview 100 Critical book report 100 Evaluation paper 100 Evaluation of 2 Cohort papers 2@ 20 pts. 40 Cases 2@ 25 50 Comments on cases 2@10 points 20 Comments on book review 1@ 20 points 20 Comments on interview 1@ 20 points 20 Total 450 Grading Scale A = 90-100% B+= 85-89.99% B = 80-84.99% C+ =75-79.99% C =70-74.99% D+ =65-69.99% D = 60-64.99% F = 59.99% and below Grading Criteria Grading criteria will be provided for each assignment. Grading will be done against the established criteria and against each other. Written work has an element of subjectivity; papers vary in depth and breadth. Papers which meet the criteria and display the greatest depth and breadth will receive the highest grades and will serve as the basis against which the remaining papers are measured. Critical Book Review-Due Sunday Week 3 (November 7), midnight You are to select a book on evaluation in your field, preferably one published recently (although there are some classics dated long ago). There are several ways you can locate books, do an internet search using key words relative to you field of work and evaluation, go to the reference section of the class book, go to Amazon.com, enter the class book and click on others like this (this a great way to find related books), etc. Your job is to read the book and write a critical book review. It is highly recommended before reading the book that you review the information on critical review reading and writing from the University of Toronto at www.utoronto.ca/writing/critrdg.html (for

reading) and www.utoronto.ca/writing/bkrev.html (for writing). It is extremely important to understand that a critical book review is not simply a recitation of the information from the book but a review of who the author is, who was the intended audience, how is information delivered, what are the authors objectives and how does the author reach or fail to reach his/her goals. The report will be a minimum of 3 pages, double spaced and will not exceed 6 pages, double spaced, exclusive of title and reference page. All notation, referencing and cites need to be in APA format. Interview Due Sunday Week 5 (November 21), midnight You are to make an appointment with and interview someone in your field that does evaluation. Your primary objective is to determine how evaluation functions within your specific field. At a minimum you must: 1. Describe how they decide who undertakes this function and how/when do they make the determination an evaluation is necessary. 2. Provide information about how your interviewee selects the tools/instruments they use for their evaluations (do they have specific criteria). 3. Describe the process they use to determine the appropriate instrument. 4. Describe how they acquire/develop their evaluation instrument. a. Provide specific information where they get their instruments/tools. b. Describe the strengths and weaknesses of the instruments they use. c. If possible, obtain a copy of the instrument; include it as appendix to your written paper. 5. Describe how they identify their subjects of evaluation. 6. Describe how the evaluation is implemented. d. Detail any special considerations of implementation. 7. Describe the purpose of their evaluation. 8. Describe the audience for whom the evaluation is done. 9. Describe the process used to transform data collected into information. 10. Describe how the information is communicated. 11. Detail how the information is used. 12. Any and all other useful information. Your paper is to be written in narrative style, no more than 6 pages in length, exclusive of a cover page and reference page. If you use references they are to be in APA style. I would recommend you set your appointment early and forward them a copy of your questions so they can give consideration to their responses or clarify if needed. It is preferable to do the interview face-to-face. Make sure you confirm where the interview will take place, preferably their office. Interviews conducted via the internet tend to be dry and devoid of useful information and will make writing your paper unnecessarily difficult. When you are sitting in front of someone you have the ability to observe their surroundings, their body language, and note the tools that assist them in their work. You may offer them confidentiality if you think it will provide better information. Regardless of confidentiality, you must include a detailed description of the company the person works

for and their position within the company. Moreover, a face-to-face interview allows you to shape the conversation. Often people will respond in an unexpected fashion, it is easier to pose a follow-up question to gain deeper insight, to be spontaneous when you are facing someone. It is a good idea to develop an interview protocol and use a tape recorder. The interview protocol should be several pieces of paper with the interview questions spaced far enough apart so that you can take copious notes in case your tape recorder somehow fails. The protocol also allows you to write down notations of things you notice in your surroundings that might add flavor to your interview. It is useful to close your interview by asking the question: Can you think of anything else you think would be useful for me to know? You are asking them to give you any secrets of the trade, which might prove to be interesting. Evaluation Paper Due Saturday Week 7 (Dec. 11), midnight. You are to chose a program and develop an evaluation plan. You are to work with an individual in an organizational setting (can be any setting) to complete this project. As the final attachment to the document you must include the completion document found under course documents. It must be signed and dated to be valid. Your final paper will include: 1. Title page (Your name, the name of the program that is being evaluated, MGMT 551, Fall 2007 Term II) 2. Table of Contents 3. Executive Summary (one-page, concise overview of findings and recommendations) 4. Purpose of the Report (what type of evaluation(s) do you plan to conduct, what decisions are being aided by the findings, who is making the decisions, etc.) 5. Background about organizational and product/service/program that is being evaluated a. Organization description/history b. Product/service/program description (that is being evaluated) i. Problem statement (in the case of nonprofits, description of the community need that is being met by the product/service/program) ii. Overall goal(s) of product/service/program iii. Outcomes (or client/customer impacts) and performance measures (that can be measured as indicators toward the outcomes) iv. Activities/technologies of the product/service/program is developed and delivered) v. Staffing (description of the number of personnel and roles in the organization that are relevant to developing and delivering the product/service/program 6. Overall evaluation goals (e.g., what questions are being answered by the evaluation) a. Rationale and purpose b. Identify key stakeholders 7. Methodology

a. Types of data/information to be collected b. How data/information is to be collected c. How data/information is to be analyzed d. Limitations of the evaluation (e.g., what questions are being answered by the evaluation) 8. Interpretations and conclusions (anticipated from analysis of data/information) a. Communicating results b. Implementation strategy 9. Recommendations (regarding the decisions that must be made about the product/service/program) a. Process for feedback, follow up 10. Appendices: content of the appendices depends on the goals of the evaluation report, e.g..: a. Instruments used to collect data/information b. Data, e.g., in tabular format, etc. c. Testimonials, comments made by users of the product/service/program d. Case studies of users of the product/service/program e. Any related literature 11. Signed/dated completion document Adapted from: Basic Guide to Program Evaluation (see: http://www.managementhelp.org/evaluatn/fnl_eval.htm) Evaluation of cohort evaluation, Due Complete Friday Week 8, midnight You are to do a formal assessment of 2 cohort evaluations Critique of all necessary components is necessary. Students will also engage in two case studies and post preliminary work related to final proposal. Case studies will be graded; participation in preliminary program discussion is required. Class Schedule Reading assignments will be made but I would recommend reading whenever you have the chance. The book is full of information, you might want to get an overview and then go back to specific sections to fulfill assignments. College s Incomplete Coursework Policy To designate a student s work in a course as incomplete at the end of a term, the instructor records the incomplete grade (I). Students may receive this grade only when serious illness, hardship, death in the immediate family, or military service during the semester in which they are registered prevents them from completing course requirements. In addition, to receive an incomplete, a student must have completed substantially all of the course s major requirements.

Unless extenuating circumstances dictate otherwise, students must initiate requests for an incomplete by filling out an Incomplete Grade Completion Contract, which requires the signature of the student, instructor, and Dean. The Incomplete Grade Completion contract cites the reason(s) for the incomplete and details the specific obligations the student must meet to change the incomplete to a letter grade. The date by which the student agrees to complete required work must appear in the contract. The Dean, the instructor, and the student receive signed copies of the Incomplete Grade Completion Contract. Even if the student does not attend Peru State College, all incomplete course work must be finished by the end of the subsequent semester. Unless the appropriate Dean approves an extension and if the student does not fulfill contract obligations in the allotted time, the incomplete grade automatically becomes an F. College s Academic Integrity Policy The College expects all students to conduct themselves in a manner that supports an honest assessment of student learning outcomes and the assignment of grades that appropriately reflect student performance. It is ultimately the student s responsibility to understand and comply with instructions regarding the completion of assignments, exams, and other academic activities. At a minimum, students should assume that at each assessment opportunity they are expected to do their own original academic work and/or clearly acknowledge in an appropriate fashion the intellectual work of others, when such contributions are allowed. Students helping others to circumvent honest assessments of learning outcomes, or who fail to report instances of academic dishonesty, are also subject to the sanctions defined in this policy. Instances of academic dishonesty may be discovered in a variety of ways. Faculty members who assign written work ordinarily check citations for accuracy, run data base and online checks, and/or may simply recognize familiar passages that are not cited. They may observe students in the act of cheating or may become aware of instances of cheating from the statements of others. All persons who observe or otherwise know about instances of cheating are expected to report such instances to the proper instructor or Dean. In order to promote academic integrity, the College subscribes to an electronic service to review papers for the appropriate citations and originality. Key elements of submitted papers are stored electronically in a limited access database and thus become a permanent part of the material to which future submissions are compared. Submission of an application and continued enrollment signifies your permission for this use of your written work. NSCS Board of Trustees Policy 4220 states that each College... will establish a distance learning assessment policy that will include, at a minimum, a substantial culminating experience that is proctored. Peru State College s policy is that each course that is offered entirely online will feature a proctored final exam that substantially measures the extent the course s stated learning objectives are achieved. Online

course syllabi will clearly state that, regardless of grades earned previously, the proctored final exam must be passed in order to receive credit for the course. Courses which feature graded site-based activities (e.g., teaching demonstrations) and/or videotaped presentations that occur near the end of the term, and that are designed to substantially assess the achievement of learning objectives, can be considered in compliance with this policy. Project-based capstone and graduate courses utilizing realtime discussions held by web-cam, phone or in person with the faculty member as part of the assessment process can also be considered in compliance with this policy. Should an occurrence of academic misconduct occur, the faculty member may assign a failing grade for the assignment or a failing grade for the course. Each incident of academic misconduct should be reported to the Dean and the Vice President for Academic Affairs (VPAA). The VPAA may suspend students for two semesters found to be responsible for multiple instances of academic dishonesty. The reason for the suspension will be noted on the student s transcript. A faculty member need present only basic evidence of academic dishonesty. There is no requirement for proof of intent. Students are responsible for understanding these tenets of academic honesty and integrity. Students may appeal penalties for academic dishonesty using the process established for grades appeals. Title IX Compliance Notice Peru State College is an equal opportunity institution. PSC does not discriminate against any student, employee or applicant on the basis of race, color, national origin, sex, disability, religion, or age in employment and education opportunities, including but not limited to admission decisions. The College has designated an individual to coordinate the College s nondiscrimination efforts to comply with regulations implementing Title VI, VII, IX, and Section 504. Inquiries regarding non-discrimination policies and practices may be directed to Eulanda Cade, Director of Human Resources, Title VI, VII, IX Compliance Coordinator, Peru State College, PO Box 10, Peru, NE 68421-0010, and (402) 872-2230.