Logistics Carleton University Master / Diploma of Philanthropy and Nonprofit Leadership PANL 5007 Policy and Program Evaluation Winter 2015 Instructor: Class Times: Marc L. Johnson According to Participant s Schedule (819) 205-7459 Online Lectures/Exercises/Discussions marc.johnson@carleton.ca Start Date: January 5, 2015 Skype : prof_mlj End Date: April 8, 2015 Class Room: Online Learning Outcomes This online course provides an overview of selected concepts, issues and processes in program and policy evaluation, utilizing both Canadian and comparative experiences in the philanthropic and nonprofit sector. 1 Its purpose is to prepare students, as leaders of nonprofit and philanthropic organizations, to be good users of evaluation, in their capacity to commission, manage or appreciate the findings of evaluation processes. At the end of the course, students will be able to: Understand the core concepts, theories, methods, and issues involved in policy and program evaluation in the philanthropic and nonprofit sector; Discern what constitutes a quality evaluation; Appreciate the role of evaluation in program design and make effective use of evaluation; Communicate the impact of an organization s work. Course Overview Context The function of evaluation has moved to centre stage in the philanthropy and nonprofit sector. It is clear that this sector s leaders must understand the evaluation process thoroughly in order to effectively design, commission, oversee, receive and utilize performance assessments at any level policy, institution, program, community, household and individual and across a range of policy domains, from environment, to health care or social enterprise. Moreover, evaluation must be 1 This online course was originally prepared by professor Ted Jackson, who kindly passed on his material to this instructor. It has been adapted, but much is still T. Jackson s contribution. 1
undertaken with integrity and efficiency in a context of globalization, change and economic austerity. At the same time, the theory and practice of evaluation itself is undergoing rapid change, and new frameworks and tools are being continually developed, tested and refined. Structure The course is divided into three parts. Part I focuses on general concepts, frameworks and issues in evaluation in the philanthropy and nonprofit sector. Part II considers a wide range of strategies, methods and tools with an emphasis on qualitative, participatory and management- oriented approaches with which to actually do evaluation. Part III involves a group work to prepare the final term paper, which can be a program theory or terms of references for an evaluation relevant to the nonprofit and philanthropy sector. The course will run over 12 weeks. Part I and II will be completed during the first eight weeks of the course schedule. Part III will be undertaken in the final four sessions of the course. In general, course lectures, videos, readings and exercises will be informed by theory and practice of evaluation in professional associations, foundations, nonprofit, international development agencies, and governments in Canada and abroad. The table below outlines the course s calendar and themes. Week Theme Deliverables January 5-11 1.1 Introduction Online discussions January 12-18 1.2 Mixed Research Methods and Technology Innovation Online discussions January 19-25 1.3 Results Frames and Stakeholder Engagement Online discussions Jan. 26 Feb. 1 1.4 Gender, Utilization and Real- World Challenges Online discussions February 2-8 2.1 Managing the Evaluation Process, Maximizing the Learning Dimension Online discussions February 9-15 2.2 Collective Impact and Social Accounting Online discussions February 16-22 - - - - Reading Week - - - - Short paper (Feb. 15) Short paper PPT/lecture (Feb. 22) Feb. 23 March 1 2.3 Participatory Methods and Tools Online discussions March 2-8 2.4 Evaluating Social Impact Bonds Online discussions March 9-15 3.1 Research and Writing I March 16-22 3.2 Research and Writing II Outline of project paper (Mar. 22) March 23-29 3.3 Research and Writing III March 30 Apr. 5 3.4 Research and Writing IV Group project paper (Apr. 8) 2
Course Requirements Participants of PANL 5007 are required to: 1. Generally, participate actively in class discussions, exercises and projects; 2. View the relevant lectures and videos for each session; 3. Read the assigned readings for each session; 4. Respond in a timely way to exercises with brief posts and comments; 5. Research and write, individually, a 5- page (2500 words) paper on an evaluation topic to be negotiated directly with the Instructor. This written output will be due on February 15 th, 2015. 6. Contribute to a group project. The project will be defined in the first part of the course, will be prepared progressively in teams of two or three students, and delivered on April 8 th, 2015. Grading Participants of PANL will be graded on the following elements: 1. Participation through posts and discussions (20%): During the first eight weeks, students will be asked to participate in 4 (4 X 5%) of the 8 course discussions on topics provided by the instructor. Each time, participants will be required to prepare and post two paragraphs on the course discussion board, and then take part to the online discussion sparked by these posts in the current week. The timeliness and quality of posts and discussion will be key in assessing the students participation. 2. Short Paper (20%): Please contact the instructor directly to propose a topic for your short paper. This paper may examine a theoretical or methodological issue in policy and program evaluation in the nonprofit sector in Canada or internationally, review a strategically important document, article or report, or analyze a specific case study of an evaluation, organization or policy. The paper must be five pages (2500 word) and use APA style for references, including non- academic references. Deadline: February 15 th. There will be penalties for late submissions. 3. Preparation of a lecture/presentation (10%): Students will also be required to prepare one brief (5-8 minutes) lecture with a PowerPoint presentation on the topic of their short paper. They may use the Instructors lectures as a model. To prepare this assignment, students will use Carleton s Capture Screen Recording Tool (http://carleton.ca/capture/). The lectures will be posted on the course s CU Learn website for other students to see and learn from. Deadline: February 22 nd. 4. Group Project Paper (50%): Again, please contact the Instructor directly to confirm your topic for the group project paper. This assignment will be done by groups of two. Its purpose is to develop or improve the students capacity in doing evaluation related tasks. Whatever the type of paper chosen, it should relate to a real initiative. At the program design level, the group may develop a program theory 2 or, at the management level, it may develop terms of references for an evaluation. 3 The assignment is two- parts: a. Groups are required to send a 2- page outline of their paper before March 22 (10%); 2 http://betterevaluation.org/plan/define/develop_logic_model 3 http://betterevaluation.org/evaluation- options/terms_of_reference 3
b. Groups will upload their final project paper before April 8 th (40%). There will be penalties for late submissions. The general grading standards of the School of Public Policy and Administration at Carleton University will be used to grade the students outputs for this course. Required Readings Participants will be assigned readings for each week of the class. PANL 5007 will rely primarily on readings that are accessible online. A full list of readings for the course, by topic area, is presented on the course site. Additional Information Instructor Lectures: There are several short (7 to 15 minute) lectures available for your viewing for this course. Focused on knowledge packets, these lectures entail PowerPoint slides accompanied by the Instructor s commentary (either by T. Jackson or M.L. Johnson). Guest Lectures: The course also features a small number of guest lectures, in the form of short videos, from thought leaders and practitioners. Most of these contributions are from Canadians and have been prepared expressly for this course. Additional Videos: Also included in the resources for the course are several short videos from other experts in the evaluation field. These videos have been drawn from the public domain. Contact with the Instructor: At any time during the term, you may contact directly and confidentially the Instructor of this course. Please use contact information provided on page one. University Policy On Academic Integrity Plagiarism is an instructional offence that occurs when a student uses or passes off as one s own idea or product work of another person, without giving credit to the source. The punishments for plagiarism at Carleton are significant. You could fail the course, or, under certain circumstances, be expelled from the university. If you are using someone else s words in a quotation refer to the source in a footnote or bracketed reference. If you are paraphrasing someone else s text (that is, not quoting directly, but closely following the line of argument), refer to the source just as you would for a quotation, except that quotation marks are not used. If you are using someone else s ideas, acknowledge this in a footnote, or by a clear reference in the text of your essay. Material copied from the Internet must be treated like material from a book or any other source. If you are quoting a source you found on the Internet, use quotation marks and refer to the location of the item (name the website; identify the electronic journal and issue, etc.) just as you would for a quotation from printed material. If you are paraphrasing material or borrowing ideas from an Internet source, the source must be identified in a footnote, just as a quotation would be. Excellent software is available for locating material that might have plagiarized from the Internet, and it will be used. Academic Accommodation: You may need special arrangements to meet your academic obligations during the term. For an accommodation request the processes are as follows: 4
Pregnancy obligation: write to me with any requests for academic accommodation during the first two weeks of class, or as soon as possible after the need for accommodation is known to exist. For more details see the Student Guide Religious obligation: write to me with any requests for academic accommodation during the first two weeks of class, or as soon as possible after the need for accommodation is known to exist. For more details see the Student Guide Academic Accommodations for Students with Disabilities: The Paul Menton Centre for Students with Disabilities (PMC) provides services to students with Learning Disabilities (LD), psychiatric/mental health disabilities, Attention Deficit Hyperactivity Disorder (ADHD), Autism Spectrum Disorders (ASD), chronic medical conditions, and impairments in mobility, hearing, and vision. If you have a disability requiring academic accommodations in this course, please contact PMC at 613-520- 6608 or pmc@carleton.ca for a formal evaluation. If you are already registered with the PMC, contact your PMC coordinator to send me your Letter of Accommodation at the beginning of the term, and no later than two weeks before the first in- class scheduled test or exam requiring accommodation (if applicable). After requesting accommodation from PMC, meet with me to ensure accommodation arrangements are made. Please consult the PMC website for the deadline to request accommodations for the formally- scheduled exam (if applicable). Thank you, and I look forward to working with you! Marc L. Johnson 5