WILFRID LAURIER UNIVERSITY Waterloo Brantford Kitchener Toronto Cyclical Program Review Handbook 2015-2016 Prepared by the QUALITY ASSURANCE OFFICE wlu.ca/qao
TABLE OF CONTENTS Contact Information... 4 Quality Assurance Office Contacts... 4 Additional Contacts... 4 Introduction... 5 Objectives of the Cyclical Review Process... 5 Quality Assurance in Ontario... 5 Changes in the Laurier Quality Assurance Process... 6 Program Review Schedule... 7 Steps in the Review Process... 7 Program Review Flowchart...10 Definitions... 11 Qualities of Effective Self-Studies... 13 Using This Handbook to Complete the Self-Study... 15 Volume 1: The Program... 15 1. Background... 15 2. Objectives of the Program... 16 3. Admission Requirements... 18 4. Structure and Administration of the Program... 18 5. Curriculum... 19 6. Assessment Methodologies... 20 7. Resources... 20 8. Quality Indicators... 21 9. Conclusion... 25 Appendices... 25 Course Syllabi... 25 2
Instructions... 25 Volume II: Curricula Vitae... 26 Instructions... 26 Volume III: Proposed External Reviewers... 28 Instructions... 28 Choosing External Reviewers... 28 Additional Advice for Selecting External Reviewers... 29 External Reviewers Site Visit... 30 Unit Roles and Responsibilities... 30 Summary... 33 Print Resources... 33 Online Resources... 33 Appendix A Undergraduate and Graduate Degree Level Expectations... 34 Appendix B Modes of Delivery Considerations... 40 Benefits and Drawbacks... 40 Selecting a Mode of Delivery... 41 Alignment with Learning Outcomes and Further Considerations... 42 Summary... 43 Appendix C Sample Alumni Letter Request... 44 3
Contact Information QUALITY ASSURANCE OFFICE CONTACTS Pat Rogers Associate Vice-President: Teaching & Learning progers@wlu.ca Ext. 3213 Sally Heath Manager: Academic Program Development and Review sheath@wlu.ca Ext. 4017 Michael Daly Quality Assurance Coordinator mdaly@wlu.ca Ext. 4654 ADDITIONAL CONTACTS Rob Arnold Associate Registrar: Systems Office of Institutional Research rarnold@wlu.ca Ext. 6352 Charlotte Innerd Head, Collections and Acquisitions (Library) cinnerd@wlu.ca Ext. 2073 Jan Basso Director: Co-operative Education & Career Development jbasso@wlu.ca Ext. 4137 Helen Paret Manager, Graduate Administration hparet@wlu.caext. 3132 Additional information, resources and templates can be found on the Quality Assurance Office website: www.wlu.ca/qao 4
Introduction This handbook was written by the Quality Assurance Office at Laurier to assist academic units in completing the cyclical review Self-Study. The purpose of this handbook is to outline the objectives, timelines and requirements of the review process, to explain in greater detail the types of evidence required for the successful completion of all sections of the Self-Study, and to identify the resources that exist to assist with data collection. In addition to this handbook, the Quality Assurance Office supports units throughout the review process in various ways, including consultations, learning outcome and curriculum mapping workshops, student focus groups and survey administration, advice on the completion of the Self-Study as well as feedback on the required draft report prior to official submission. Please do not hesitate to contact us with any questions you might have. Objectives of the Cyclical Review Process The objectives of the cyclical review process at Laurier are: To promote high quality undergraduate and graduate curricula through regularized examination of academic programs; To improve the quality of the learning experience for students through the identification of strengths and weaknesses of the program design, as well as opportunities for improvement; To ensure that each academic program has articulated program-level learning outcomes for its graduates, which are aligned with its curricular, learning and assessment activities; To encourage academic units to reflect, analyze, and evaluate the current state of their program curriculum and to encourage planning for the future; To promote transparency in program review for the purposes of quality enhancement. Quality Assurance in Ontario Within higher education, there has been an international movement towards increased transparency and accountability through rigorous quality assurance processes. For example, higher education institutions in Europe, Australia, the United States and Canada have all been moving towards an outcomes-based assessment or review approach, which can be defined as a systematic process in which program faculty and/or professionals articulate the intended results of the cumulative contribution of their program. (Bresciani, 2006, p.14). 1 Ontario has engaged in the process of externally audited program reviews at the graduate level since 1968 through the Ontario Council of Graduate Studies (OCGS) and at the undergraduate level since 1996 through the Undergraduate Program Review Audit Committee (UPRAC). In 2005, the Ontario Council of Academic Vice- Presidents introduced a set of province-wide standards articulated through the Undergraduate and Graduate 1 Bresciani, M.J. (2006). Outcomes-based Academic and Co-Curricular Program Review. Sterling, VA: Stylus. 5
Degree Level Expectations. The UDLEs and GDLEs, as they have come to be known, were accepted at Laurier as the institutional degree level expectations for students completing undergraduate or graduate degrees. These degree level expectations provide a framework for academic units to establish their own discipline-specific program-level learning outcomes. In 2010, a new quality assurance body was created in Ontario to harmonize the cyclical reviews of undergraduate and graduate programs. This governing body, the Ontario Universities Council on Quality Assurance (Quality Council), created a new Quality Assurance Framework (QAF) that drew from the best practices within Ontario and from which each publicly-funded higher education institution in Ontario would base their own Institutional Quality Assurance Procedures (IQAP). In order to fulfil the requirements of the QAF, Laurier significantly revised its two academic policies that fell under the category of quality assurance: Policy 2.1 and 2.2. These two policies now comprise Laurier s IQAP and were ratified by the Quality Council on June 20 th, 2011. Changes in the Laurier Quality Assurance Process Like all other universities in Ontario, Laurier has revised its quality assurance processes to ensure alignment with the Quality Council s Quality Assurance Framework. The policy and protocol for new program development, program modification and cyclical review contained in Laurier Policy 2.1 (cyclical review) and Policy 2.2 (new programs and curriculum changes) now constitute Laurier s Institutional Quality Assurance Procedures. Previously governed at the undergraduate level by the Undergraduate Program Review Auditing Committee (UPRAC) and at the graduate level by the Ontario Council for Graduate Studies (OCGS), the program development and program review processes for both degree levels are now governed by the Quality Council. What s Changed? Policies 2.1 and 2.2 have been revised to accord with the Quality Assurance Framework Establishment of the Quality Assurance Office to provide pedagogical and administrative support to academic units in the development of new programs and the cyclical review of existing programs Increased emphasis on program-level learning outcomes and their achievement Creation of a Program Review Sub-Committee of the Senate Academic Planning Committee that manages the program development and review processes Move from a maximum 7 to 8 year review cycle Undergraduate and graduate reviews scheduled as augmented (combined) reviews whenever possible and desirable; accreditation reviews accommodated in schedule where feasible Final Assessment Reports for all cyclical reviews submitted to the Quality Council and posted on the Quality Assurance Office website Annual Implementation Reports submitted in the years between review cycles Auditing of the university s adherence to its own IQAP performed by the Quality Council on a 8 year cycle 6
Program Review Schedule The schedule for cyclical reviews is managed by the Program Review Sub-Committee and reviewed annually. With the harmonization of the review process at the undergraduate and graduate levels, every effort has been made to schedule these as augmented reviews so that both degree levels may be reviewed at the same time. Joint programs at other institutions were also consulted in an effort to coordinate schedules, as were professional programs subject to external reviews through accreditation bodies. The current schedule for all program reviews through 2025 can viewed online. Requests for changes to the current review schedule should be submitted in writing to the Quality Assurance Office prior to the commencement of the review cycle. Deferral requests will be evaluated by the Program Review Sub-Committee and submitted to the Senate Academic Planning Committee for information. All programs must be reviewed within eight years of their initial approval or their last review. Steps in the Review Process The cyclical review process commences with a letter from the Associate Vice-President: Teaching & Learning notifying the Department Chair or Program Coordinator of the academic unit that their cyclical review will be taking place in the upcoming year. This letter will be sent in September of the year prior to the final submission of the Self-Study Report and will be in accordance with the current review schedule as posted on the Quality Assurance Office website. The review schedule is determined and managed by the Program Review Sub- Committee and is updated annually. Requests for changes to the review schedule should be made in writing and submitted to the Quality Assurance Office. Provided that there is sufficient rationale and that the request does not breach the requirement that each program be reviewed within eight years of its previous cyclical review, the deferral request will be taken to the Program Review Sub-Committee for discussion and approval and the academic unit will be notified of the decision in writing. The unit will identify a single self-study author. Typically, authorship of the review is assigned to the program Chair or Coordinator, although the expectation is that the Self-Study represents the collective faculty perspective and effort. The most successful and effective reviews are those in which responsibility for the Self- Study writing is shared amongst faculty. Between notification of the cyclical review and July of the following year, the Self-Study is completed by the designated author in collaboration with his/her colleagues. As outlined in later sections of this manual, much of the data required for the completion of the Self-Study will be provided by other departments on campus, such as Institutional Research and Planning, the Faculty of Graduate and Postdoctoral Studies (for graduate programs), the Library, and Co-operative Education and Career Development. A draft of all three volumes of the Self-Study is due by July 1 st. This draft is submitted for feedback to the Quality Assurance Office and the relevant Dean(s). Feedback will be provided within six weeks of receipt. During this time, the Quality Assurance Office will also contact the prospective external reviewers identified in Volume III to complete the template for each candidate, as well as to determine their availability and interest in serving. This will help to expedite the process of scheduling the site visits once the Program Review Sub-Committee has met to rank the proposed reviewers. Feedback from all sources should be taken into consideration when preparing the final version of the Self-Study, which is due to the Quality Assurance Office by September 15 th. Timely submission of the Self-Study is required 7
to ensure that the remaining steps in the process can take place on schedule. Requests to postpone the due date of the Self-Study should be made in writing to the Quality Assurance Office prior to the review cycle being initiated. The Program Review Sub-Committee holds a meeting each year in late September or early October to rank the proposed external reviewers. Once the reviewers have been ranked, they will be contacted in order by the Quality Assurance Office to confirm their interest and availability for a site visit between November and March to ensure that it can take place prior to the end of the winter term, since meetings with students are a key component of the site visit. One internal reviewer from outside the academic unit will be selected, along with one external reviewer for undergraduate programs or two external reviewers for graduate program or augmented reviews. If applicable, requests for additional reviewers for an undergraduate review can be made in writing to the Quality Assurance Office. The Quality Assurance Office covers all costs associated with the external reviewers honorariums and travel expenses. In advance of the site visit, the Quality Assurance Office will provide the reviewers with a copy of the unit s Self- Study, as well as information to ensure that they understand their role and obligations, which include: To identify and comment on the program s strengths and creative attributes; To identify areas for improvement and opportunities for enhancement, as well as to recommend specific steps that can be taken to improve the program, distinguishing between those the program can itself take and those that require external action; To recognize Laurier s autonomy to determine priorities for funding, space, and faculty allocation; To respect the confidentiality required for all aspects of the review process. The Quality Assurance Office will work with the unit to determine the itinerary for the site visit. During the two day site visit, it is required that the reviewers meet with the following: Vice-President: Academic Associate Vice-President: Teaching & Learning Faculty Dean and Dean of Graduate and Postdoctoral Studies (where applicable) Program Chair or Coordinator Faculty and staff Undergraduate and graduate students (where applicable) The University Librarian The reviewers may also meet with any other relevant community members who can provide useful information about the program under review. Time will be built into the site visit itinerary to allow the reviewers to begin drafting their report, the final version of which is due to the Quality Assurance Office within six weeks of the visit. The Reviewers Report will comment on the program s adherence to the evaluation criteria set out in Policy 2.1 and include an executive summary suitable for being made public on the Quality Assurance Office website. Once the Reviewers Report has been submitted, the academic unit will have one month to prepare the Unit Response and submit it to the Quality Assurance Office. The Unit Response is an opportunity to clarify or correct 8
issues raised in the Reviewers Report, as well as to answer any questions or concerns raised by the reviewers. The Quality Assurance Office will forward the Unit Response along with other relevant documents, to the Vice- President: Academic, the Associate Vice-President: Teaching & Learning and Dean(s). Following receipt of the Unit Response, the Dean(s), in consultation with the Vice-President: Academic and Associate Vice-President: Teaching & Learning, will write the Final Assessment Report. The Final Assessment Report will: Summarize / synthesize all previous review documents; Identify program strengths and opportunities for improvement; Prioritize those recommendations approved for implementation; Include an Implementation Plan identifying the responsibility and timelines for recommendations to be implemented. The Final Assessment Report is a public document that will be posted on the Quality Assurance Office website. Examples of completed Final Assessment Reports can be found here. A separate confidential section may be included to deal with personnel issues if applicable. The completion of the Final Assessment Report is coordinated by the Quality Assurance Office and is submitted to the Program Review Sub-Committee upon completion. The Program Review Sub-Committee will review the Final Assessment Report and make a recommendation to the Senate Academic Planning Committee. It will also be submitted to Senate as part an annual package on program reviews, as well as to the Board of Governors and the Ontario Universities Council on Quality Assurance. The Final Assessment Report is the last phase of the cyclical program review. However, in the years following the review, the unit will submit an annual Implementation Report to the Program Review Sub-Committee that addresses the progress made on recommendations approved for implementation in the Final Assessment Report. The Program Review Sub-Committee will review the Implementation Report and submit it to the Senate Academic Planning Committee. The completed report will then be returned to the unit. The Implementation Report is due by June 15 th of each year until the next cyclical review takes place or until all the recommended changes agreed to be implemented have been made. Any questions or concerns about the cyclical review steps and process should be directed to the Quality Assurance Office. 9
Program Review Flowchart The following flowchart offers a visual representation of the cyclical review process: Self-Study (September 15 th ) Selection of Reviewers by PRS (October) Reviewers Site Visit (November- March) Reviewers Report (within six weeks of site visit) Unit Response (within one month of Reviewers Report) Final Assessment Report (within one month of Unit Response) Review by Program Review Sub-Committee (by end of academic year) Approval by Senate Academic Planning Committee (by end of academic year) Final Assessment Report to Senate and Board of Governors (by end of academic year) Annual Follow-up on Implementation Plan by Program Review 10 Sub-Committee
Definitions The following definitions, some of which have been excerpted from the Council of Ontario Universities Quality Assurance Framework and the Laurier calendar apply to the terms described in the Self-Study and may help to clarify the requirements. Academic Year: For the purposes of the Self-Study Report, the academic year refers to all three academic terms unless otherwise indicated. Academic Services: Academic Services are defined as those services integral to a student s ability to achieve the learning outcomes expected from a program. Such services would typically include, but are not limited to, academic advising and counselling appropriate to the program, information technology, library and laboratory resources directed towards the program, and internship, co-operative education and practicum placement services where these experiential components are a required part of a program. Blended Learning Course: A blended learning course is an integrated teaching/learning approach that combines regular classroom contact time with an enhanced use of online instruction and activities. Typically a blended learning course has a reduction in the number of required classroom contact hours in lieu of work done online outside of the classroom. Core Faculty: Core faculty are defined as those faculty members who are considered essential to the unit s teaching staff. Core faculty includes all full-time teaching staff, but may also include Contract Academic Staff members with a history of teaching within the department. Course: A unit of study in a given discipline identified by a unique number and name in a given department. The weight or value of a course is defined as follows: a 1.0 credit (or full-credit) course normally consists of three contact hours per week taken over an eight-month period (September - April). A half (0.5) credit course is normally conducted over a four-month period. A credit denotes a full-credit course with 1.0 indicated in the course description. A half-credit course is indicated with the 0.5, and a quarter-credit course as 0.25 in the course description. Degree: An academic credential awarded on successful completion of a prescribed set and sequence of requirements at a specified standard of performance consistent with the OCAV s Degree Level Expectations and the institution s own expression of those expectations. Degree Level Expectations: The Degree Level Expectations established by OCAV serve as Ontario universities academic standards and identify the knowledge and skill outcome competencies that reflect progressive levels of intellectual and creative development. They may be expressed in subject-specific or in generic terms. Graduates at specified degree levels (e.g., BA, MSc) are expected to demonstrate these competencies. Each university has undertaken to adapt and describe the degree level expectations that will apply within its own institution. Likewise, academic units will describe their institution s expectations in terms appropriate to its academic program(s). Degree Program: The complete set and sequence of courses, combinations of courses and/or other units of study, research and practice prescribed by an institution for the fulfillment of the requirements of a particular degree. Emphasis, Option, Minor Program (or similar): An identified set and sequence of courses, and/or other units of study, research and practice within an area of disciplinary or interdisciplinary study, which is completed on an 11
optional basis in partial fulfillment of the requirements for the awarding of a degree, and may be recorded on the graduate s academic record. Mode of Delivery: The means or medium used in delivering a program (e.g., lecture format, blended/hybrid, online, problem-based, compressed part-time, different campus, inter-institutional collaboration or other nonstandard form of delivery). MyLearningSpace: Laurier uses MyLearningSpace by Desire2Learn as its Learning Management System (LMS) for delivery of online courses as well as for support in classroom courses. The LMS allows instructors and administrators to organize and manage course content and students in an online environment. Online Learning Course: An online learning course (with the registration section of 'OC') is offered through a learning management system such as MyLearningSpace and uses technologies to engage learners. Program-Level Learning Outcome: A program-level learning outcome is a statement about what a graduate of the program should be able to know and/or do upon completion of their program. Program of Specialization (e.g., a major, honours program, concentration or similar): An identified set and sequence of courses, and/or other units of study, research and practice within an area of disciplinary or interdisciplinary study, which is completed in full or partial fulfillment of the requirements for the awarding of a degree, and is recorded on the graduate's academic record. It should be noted that: A. A program constitutes full fulfillment of the requirements for the awarding of a degree when the program and degree program are one and the same; B. A program constitutes partial fulfillment of the requirements for the awarding of a degree when the program is a subset of the degree program. Typically, a bachelor s degree requires the completion of a program of specialization, often referred to as a major, an honours program, a concentration or similar. Service Teaching: Service teaching refers to the delivery of courses or elements of a program by a discipline to students from a different faculty, department or discipline. Courses may be compulsory for the student s home program or may be taken as electives. 12
Qualities of Effective Self-Studies The following chart is adapted from the Quality Council s Guide to the Quality Assurance Framework to illustrate the differences between strong and weak self-study practices. FEATURE STRONG PRACTICE POOR PRACTICE GOAL / PURPOSE FOCUS CHARACTER/NATURE OF THE REPORT TREATMENT OF CURRICULUM DEGREE LEVEL EXPECTATIONS / LEARNING OBJECTIVES / OUTCOMES TREATMENT OF DATA AUTHORSHIP The Self-Study is aimed at quality improvement. It analyzes strengths and weaknesses of the program, and identifies areas requiring improvement. The Self-Study focuses on the undergraduate /graduate programs (as required by the IQAP and Quality Assurance Framework). The Self-Study is reflective, analytical, self-critical, and evaluative. The curriculum is critically examined, with an eye to degree level expectations, learning objectives, learning outcomes and to change and improvement. The Self-Study expresses degree level expectations and learning objectives that operationally drive admission requirements, curriculum content, modes of delivery, bases of evaluation of student performance and commitment of resources. Data are analyzed e.g. used as the basis for performance evaluation. Data analysis contributes to the assessment of strengths and weaknesses of the program. The Self-Study results from a participatory self-critical process and documents involvement in its preparation of all faculty in the program, of students and, where The Self-Study is aimed at defending or justifying the status quo. The Self-Study focuses on the academic unit (department), rather than on the undergraduate/ graduate program. The Self-Study is descriptive rather than reflective, analytical, selfcritical, and evaluative. The curriculum is described. The Self-Study does not address or only superficially addresses Degree Level Expectations, learning objectives, or learning outcomes. Raw data are attached as appendices, or used only in a descriptive manner. The Self-Study is written by the Chair, without evidence of buy-in (or sometimes even knowledge) of faculty and students. 13
STUDENT INVOLVEMENT STUDENT ROLE STUDENT SURVEY RELATIONSHIP TO EXTERNAL REVIEWER MANDATE IQAP / QUALITY ASSURANCE FRAMEWORK ELEMENTS INSTITUTIONAL CRITERIA appropriate, community partners. The Self-Study shows active involvement of students in the agenda-setting, the self-analysis, and the preparation of the Report. Students contribute to the preparation of the Self-Study, as well as meeting with the external reviewer(s). A student survey provides another valuable source of input to the Self- Study. The Self-Study does address, and inform, all of the issues external reviewers are asked to review. The Self-Study explicitly addresses each of the elements specified in the IQAP and Quality Assurance Framework. The institution specifies the criteria to be used for judging quality in its program review process and the Self-Study addresses them explicitly. There is no evidence of active involvement of students in the preparation of the Self-Study. Students meet with the external reviewer(s), but have made no input to the self-appraisal. Missing or, if a student survey is included, it is conducted after the Self-Study is prepared, and so makes no input to that document. The Self-Study does not address, or inform, most of the issues external reviewers are asked to review. Some elements specified in the IQAP and Quality Assurance Framework, are not addressed at all, and others are addressed inadequately. Either the institution does not specify the criteria to be used for judging quality used in its program review process, or the Self-Study does not address them adequately. 14
Using This Handbook to Complete the Self-Study The Self-Study template has been designed as a standalone document to be completed by the academic unit during their cyclical review. A blank template is available for download on the Quality Assurance Office website and is reviewed and updated on an annual basis. Feedback on the usability of this document is always welcomed. The template is structured to correspond with the evaluation criteria outlined in Policy 2.1, and this section of the Handbook uses the same structure. In the section below, additional notes have been written to provide clarification and information on the types of evidence required and, where applicable, what resources are available to assist in retrieval or interpretation of the information required. Any questions not answered here can be directed to the Quality Assurance Office. There are three volumes to the Self-Study: Volume I: The Program (plus Syllabi and Appendices) Volume II: Faculty CVs Volume III: Proposed External Reviewers When submitting the draft and final versions of the Self-Study, please ensure that each is saved and formatted as a separate file that is clearly identified. Documents should be submitted in Microsoft Word format to allow for feedback. Volume I should include a Table of Contents and should be paginated accordingly. Volume II should include a title page, Table of Contents with an alphabetical listing of all CVs that have been included, and appropriate pagination. Volume III will be prepared by the Quality Assurance Office based on a list of names and contact information supplied by the unit (see Volume III notes). VOLUME 1: THE PROGRAM 1. BACKGROUND a) This opening section of the Self-Study should provide reviewers with a brief (roughly one page) history and evolution of the program(s) under review, its relevant milestones, and its purpose within the institution. Academic units may find it helpful to refer to the last cyclical review document for some of this historical information; if this information is not on file, please contact the Quality Assurance Office. Please include links to any relevant webpages. b) This section is designed to give reviewers a sense of the typical experience for students within the program and the different pathways / options available to them during their course of study. Please include links to any relevant webpages and/or calendar information. Syllabi for all required courses should be included in a separate appendix with a table of contents (see syllabi instructions). In the case of courses that are taught frequently, a single representative syllabus should be included. c) The Self-Study should be a consultative and collaboratively produced document that represents the collective faculty perspective on the program s strengths, weaknesses, and opportunities for improvement and quality enhancement. Although a primary author is identified, core faculty should be involved in contributing to and providing feedback on the document. Student involvement in the review process is strongly encouraged (e.g.
through surveys and/or focus groups). Indicate in this section who drafted the document, who was consulted, and how the document was reviewed. Any and all contributors to the Self-Study should be identified in this section, including those that assisted with data collection. d) This section should list all recommendations made in the last cyclical review and outline any changes to the program that resulted from those recommendations. For undergraduate reviews, these recommendations will be located in the unit s previous cyclical review documents. For graduate appraisals that were conducted under OCGS, please refer to the OCGS letter that outlines the appraisal outcome and concerns to be addressed in the next periodic appraisal. If any of these documents cannot be located, please contact the Quality Assurance Office or the Faculty of Graduate and Postdoctoral Studies. e) This section is only applicable to professional programs that are subject to external accreditation reviews. Please summarize the results of the any accreditation or professional reviews that have taken place within the scope of the program review cycle (8 years). Depending on length, summary reports may be included in the appendices to Volume I, but full reports should not be included here. f) This section has been purposefully located at the beginning of the Self-Study to allow the unit to foreground any questions they might have for the review committee, as well to identify any concerns the unit has with the quality of the program so that reviewers can provide feedback on these areas. 2. OBJECTIVES OF THE PROGRAM INTRODUCTION A program-level learning outcome is a statement about what a graduate of the program should be able to know and/or do upon completion of their degree. The terms outcome and objective are often used interchangeably, but each implies a different level of measurability and specificity. An objective tends to be more general and broad than an outcome and can be compared to a goal or an aim. Objectives may also be phrased from the perspective of what a course or program will do, rather than what a student should be able to demonstrate. An example of a general program objective might be: this program will equip students with the necessary tools for success in their future careers. This statement does imply a goal of the program, but it is both general and would be difficult to measure, since the necessary tools aren t specified. It also isn t clear what role the student plays in this process. The verb equip is similarly vague, and doesn t indicate how this outcome will be achieved. In contrast to objectives, learning outcomes are often written using the SMART acronym as a guideline. SMART stands for Specific, Measureable, Attainable, Relevant and Time-bound. These criteria can be very useful in developing learning outcomes. Program-level learning outcomes should be meaningful and identify what the unit wants graduates of a program to be able to know or do. They should be written from the perspective of the student rather than the course or program. They should be realistically achievable within the time period that the student will be enrolled in the program, and they have to be able to be measured in some way. The Self-Study asks not only for a statement about what the program-level learning outcomes are, but how they have informed and are aligned with the design of program curriculum and how their achievement is assessed and documented. It is recognized that employing an outcomes-based approach to program development and review is still relatively new process and that it will take some time for the alignment between program outcomes and curriculum to be achieved, as will developing mechanisms for providing sound evidence to support the achievement of these outcomes. The Quality Assurance Office and the Centre for Teaching Innovation and
Excellence can assist academic units in developing a set of program-level learning outcomes if none currently exist. They can also help units in revisiting their outcomes and assisting with alignment between outcomes and curriculum. The province s Undergraduate or Graduate Degree Level Expectations (UDLEs and GDLEs) can be used as a starting point for developing program-specific learning outcomes. Using the UDLEs and GDLEs as a framework for developing outcomes can be useful since alignment between the program s own outcomes and the provincial expectations is required in the Self-Study. The UDLEs and GDLEs also provide an outline of the different types of knowledge and skills that a program-level outcome might address. In writing program learning outcomes, the unit should choose the language that best reflects their discipline as well as the type and level of learning that is being targeted. The movement towards an outcomes-based approach to program development and review has numerous potential benefits for students, academic units, and the institution as a whole. Articulating a set of program learning outcomes can lead to the development or revision of a curriculum that allows the outcomes to be achieved systematically as the student progresses throughout the program. Evaluating the achievement of program outcomes can prompt reflective dialogue about what is working well within the curriculum, as well as provide the opportunity and motivation to make changes when necessary. Learning outcomes can be used to attract students to the program insofar as they make transparent what students should be able to achieve in the course of their degree (in other words, what they will get out of it). Lastly, an institution that makes public its program-level learning outcomes makes a statement about the quality, transparency and potential transferability of the educational experiences that it offers. Units will get the most out of their Self-Study if program-level learning outcomes are already well in advance of the cyclical review. The goal is to be able to use the Self-Study to review the suitability of the current program learning outcomes and their degree of achievement by graduates of the program. Done properly, the Self-Study can be used to analyze whether or not the current program learning outcomes adequately represent the skills and knowledge of program graduates and if not, what adjustments could be made to program curriculum to better ensure their achievement. Reviewing the program learning outcomes in the Self-Study may lead a unit to realize that either the outcomes need to be adjusted or the manner in which they are being realized throughout the program curriculum. For more information about program-level learning outcomes, or to arrange for a workshop or consultation, please contact the Quality Assurance Office. a) This list of outcomes should indicate the knowledge, skills and values that graduates of the program should possess, and that have been met through the design of the program curriculum and its selected assessments. The OCAV Undergraduate and Graduate Degree Level Expectations (see Appendix A) documents can be used as starting points for a program to develop their own set of learning outcomes. Ideally, these outcomes will have been in place for some time prior to the cyclical review. If the outcomes were created for the purposes of the review, this should be indicated, as well as in relevant sections that speak to measurements of achievement of the outcomes. b) In this section, the learning outcomes for students in the program(s) under review should be aligned with Laurier s guiding principles and values as well as the current Academic Plan and this alignment explained in narrative. Some units have found it helpful to present this information in table form.
c) As noted elsewhere in this document, Laurier has accepted the OCAV Degree Level Expectations as the institutional expectations of its graduates. This section of the Self-Study asks the unit to explain the alignment between the program-level learning outcomes and the institutional degree level expectations. A table, accompanied by narrative, may be used to show how the program s own outcomes are aligned with the larger educational goals of the institution. The principle here is that the program curriculum should allow students to demonstrate achievement of both the discipline-specific program learning outcomes as well as the more general competencies of the Degree Level Expectations. d) This section only applies to professional programs, Schools or Faculties (e.g. Business, Education) that must also satisfy accreditation requirements that have its own outcomes, competency expectations or standards. If the accrediting body has its own learning objectives or outcomes that are required to be demonstrated, the Self- Study should indicate how the program s learning outcomes are consistent with these requirements. 3. ADMISSION REQUIREMENTS a) This section of the Self-Study asks the unit to articulate its admission requirements and outline how the admission standards for the program are set in such a way that students may reasonably be expected to achieve the learning outcomes after exposure to and completion of the program curriculum. Note that in many instances, admission cut-offs are set at the Faculty level by the Enrolment Management Committee. Information on faculty and program-specific admission requirements at the undergraduate level can be found here. Graduate program admission requirements can be found here. b) This section is not applicable to all programs, and should be used to outline any additional and/or alternative requirements for admission into the program if such alternatives exist (e.g. reference letters, writing samples). 4. STRUCTURE AND ADMINISTRATION OF THE PROGRAM a) In this section, the unit should explain how the curriculum is designed in such a way that it progressively supports the achievement of the program learning outcomes. This section may be supported by the use of a curriculum map that demonstrates the relationship between the courses within the program and the learning outcomes. For more information about curriculum mapping, please contact the Quality Assurance Office. In the absence of a curriculum map, or in addition to one, narrative should be included in this section that explains how the course design allows for progressive achievement of students in each year of the program. This section also presents an opportunity for units to examine their current course offerings and the relationship between the courses and the program s outcomes and broader goals. In addition, commentary on the suitability of the current complement of course offerings to meet the program learning outcomes and any adjustments that might be necessary to better address the outcomes should be included here. b) This section applies to graduate reviews only. It is a Quality Council (and former OCGS) requirement that at least two-thirds of the courses taken by graduate students in the program are being offered at the graduate (as opposed to undergraduate) level. Please comment on any program regulations regarding the inclusion of undergraduate courses in the curriculum of a graduate student. A table identifying the courses offered to graduate students during the time period under review and their enrolment history will be provided by Institutional Research and Planning.
c) This section applies to graduate reviews only. It is not a requirement for graduate programs to offer fields, but if the graduate program(s) under review do advertise OCGS or Quality Council approved fields, then the unit should explain how these fields are supported appropriately for students, referencing availability of sufficient course offerings and faculty expertise within the fields. d) This section is applicable to graduate reviews only and should be used to describe any opportunities within the program for students to develop professional and/or transferable skills to prepare them for future education or employment (e.g. academic and non-academic career preparation, applying for research funding). Any program initiatives that support the development of these skills should be included. e) The section applies to undergraduate reviews only. If the program(s) under review offer a Thesis option to students, the number of students completing this option during the time period under review should be documented in the table that will be provided by Institutional Research and Planning (assuming that the Thesis course/option is associated with a course code). Units should also comment on changes in Thesis enrolment and/or any changes to thesis regulations during the time period under review. f) The applicability of this section may vary by degree level and discipline. Service teaching refers to the delivery of courses or elements of a program by an academic unit to students from a different faculty, department or discipline. Courses may be compulsory for the student s home program or may be taken as electives. If the unit has access to quantitative information about the percentage of service teaching performed, they may include that data in this section, along with commentary on the impact of service teaching, either positive or negative, to the program(s) under review. Graduate reviews should comment on policies related to course registration from students outside of the program. g) This section applies only to units that offer both undergraduate and graduate programs. The unit to identify any linkages between the programs of the two degree levels, such as the number of students who continue into the graduate program from the undergraduate, any courses open to both graduate and undergraduate students, or the use of teaching assistants from graduate programs in the undergraduate programs. Both advantage and disadvantages of these relationships should be documented here. h) This section is designed to allow the unit to articulate and the reviewers to comment on the future directions and aspirations for the program. Only a summary of the unit s strategic plan should be included here, but fuller documentation can be included as an appendix to Volume I. Many units have found it helpful to hold faculty retreats during the review cycle to discuss future directions of the unit. i) This section is designed to allow reviewers to better understand how decision-making is accomplished within the unit. The unit should provide sufficient information here for the external reviewers to understand this structure and the advantages / disadvantages of the current system. 5. CURRICULUM a) This section asks the unit to explain the currency of its curriculum in relation to the state of the discipline. Comments on how this currency is maintained by faculty are often included. Any perceived gaps in the curriculum and their potential causes should be identified and explained. b) If the program curriculum involves the use of High Impact Practices, then these should be explained here. Narrative regarding the role that High Impact Practices occupy within the broader program curriculum or any
significant changes to the structure or availability of these types of learning experiences during the time period under review can be included here as well. c) This section provides units with an opportunity to draw attention to any unique or innovative aspects of the program, such as the use of blended learning, technology-enhanced courses, or community service learning components. This comparative section also allows units to highlight features of their program unique to Laurier that may not be available to students taking similar programs at other institutions. Barriers to innovation may also be addressed. It is up to the discretion of the unit to decide what might constitute innovative or unique within the context of the program or discipline. Innovation may include but is not limited to use of educational technologies or alternative delivery modes, experiential learning opportunities or high impact practices utilized by the department that students at other institutions may not be exposed to. d) This section allows units to comment on how their academic programming is delivered and whether or not the unit believes these methods are suitable to ensure student achievement of the program learning outcomes (see Appendix B: Modes of Delivery Considerations for additional information on the suitability of different modes of delivery). Units which offer extensive online programming may wish to highlight this aspect of their curriculum here (e.g. number of courses offered to students online, new online course developments). Unit or Faculty-level decisions not to offer any parts of the curriculum via blended or online learning modes should also be addressed, as should future directions or aspirations to incorporate different modes. More information about the unit s current online learning course offerings and registration statistics can be obtained through the Centre for Online Learning. e) This section should document major curricular changes that have taken place since the last cyclical review and are not mentioned elsewhere in the document; in particular those changes that relate to helping students to better meet the program learning outcomes. Any planned or anticipated curriculum changes that would help the reviewers to better understand the future intentions of the program so that they are better positioned to comment on these can be included here as well. 6. ASSESSMENT METHODOLOGIES a) The purpose of this section is to describe the current methods of assessment employed within the program to measure student progress and achievement and to provide an evaluation of the effectiveness of those methods. It is not necessary to describe each assessment used at the course level (although some units provide a summary of this in table form), but instead to profile the signature or disciplinary assessment mechanisms of the program used across courses. This section should also comment on the degree to which graduating student achievement of the program s learning outcomes and the university s Degree Level Expectations are assessed. The evidence used by the academic unit to evaluate whether graduates are able to demonstrate successful achievement of these indicators should be explained. 7. RESOURCES a) This section asks the unit to explain how it makes appropriate and effective use of the existing resources (e.g. human, physical, financial) allocated to it in delivering the program. It is recognized that many units have been subject to financial restraint and budget cuts; in which case, the unit should comment on how adjustments to the program curriculum and administration have been made to adapt to changing economic circumstances.
b) This section should be used to profile and comment on the adequacy of support services used by the unit to support students academic success (e.g. library, lab access, technology support, Writing Centre, Centre for Teaching Innovation and Excellence, Centre for Student Success, Co-operative Education, academic advising). A library report for the program that profiles holdings and resources should be included in the appendices to Volume I and can be obtained by contacting the Head of Collections. Graduate reviews should include the services provided by the Faculty of Graduate and Postdoctoral Studies. Units should try to address any other resources necessary for academic success other than those identified above. Specific mention should be given to the availability of academic advising for students within the program. c) This section asks units to describe the roles and responsibilities of program support staff. Units may wish to use the job descriptions for administrative staff to assist in the completion of this section. Comments on the sufficiency of support staff for the current program may also be included here. d) This section focuses on the external research funding awarded to full-time faculty during the time period under review. Data for the table can be provided by the Office of Research Services, however, the ORS only has access to records for research funding that has been administered through the office. Units may wish to either cross-check and/or supplement this information with its own records or that contained in full-time faculty CVs. Narrative should be included to help reviewers understand the information presented in the table. 8. QUALITY INDICATORS A) FACULTY QUALITY INDICATORS The following three sections (i-iii) all ask the unit to summarize aspects of the collective faculty contributions to scholarship and service. Since the external reviewers will have access to all full-time faculty CVs in Volume II of the Self-Study, it is not necessary to provide a great amount of detail but instead to summarize broadly the types of activities that the faculty is involved in and that contribute to overall program quality. i) In this section, the unit should provide a context for the reviewers to understand how the expertise represented by the collective faculty is appropriate for sustaining the program(s) under review, promoting innovation, and fostering an appropriate intellectual climate. ii) This section should summarize the broader types of service performed by faculty and highlight any outstanding contributions that warrant particular mention and that the unit wants to draw the reviewers attention to. iii) As noted above, full information on scholarly activity conducted by faculty will be available in Volume II (CVs). Units should comment more broadly here about the activity of faculty members within the discipline, as well as the role that students (both undergraduate and graduate) have in faculty research activities. Some units elect to include their own tables here to provide a snapshot of research activity drawn from the faculty CVs. iv) This section focuses on the qualifications and appointment status of faculty who teach and supervise within the unit. A table identifying the number of faculty who were classified as full-time, limited term appointment and contract academic staff during the time period under review will be provided from Institutional Research and Planning. Note that this table is based on information provided through Statistics Canada, and those units who have full-time faculty who are cross-appointed between programs or departments will likely need to add additional notes to this table.