PROGRAM QUALITY ASSURANCE PROCESS

PROGRAM QUALITY ASSURANCE PROCESS Curriculum Renewal and Program Review Centre for Learning and Teaching November 23, 2007 (Revised October 31, 2013)

TABLE OF CONTENTS 1. Introduction... 3 2. Goal... 3 3. Outcomes... 3 4. Key Benefits... 3 5. Stages... 4 6. External Audit for Quality Assurance... 4 7. Performance Indicators Used to Measure Program Academic Quality... 5 8. Curriculum Renewal - Overview... 5 8.1 Outcomes... 6 8.2 Timeframe... 6 8.3 Guidelines and Template for Curriculum Renewal... 6 9. Program Review - Overview... 7 9.1 Outcomes... 8 9.2 Timeframe... 8 9.3 Guidelines and Templates for Program Review... 8 Written responses to the self study are required in point or note form... 10 9.4 Deliverables... 11 9.5 Key Players - Roles and Responsibilities... 11 Appendix A... 14 Program Review Guide... 19 Written responses to the self study are required in point or note form... 21 Appendix C... 36 Appendix D: Map of Centre for Learning and Teaching (CLT) shared directory... 43 The online version of this document can be located on the CLT website under Program Quality Assurance: http://flemingclt.ca 2

PROGRAM QUALITY ASSURANCE PROCESS Curriculum Renewal and Program Review 1. Introduction In the fall of 2006, the Centre for Learning and Teaching (CLT) was commissioned to review Fleming s program quality assurance process. The objective was to establish an enhanced approach to program review that would ensure vibrant curriculum with quality programming. The revised process would not only need to align and build on current practices and respond to changing external expectations for quality, but meet the needs of the new Provincial Quality Assurance Process Audit (PQAPA) and province-wide standards and expectations. Following extensive consultation throughout the fall of 2006 and a pilot in the winter of 2007 involving six programs, a new model was developed. This model was subsequently reviewed again in May, 2012 and minor modifications have been included in this version of the process document. 2. Goal The goal of Fleming s Quality Assurance process is to ensure programs are: Vital program curriculum is dynamic, alive, and provides the most valuable learning for entry-level positions in the field of study. Viable programs are relevant, sustainable, aligned with strategic priorities, college initiatives, and meet industry needs and standards. 3. Outcomes The primary outcomes of the program quality assurance process provide a: i. Measurement of program quality and standards ii. Direction for continuous program improvement 4. Key Benefits 1 The key benefits of the revised model include: A standard, college-wide process that is aligned with college values and strategic priorities A design based on consultation and input from key stakeholders A process that is manageable in scope and scale, sustainable, and engaging to faculty Alignment with other academic processes and external demands, eliminating duplication of effort A transparent and constructive approach Systematic data collection over time to inform curriculum change 1 With reference to Program Review, 2000, feedback from APDC & ALT and Curriculum Renewal, 2004-06. 3

Identification of effective practices and the development of a program portfolio 2 A focus on student learning Increased responsiveness and improved capacity to meet the needs of students and employers. 5. Stages The redesigned process comprises two stages: 1. Curriculum Renewal Annually, all programs will engage in an assessment of curriculum content, design and delivery to ensure program currency and vitality. As an outcome of this process, curriculum plans will be developed to guide program activity for the subsequent year. 2. Program Review Every five years, programs will complete a full program review characterized by both depth and breadth of analysis. Program data will be collated, reviewed, and assessed against given criteria, providing a measure of both the vitality and viability of each program. Based on this analysis, long-term program plans will be created to guide continuous curriculum improvement and build on the cumulative outcomes of annual Curriculum Renewal. Any program mandated for a review or accreditation by a governing external body will always use that agency s process rather than Fleming s Program Review. These programs will, however, implement Curriculum Renewal annually, when not involved in an external review process. 6. External Audit for Quality Assurance A new quality assurance review process has been initiated by the Ministry of Training, Colleges & Universities (MTCU) under the direction of the Ontario College Quality Assurance Service (OCQAS) (www.ocqas.org). Each college will be reviewed once every five years in the form of an audit called the Program Quality Assurance Process Audit (PQAPA). The goal of the PQAPA is to ensure that college quality assurance processes are meeting specific criteria and do in fact, effectively measure quality 3. 2 In the context of the program self study, a Program Portfolio is considered to be a collection of the most useful and relevant materials that have helped inform the Renewal/Review activities and that provide a foundation for further exploration. 3 Additional materials related to the PQAPA can be found at the CLT web site under Quality Assurance: http://flemingclt.on.ca 4

7. Performance Indicators Used to Measure Program Academic Quality Program performance indicators include quantitative and qualitative descriptors that provide a measure of how well a program is performing. Performance indicators may have reference points or benchmarks which can be used to compare program performance data to a goal or standard. The indicators selected for Curriculum Renewal address: 1. Industry / sector trends 2. Curriculum development and framework 3. Student and graduate satisfaction 4. Employment trends In addition to the above, Program Review indicators include: 5. Strategic positioning of the program 6. Enrolment trends 7. External relations 8. Program resources 8. Curriculum Renewal - Overview 5

On an annual basis, all programs will engage in Curriculum Renewal. 8.1 Outcomes The outcomes of the Curriculum Renewal process will be: The identification of recent changes in the sector or industry that have an impact on program content or structure. An updated, short term plan for the following year based on the progress made since the most recent Curriculum Renewal or Program Review and the self study exercise The setting of annual performance goals to guide the continuous improvement of curriculum and program quality. 8.2 Timeframe Curriculum Renewal can be completed within a two to three month time frame. The work can be scheduled in the semester that best suits the program and the School. It is recommended that all program faculty and staff participate in the Curriculum Renewal meetings. Much of the work can be completed as part of regular program meetings (weekly or otherwise) and through the engagement of the Program Advisory Committee. The culminating action plan will guide program activities over the following year. Each May / June period, new statistics will be provided by Fleming Data Research to inform this ongoing activity. 8.3 Guidelines and Template for Curriculum Renewal A template to guide Curriculum Renewal is attached in Appendix A. The process is outlined in its entirety in this appendix, which is designed as a stand-alone tool. The Program Co-ordinator will complete the report template, file it in the CLT - School Quality Assurance directory, and submit it to the Dean for discussion and follow-up. The Dean will forward summary reports to the Vice President Academic (VPA). 1. Step One: Getting Started with Curriculum Renewal Key players The Program Co-ordinator in consultation with the Chair will identify the appropriate time for project implementation and the people who need to be involved (see: 9.5 Key Players: Roles and Responsibilities). The most successful Curriculum Renewal process will be one that engages all faculty and staff aligned with the program, as well as students. Organization All programs will use the same, uniform format for reporting and storage of data. A directory has been set up for each program and folders containing some of the key background materials can be found there: S:\shared data\clt\school Name\Program Name 6

All working documents and final reports will be stored in this directory (see Appendix D for directory format). Contributors such as CLT, Program Co-ordinators, and Chairs have full read / write access; others have read only rights. Resources A list of resources considered useful in promoting and informing discussion can be found in Appendix A: Step One. The Centre for Learning and Teaching (CLT) will file updated college reports and Key Performance Indicator (KPI) data as they are released. Additional data can be added by program staff. 2. Step Two: Curriculum Analysis and Development of the Action Plan The program team will examine the curriculum, affirm the strengths of its design and delivery, and identify areas for improvement in response to the key findings. These discussions may occur during program faculty meetings, Program Advisory Committee meetings, and / or special focus groups. Student involvement is essential. The ideas and recommendations that emerge will form the program action plan for the upcoming academic year. The Analysis and Action Plan template (see Appendix A) includes four sections to guide the program team: A) Analysis of indicators: Curriculum indicators include: industry / sector trends, curriculum development and framework, student and graduate satisfaction, and employment trends. The team will reflect on each of these as they examine aspects of curriculum quality. No recording of outcomes is required in the template, but discussion notes will be useful for reference into the future. B) Curriculum strengths and challenges: Curriculum strengths, and any areas requiring improvement, will be identified, outlined, and recorded in this section based on the team s analysis of the indicators. These outcomes are to be recorded in point or note form with a rationale for their inclusion. C) Priority actions: Based on its findings, the program team will develop an action plan for the following year. Recommendations should have a clear rationale and be both realistic and achievable within a one-year timeframe. Action items should include a designated lead and projected timelines for completion. The plan is to be recorded in point or note form. D) Deferred actions: Some curriculum issues may need to be addressed at a later date. Briefly describe those that need to be monitored, researched, or deferred for future action, and the reason for the deferment. Use the Analysis and Action Plan template should be used for reporting - maximum two-to-three pages. Program Co-ordinators will submit plans to their respective Chair/Dean/Principal for discussion and review. The final version will be submitted to the Vice President Academic. 9. Program Review - Overview Every five (5) years, a program will complete an intensive, yet comprehensive review. This will be conducted over an eight-month time period and result in a detailed analysis of program quality. 7

9.1 Outcomes The outcomes of Program Review will be: A comprehensive assessment and analysis of program indicators to determine the viability and vitality of each program A demonstration of a program s alignment with the College Mission, Vision and Strategic Directions and School positioning, priorities, and growth A review of program alignment with new or revised Ministry of Training Colleges and Universities (MTCU) program standards and / or other external standards An examination of internal and external changes or trends that have an impact on the discipline and the program; a critique of the program in relation to them, and a plan to address areas of concern An evaluation of the effectiveness of previous internal recommendations, such as the outcomes from the most recent Curriculum Renewal process A plan to direct curriculum revisions for the upcoming five years that promotes student learning A comparison with other colleges achievements and a revised set of benchmarks to aim for over the next five years Feedback from, and recommendations by, both internal and external stakeholders A resource portfolio that will help to build the program profile and context, over time. 9.2 Timeframe Each year, the Chairs will reconfirm the programs scheduled for Program Review. Those Schools with fewer programs will complete one only, while larger Schools will implement four or more. All programs will be captured in the five year cycle. Schools managing multiple reviews should consider staggering their start times to even out workload. Most new programs will operate for two years before undergoing a full program review, subject to the Dean s discretion. Long-range workload planning will need to be considered when organizing Program Review. It will take two semesters or approximately eight months to complete. Release time for Program Co-ordinators will be required, as well as the scheduling of regular program team meetings to ensure that all faculty and staff teaching in, and supporting, the program have the opportunity to participate. Chairs will need to communicate with each other in order to plan the participation of staff with cross appointments. 9.3 Guidelines and Templates for Program Review Templates to guide the Program Review process are attached in Appendix B. The Chair, with assistance from the Program Co-ordinator, will complete the report template, file it in the directory, and submit it to the Dean. At the end of the process, the Dean will convene a Program Review Panel to evaluate and approve the program long term plan. The Panel is charged with the responsibility of making recommendations concerning program viability as well as suggesting program / curriculum revisions. The Dean will forward an annual summary report to the Vice President Academic (VPA). 8

1. Step One: Getting Started on Program Review Key players The Chair, in collaboration with the Program Co-ordinator, will be responsible for coordinating and guiding Program Review (see 9.5: Key Players Roles and Responsibilities). Support and guidance from CLT Consultants will be provided as needed. The most successful review process will be one that engages all faculty and staff associated with the program. Student focus groups must be held to contribute to the discussions and provide input into recommendations. Organization All programs will use the same reporting format and storage process to promote uniform management of data. A directory has been set up for each program and folders containing some of the key background material can be found there: S:\shared data\clt\school Name\Program Name This directory can be used to store all working documents and final reports (see Appendix D for directory format). Hard copy resources can be stored in the Program Co-ordinator s office. Resources A list of resources considered useful in promoting and informing discussion can be found in Appendix B. Fleming Data Research will file provide updated College reports and KPI data as they are released via their internal website on the College portal. Additional data can be added by program staff. Note: Auditors from the PQAPA will request hard copy evidence of a few program reviews as well as reports on whether recommendations have been implemented. 2. Step Two: Self Study and Action Plan for Program Review Program Review aligns with Curriculum Renewal, but it goes further by examining program issues of greater complexity, as well as depth and breadth. Using the resources available, the team will review, discuss, and summarize program characteristics, affirm the strengths of its design and delivery, and identify areas for improvement. These activities can be conducted during regularly scheduled program faculty meetings and Program Advisory Committee meetings. Student focus groups are required and will need to be scheduled as part of the process. The ideas and recommendations that emerge from these discussions will form a long-term program action plan that will be reviewed annually in the Curriculum Renewal process. 9

The Self Study and Action Plan template for Program Review (see Appendix B) is divided as follows: A) Self Study: Program strengths and challenges will be reviewed, discussed, and summarized across the following eight indicators: industry / sector trends, curriculum development and framework, student and graduate satisfaction, employment trends, strategic positioning, enrolment trends, external relations, and program resources. A range of indicators are included in each section to guide the review. Specific program attributes that are not identified in the template can be noted for future reference. Written responses to the self study are required in point or note form. Program performance in relation to the benchmarked Key Performance Indicator (KPI) scores will be reviewed and interpreted. Based on that assessment, new benchmarks will be set for the following five years (see Appendix C for use and interpretation of provincial KPI scores and setting benchmarks for ongoing improvement). The existing, as well as the new benchmarks should be submitted as an attachment to the plan. B) Action Plan: The review team will establish key recommendations, priority actions and a program plan to guide activity over the next five years. Plans are to be realistic and achievable and will be re-examined annually during Curriculum Renewal. The Self Study and Action Plan template is to be completed in sufficient detail as to demonstrate the quality attributes of the program. During the development of the Self Study, periodic updates should be provided to the Dean and the final report will be reviewed by the Dean prior to convening the Program Review Panel. 3. Step Three: Program Review Panel The Dean will convene a Review Panel to assess the final report. This panel will include the Dean, Program Co-ordinator, Chair, CLT Consultant as well as additional internal and external members as stipulated below: Internal program-specific members, to a maximum of four, and in addition to the Dean, Program Co-ordinator and Chair, may include faculty from the program, faculty from the School of General Arts and Sciences (GAS), program technician / technologists, and student representatives. External members may include faculty from another School external to the college (not more than one), program alumnae, Program Advisory Committee members, and / or other industry or sector experts. At least three (3) external members must be included on the panel. The Panel s responsibility will be the evaluation and approval of the program s Self Study and Action Plan. The panel will be assessing overall program viability, as well as making recommendations for program revisions. The Dean will forward summary reports to the Vice President Academic (VPA). 10

9.4 Deliverables Program Review will culminate in a report demonstrating the following attributes: Assessment of current and changing trends / issues that have an impact on the program, and the intended response to them An analysis of the progress made towards planned curriculum revisions and the responses to past recommendations, and / or an assessment of the effectiveness of changes that have been implemented An evaluation of the effectiveness and currency of the program of study including program outcomes, alignments, teaching and learning strategies, student assessment activities, and evaluative processes A plan for program revisions and revitalization that supports student learning and community engagement. 9.5 Key Players - Roles and Responsibilities The following list identifies the key players and their roles and responsibilities in, and contributions to, Curriculum Renewal and Program Review. Dean: Promotes the need for, and benefits of, program quality assurance processes Oversees process, ensuring timelines are met and appropriate resources allocated Supports Program Co-ordinators in engaging program faculty and staff, Program Advisory Committees, and program stakeholders Ensures participation of cross appointed faculty through consultation with other Deans Monitors and tracks progress of renewal and review initiatives within the School Provides guidance and counsel, as required Assesses curriculum patterns, themes, and new clusters that emerge at the School level Ensures that Renewal and Review updates are regular agenda items at School meetings Convenes final Program Review Panel meetings to assess Program Review reports and recommendations Submits findings and reports for discussion with the VPA Chair: Brokers curriculum and other resources among faculty and staff Facilitates discussions at School, program, and Program Advisory Committee meetings Monitors progress and supports Curriculum Renewal and Program Review initiatives for all programs within the School Aligns resources and integrates activities across programs, where possible Helps co-ordinate faculty / staff participation within, and between, Schools Tracks curriculum patterns, themes, and new clusters that may emerge Prepares and submits Program Review: Self Study and Action Plan report in collaboration with the Program Co-ordinator 11

Attends the final review meeting to assess program report and recommendations Participates in the Program Review Panel Program Co-ordinator: Organizes and distributes key documents generated from previous Renewal and Review activities with assistance from School Operations Liaison (SOL), FDR, Academic Lead, and CLT consultant Schedules regular meeting times that will enable participation of all faculty and staff involved in the program Promotes the participation of ALL faculty and staff engaged in the program (e.g. vocational, non-vocational, general education, technicians / technologists) Schedules Program Advisory Committee meetings and ensures appropriate stakeholder representation (including that of students) Reviews Program Curriculum Maps with faculty / staff, and updates them as required Prepares Curriculum Renewal: Analysis and Action Plan in collaboration with program faculty and staff and submits to the Dean for discussion Assists the Chair in preparing and submitting the Program Review: Self Study and Action Plan and establishing new benchmarks Consults with the Dean on report outcomes and participates in the Program Review Panel Program-affiliated Faculty (all faculty teaching any course in the program): Maintains an active involvement in the renewal and review processes Prepares for meetings, contributes to the process and reviews documents, as required Identifies course-specific issues and contributes to their resolution Contributes to the development of program plans Participates in the review and update of the Program Curriculum Map Centre for Learning & Teaching (CLT) Consultant: Provides support for Deans, Chairs, and Program Co-ordinators Provides guidance and / or continuous learning opportunities on all aspects of program quality assurance and curriculum, to individuals and / or teams Assists in the identification, organization and dissemination of key documents and resources Monitors and tracks progress and supports initiatives for all programs Identifies and tracks curriculum patterns, themes, and issues that emerge within Schools and across Schools School Operations Liaison (SOL): Assists in gathering and organizing required documents Provides support as needed for each of the programs Program Advisory Committee: Identifies and provides context for recent industry / sector related trends or issues of significance to the program 12

Provides feedback and makes recommendations concerning curriculum relevance and overall program quality Participates as a member of the Review team and in the Program Review Panel Program students and graduates: Attend Program Advisory Committee meetings Participates in focus groups and contributes feedback concerning curriculum Participate as members of the Program Review Panel Fleming Data Research (FDR): Provides statistical data and relevant analysis, as required Collaborates with CLT and Schools in the systematic collation and storage of program data Clarifies data, as required Budget Services: Provides program financial data and relevant analysis, as required Assists in gathering and organizing required documents Provides support as needed for each of the programs Learning Support Services (LSS): Provides an annual departmental report regarding the support required and provided for specific programs or courses 13

Appendix A Curriculum Renewal Guide The following is a step-by-step guide to the Curriculum Renewal process: Step One: Getting Started A. Key Players The Program Co-ordinator in consultation with the Dean will identify the appropriate time for project implementation and the people who need to be involved (see 9.5 Key Players: Roles and Responsibilities) The most successful Curriculum Renewal process will be one that engages all faculty and staff aligned with the program, as well as students. Important: Communicate often regarding the importance of the process and schedule meetings early to ensure maximum participation and engagement. B. Organization All programs will use the same, uniform format for reporting and storage of data. A directory has been set up for each program and folders containing some of the key background materials can be found there: S:\shared data\clt\school Name\Program Name All working documents and final reports will be stored in this directory (S:\shared data\clt\school Name\Program Name) under folders named Curriculum Renewal Reports and the map under Program Curriculum Map. Contributors such as CLT, Co-ordinators, and Chairs have full read / write access; others have read only rights. C. Resources A list of documents and data sources of relevance to Curriculum Renewal is included here. KPI data is available via the FDR website. Additional data can be added by program staff. Relevant Resources for Curriculum Renewal Indicates data filed in directory by CLT 1. Industry Trends: Advisory Committee Minutes Faculty observations and industry networking Recent sectoral or labour market reports 2. Curriculum Development and Framework: 14

Previous Curriculum Renewal initiatives Credentials Framework Program Standards Program / faculty meeting minutes Program Curriculum Map (current version) Field placement manuals Other curriculum changes made within the past year (such as calendar descriptions) High school articulations / pathways Program laddering options and university or college transfer options New competitor programs 3. Student and Graduate Satisfaction: KPI Program Data Testimonials from graduates where available Classroom feedback Class representatives feedback Focus groups Program awards 4. Employment Trends: Focus groups Current labour market reports Feedback re graduate readiness for entry-level positions Step Two: Curriculum Analysis and Development of Action Plan The program team will examine the curriculum, affirm the strengths of its design and delivery, and identify areas for improvement in response to the key findings. These discussions may occur during program faculty meetings, Program Advisory Committee meetings, and student feedback. The ideas and recommendations that emerge will form the program action plan for the upcoming academic year. The Analysis and Action Plan template includes four sections to guide the program team in its discussions: A) Analysis of indicators: Curriculum indicators include: industry / sector trends, curriculum development and framework, student and graduate satisfaction, and employment trends. The team will reflect on each of these as they examine aspects of curriculum quality. No detailed recording of outcomes is required in the template, but discussion notes will be useful for future reference. B) Curriculum strengths and challenges: 15

Curriculum strengths, and any areas requiring improvement, will be identified, outlined, and recorded in this section based on the team s analysis of the indicators. Outcomes are to be recorded in point or note form with a rationale for their inclusion. C) Priority actions: Based on their findings, the program team will develop an action plan for the following year. Recommendations should have a clear rationale and be both realistic and achievable within a one-year timeframe. Action items should include a designated lead and projected timelines for completion. The plan is to be recorded in point or note form. D) Deferred actions: Some curriculum issues may need to be addressed at a later date. Briefly describe those that need to be monitored, researched, or deferred for future action, and the reason for the deferment. Use the Analysis and Action Plan template to document results - maximum two-to-three pages. Program Co-coordinators will submit plans to their respective Deans for discussion and review. The final version will be submitted to the Vice President Academic. 16

Curriculum Renewal: Analysis and Action Plan Template 2013/14 Program Coordinator: School: Program Code: Date Completed: Program Name: A. Analysis of Indicators Note: data is not recorded in this section of the template. Reflect on, and discuss, the following indicators in the context of the curriculum and program: 1. Industry / Sector Trends a. Are there new or emergent industry or sector related issues and trends identified over the past year and their potential impact on the program? b. What are the Advisory Committee recommendations from the past year that will affect the positioning, nature, or scope of the program? c. What information / observations have been generated via faculty and staff professional development, engagement in sectoral and profession associations, and involvement in community and employer networks connected to the field? d. Are there new or changing employment trends in the industry or sector? e. What are the curriculum issues / strengths that have been identified by employers pertaining to graduate job readiness? 2. Curriculum Development 2.1 Have there been any curriculum changes in the last year such as changes in course content and course materials, course / program outcomes, innovative delivery approaches, assessment practices, applied learning experiences, e-learning / blended learning? If yes, please provide details. 2.2 Does the current curriculum align with the college s e-learning strategy which strives to have all Fleming graduates experience e-learning in each semester of their program? 2.3 Are there any recent or anticipated initiatives that promote student pathways including dual credits, partnerships with high schools, program laddering, and university transfer / articulations, continuing education? 2.4 Are there any new competitor programs and/or re-positioning of existing programs? 2.5 Are there any new or changing provincial standards, standards for accreditation, credentials, and / or industry or sector certifications over the past year? 2.6 What is the progress made from the last curriculum renewal initiative? 3. Applied Learning 17

3.1. Does the current program contain a discrete Applied Learning opportunity for students? If yes, which category of Applied Learning is fulfilled? Field Work (Indirect Supervision) Field Work (Direct Supervision) Co-op Applied Project / Applied Research Project 3.2. In the 2013/14 academic year, Fleming College will ask all programs with Applied Learning opportunities to align to an agreed upon framework. To confirm program alignment, please complete the appropriate Applied Learning Framework Checklist and attach it to this document. After completing the checklist, please answer the following: Is the program in alignment with the Applied Learning Framework? If no, what are the strategies in place to bring the program into alignment? 3.3. If the answer to 3.1 is no, are there plans to create a discrete Applied Learning opportunity for students within this program? Why or why not? 3. Student and Graduate Satisfaction 3.1 Key performance indicators # 4, 8, 9, and 11 (see Appendix of Curriculum Guide for a description of these). 3.2 Review and discuss student retention on a semester by semester basis over the past year. B. Curriculum Strengths and Challenges Summarize the curriculum strengths and challenges identified by the team. C. Action Plan Identify priority actions for the next year and the rationale for their inclusion. For each, indicate the project lead, and the proposed timelines for completion. What resources are required to complete the action plan, i.e., software, equipment, and training? D. Deferred Actions Record any issues that will need to be monitored, researched, or deferred for future action. 18

E. Attach an updated Program Curriculum Map to your report Please file an updated Program Curriculum Map in folder named Program Curriculum Map.: S:\shared data\clt\school Name\Program Name\Program Curriculum Map Program Review Guide The following is a step-by-step guide to conducting the Program Review process. 1. Step One: Getting Started on Program Review A. Key Players The Chair, in collaboration with the Program Coordinator, will be responsible for coordinating and guiding Program Review (see 9.5: Key Players Roles and Responsibilities) in the main document located at: http://flemingclt.ca/index.php?option=com_content&task=view&id=41&itemid=57 Support and guidance from CLT Consultants will be provided as needed. The most successful review process will be one that engages all faculty and staff associated with the program. Students must be invited to contribute to the discussions and formulation of recommendations. Important: Communicate often regarding the importance of the process and schedule meetings early through the fall and winter to ensure maximum participation and engagement. Final deliberations can then occur in the May/August period in preparation for next planning cycle. B. Organization All programs will use the same reporting format and storage process to promote uniform management of data. A directory has been set up for each program and folders containing some of the key background materials can be found there: 19

S:\shared data\clt\school Name\Program Name All working documents and final reports will be stored in this directory (S:\shared data\clt\school Name\Program Name) under folders named Program Review Reports and Program Curriculum Map (save copy of updated Program Curriculum Map here). Hard copy resources can be stored in the Program Coordinator s office. C. Resources A list of documents and data sources of relevance to Program Review is included here. FDR will file updated KPI data on their website as they are released. Additional data can be added by program staff. Relevant Material for Curriculum Renewal Indicates data filed in directory by CLT 1. Industry Trends: Advisory Committee minutes Faculty observations and industry networking Key sector / labour market reports 2. Curriculum Development and Framework: Previous Curriculum Renewal initiatives Credentials Framework and Program Standards Industry related standards Program / faculty meeting minutes Outcomes from library research Faculty observations Program Curriculum Map (current version) Program course outlines Field placement manuals Other descriptions of curriculum changes made within the past six years High school articulations Program laddering options and university or college transfer options New competitor programs 3. Student and Graduate Satisfaction: KPI Program Data (FDR website) Testimonials from graduates where available Classroom feedback Class representatives feedback Course evaluation summaries 4. Employment Trends: Outcomes from Focus groups Current labour market reports Feedback re graduate readiness for entry level position 20

KPI graduate placement trends (FDR website) 5. Strategic Positioning: College Strategic Plan Academic Business Plan School Business Plan Program Advisory Committee minutes, program meeting minutes Provincial co-ordinator meeting minutes Program Curriculum Map Sectoral and labour market reports, other relevant documents 6. Enrolment Trends: Competitor programs KPI application and registration trends (FDR website) 7. External Relations: Alumnae and Bursary office information Outcomes from Focus groups Program awards 8. Program Resources: Program revenue, expenses, and capital requirements Faculty and staff profiles 2. Step Two: Self Study and Action Plan for Program Review Program Review aligns with Curriculum Renewal, but it goes further by examining program issues of greater complexity as well as depth, and breadth. Using the resources available, the team will review, discuss, and summarize program characteristics, affirm the strengths of its design and delivery, and identify desired improvements. These activities can be conducted during program faculty meetings, Program Advisory Committee meetings, and student focus groups. The ideas and recommendations that emerge from these discussions will form a long-term program action plan that will be reviewed annually in the Curriculum Renewal process. The Self Study and Action Plan template is divided as follows: A) Self Study: Program strengths and challenges will be reviewed, discussed, and summarized across the eight indicators listed in the template. Specific program attributes that are not identified in the template can be noted for future reference. Written responses to the self study are required in point or note form. 21

Program performance in relation to the benchmarked Key Performance Indicator (KPI) scores will be reviewed and interpreted. Based on that assessment, new benchmarks will be set for the following five years. (See attached Appendix for use and interpretation of provincial KPI scores and setting benchmarks for ongoing improvement). The existing, as well as the new benchmarks should be submitted as an attachment to the plan. B) Action Plan: The review team will establish key recommendations, priority actions and a program plan to guide activity over the next five years. Plans are to be realistic and achievable and will be re-examined annually during Curriculum Renewal. During the development of the Self Study, periodic updates should be provided to the Dean and the final report will be reviewed by the Dean prior to convening the Program Review Panel. The Self Study and Action Plan template is to be completed in sufficient detail as to demonstrate the quality attributes of the program. An updated Program Curriculum Map should accompany the report. 22

3. Step Three: Program Review Panel The Dean will convene a Review Panel to assess the final report. This panel will consist of both internal and external members. Internal program-specific members, to a maximum of four, and in addition to the Dean, Chair, Program Co-ordinator, CLT Consultant and may include faculty from the program, faculty from the School of General Arts and Science (GAS), program technician / technologists, and student representatives. External members may include faculty from another School (not more than one), program alumnae, Program Advisory Committee members, and / or industry or sector experts. At least three external members will be included on the panel. The Panel s responsibility will be the evaluation and approval of the program s Self Study and Action Plan. The panel will be assessing overall program viability, as well making recommendations for program revisions. The Dean will forward summary reports to the Vice President Academic (VPA). 3.1 Indicators of Quality Programming The following indicators of quality programming will be used by the Program Review Panel in order to assess the program and provide feedback: Program responsive to changes and trends in industry Program Advisory Committee engaged Curriculum renewal plans enacted Curriculum relevant in present market Pathways for students identified Assessment aligns with program outcomes Students and graduates satisfied with learning experiences Graduates are job-ready Program aligns with both College and School priorities Students interested and enrolled in program Students successfully progress and graduate from program or ladder into other programs Evidence of community support Faculty aligned with program have expertise in their field and the teaching profession Program resources are current Program is financially healthy 23

Instructions for use of Program Review template: The attached template is designed to guide the self study. Reference to background documents may be required. The first column Indicator provides a list of key program indicators to guide the team s thinking and discussion. The second column Summary of Key Findings provides space to capture the outcomes of the discussion, which should be summarized in point or note form. If supporting documentation is required, it should be referenced in the template and included as a separate attachment. 24

Program Review Self Study Template Program Coordinator: School: Program Code: Date Completed: Program Name: Indicator 1.0 Industry Trends Summary of Key Findings 1.1 Sectoral Standards and Industry Trends Review / discuss: New or emergent industry / sector themes or issues that may have a potential impact on program positioning Industry / sector issues identified by the Program Advisory Committee Recent labour market data or sector reports Recent or anticipated changes in occupational standards, level of entry and credential and / or standards of accreditation Program alignment to labour market and sectoral trends Trends identified by the Program Advisory Committee 1.2 Industry Liaison Review / discuss: Program initiatives to maintain involvement with the industry / sector such as field placement supervisions, clinical, faculty renewal, professional learning, other professional affiliations, or communitybased projects 25

Program Coordinator: School: Program Code: Date Completed: Program Name: 2.0 Curriculum Development and Framework Summary of Key Findings 2.1 Curriculum Framework Review / discuss: Describe how your program demonstrates a learner centered approach and addresses our core promise to students concerning personalized learning and support. 2.2 Outcomes from Curriculum Renewal Review / discuss: Key outcomes from the Curriculum Renewal processes of the past few years Progress to date in implementing the recommendations arising from Curriculum Renewal Success of the changes implemented and the means by which they are being evaluated 2.3 Curriculum Sequencing and Alignment with Standards Review / discuss: The Ontario College Credentials Framework and the extent to which the program aligns with the provincial standards. The program s current admission requirements and their suitability in relation to program rigour and student preparedness. The extent to which course content, levels of learning, and assessment methodology are successfully sequenced and aligned between courses and across semesters. 26

Program Coordinator: School: Program Code: Date Completed: Program Name: 2.4 a) Curriculum Map Review the Program Curriculum Map and discuss the extent to which there is alignment of vocational and course outcomes. Review / discuss the distribution and progression of Vocational Learning Outcomes, Essential Employability Skills, and General Education themes across the curriculum. 2.4 b) Curriculum Map Submit an updated curriculum map as an attachment to the Program Review Report. 2.5 Delivery Mode Review / discuss: The primary modes used to deliver curriculum such as lecture, seminar, lab, applied project, field camp and web based courses. The rationale for, and appropriateness of, these delivery modes in relation to program learning outcomes. The degree and depth to which the program is providing work integrated learning experiences. The degree and depth to which the learning experiences are enhanced by the use of educational technology. 27

Program Coordinator: School: Program Code: Date Completed: Program Name: 2.6 Assessment and Evaluation Methods Review / discuss: The program approach to learning assessment. The balance and frequency of assessment types across the curriculum and their appropriateness to course / vocational outcomes. Reflect and comment upon the variety of methods used to demonstrate outcomes. Are learner centered principles part of the assessment approaches? 2.7 Curriculum and Diversity Review / discuss: Program strategies that support student diversity and promote understanding of diversity, including program culture / climate, curriculum content and approaches to teaching and learning. 2.8 Learning Pathways Review / discuss: Recent or anticipated initiatives that promote student pathways including high school articulations, dual credit, program laddering, dual diplomas, and university transfer, articulations, and partnerships. 28

Program Coordinator: School: Program Code: Date Completed: Program Name: 3.0 Student and Graduate Satisfaction Summary of Key Findings 3.1 Formal Measures of Student and / or Graduate Satisfaction Review / discuss: Key Performance Indicator results for the program with a focus on #s 4, 8, 9, and 11. Program status and positioning in relation to the KPIs of other programs of a similar type (where applicable). Feedback and summary report from Learning Support Services (LSS) summary. Themes or issues emerging from a review of course evaluation summaries (Chair/Dean response here). 3.2 Other Measures of Student and Graduate Satisfaction Review / discuss outcomes from: Student focus groups (mandatory component). Student Advisor observations / reports. Formal or informal discussions with students and graduates such as class councils, class representatives, individuals or delegations. Debriefing sessions following a field placement, clinical placement, or practicum. 29

Program Coordinator: School: Program Code: Date Completed: Program Name: 4.0 Employment Trends Summary of Key Findings 4.1 Employment Review / discuss: Graduate employment statistics over the last few years, including those of students employed in the field, in a related field, outside the field, or unemployed, and any emerging patterns in this data. Student preparedness for entry-level positions. Emergent employment trends such as new types of positions, changing job market, regional distinctions, changing employer profile, or emerging skill shortages. 4.2 Other Graduate Destinations Review / discuss: Alternative graduate destinations such as further education, international opportunities, volunteer service, or other experiences. 5.0 Strategic Positioning Summary of Key Findings 5.1 College Alignment Review / discuss: Program alignment with college priorities such as vision, mission, values, strategic plan, academic framework, and the educational mandate, and / or academic priorities of the School. 30

Program Coordinator: School: Program Code: Date Completed: Program Name: Opportunities for new program initiatives based on Program, School, or community strengths and alliances. 5.2 Competitor Programs Review / discuss: Key parallels and differences between this program and those of its closest competitors, where applicable. Value-added program distinctions and their attractiveness to prospective students. 6.0 Enrolment Trends Summary of Key Findings 6.1 Demand for the Program Review / discuss: Patterns in the number of program applicants, qualified applicants, and actual registrants over the past 6 years. Changes, if any, in the student demographic profile, including level of maturity, diversity, prior knowledge, technological literacy, work experience, and expectations. Impact, if any, of this changing student profile on program curriculum. 31

Program Coordinator: School: Program Code: Date Completed: Program Name: 6.2 Student Progression Review / discuss: Patterns of student success and retention on a semester by semester basis over the last six years. The effectiveness of any strategies adopted to improve student success and retention. 7.0 External Relations Summary of Key Findings 7.1 Alumnae Review / discuss: The type and range of alumnae involvement in the program. Current and future strategies to engage alumnae in the program. 7.2 Community Relations Review / discuss: Significant partnerships, relationships, connections, or offers of support from the community that help to enrich the program and the student experience. Faculty, staff, and student involvement in volunteer projects and events. Contributions to the not for profit sector such as committee or board service by program-associated faculty and staff. 32

Program Coordinator: School: Program Code: Date Completed: Program Name: Community recognition in the form of student bursaries, awards and scholarships. 7.3 Program Advisory Committee Review / discuss: The distribution of Committee membership by constituency, sector, and / or region. The vitality of the Committee such as the frequency of meetings, and members level of participation, engagement, and turnover The extent to which Committee operations are aligned with the Fleming College Advisory Committee Orientation Manual and Advisory Committee policy. 8.0 Program Resources Summary of Key Findings 8.1 Human Resources Review / discuss: The number and distribution of all faculty, technicians, and technologists associated with the program including full-time, parttime, sessional, and cross-appointments. Profile of the Dean, faculty, and staff associated with the program including cumulative credentials, scholarship, work-related and teaching experience, and expertise in education. Significant faculty or staff accomplishments such as professional recognition and awards, achievement of credentials, and appointments. 33