GUIDELINES FOR ACADEMIC PROGRAM REVIEW For self-studies due to the Office of the Provost on October 1, 2015 GRADUATE PROGRAMS

Transcription

1 GUIDELINES FOR ACADEMIC PROGRAM REVIEW For self-studies due to the Office of the Provost on October 1, 2015 GRADUATE PROGRAMS OVERVIEW OF PROGRAM REVIEW At Illinois State University, primary responsibility for the quality of academic programs resides with its faculty. Review of academic programs offered by the University is conducted by the Academic Planning Committee, an external committee of the Academic Senate. Work of the committee is facilitated by the Office of the Vice President for Academic Affairs and Provost. Program review is carried out in a manner compatible with institutional academic planning mechanisms and with guidelines established by the Illinois Board of Higher Education (IBHE). The program review process is both a critical and constructive process designed to help an academic unit identify strengths and weaknesses of its academic programs. The process is intended to encourage development and maintenance of high-quality academic programs that are administered efficiently and align with the university mission. Program reviews result in recommendations for program development and modification that serve to inform the department or school, college, and University on decisions regarding resource allocation, faculty staffing, program focus, admissions standards, curricular content, and other academic matters. The Academic Planning Committee provides these recommendations in summative reports submitted to the Academic Senate, Board of Trustees, and IBHE. IBHE guidelines require institutions to review all degree programs every eight years to monitor quality and identify needed improvements. Under IBHE guidelines, institutions must conclude if the program being reviewed is in good standing or if an annual interim report is needed to resolve concerns regarding program quality. Such programs are considered flagged and are reviewed annually thereafter until the status of good standing has been achieved. Institutions may also recommend that IBHE temporarily suspend enrollment in programs requiring attention sufficient to warrant such action. Information regarding program review at Illinois State, including program review schedules and guidelines, is available on the Office of the Provost website, at PRINCIPLES OF PROGRAM REVIEW AT ILLINOIS STATE UNIVERSITY 1. Program review occurs in an environment that encourages and values honest, critical, and productive analysis among all participants in the process at all levels. 2. Program review is outcomes-based and forward looking. 3. Program review involves input from multiple program stakeholders, including faculty, administrators, students, staff, alumni, accreditors, and prospective employers. All program faculty members are provided meaningful opportunities to participate in the self-study process. 4. Program review is guided by the student learning outcomes assessment plan for the program. The program review self-study report provides evidence that the assessment plan provides for an ongoing, systematic, and methodologically-sound process for evaluating student learning outcomes as well as evidence that program faculty and administrators are utilizing assessment results to identify and implement improvements to the program. Page 1 of 13

2 5. Goals and quality measures appropriate for guiding improvements to an academic program are established by program faculty. Goals are consistent with long-term university priorities articulated in Educating Illinois (the university strategic plan) and The Illinois Public Agenda (the strategic plan for higher education in the state). 6. Academic units look nationally to identify aspirational institutions for each program quality measure as a means of identifying ways to further improve the program. 7. Academic units compare their program with similar programs at public universities in Illinois to determine whether the program is meeting the needs of the citizens of Illinois in a cost-effective manner. 8. Academic units collaborate with Milner Library to conduct an analysis of resources and services provided by the library for the discipline and to identify opportunities for collaboration between the academic unit and library to further integrate those resources and services into the curriculum and into faculty and student research. PROGRAM REVIEW PROCESS See the program review timeline included with this section. Steps in the program review process are described below. COMPILATION AND ANALYSIS OF DATA Listed below are the primary sources of data available to program faculty for conducting their program review self-study. While these sources should help faculty respond to most questions in these guidelines, additional data may be needed, including data compiled and archived by the department or school. A key local data source is student learning outcomes assessment data compiled on the program level and archived in the unit. That data is not archived centrally. Questions regarding data, including requests for data not otherwise available through the sources below, should be addressed to Bruce Stoffel in the Office of the Provost ([email protected], ). Responses may be shared with other programs conducting self-studies to help them as well. Academic Program Profile The single most complete source of data on the program level is the academic program profile. A profile for each degree program is compiled annually by the Office of Planning, Research, and Policy Analysis and is disseminated by the Office of the Provost to department chairpersons and school directors in October. When discussing program review self-study reports, the Academic Planning Committee (see below) relies heavily on data provided in the academic program profiles. University FactBook The FactBook is compiled each fall by the Office of Planning, Research, and Policy Analysis. The FactBook provides aggregate data regarding the University but not program-level data. See the PRPA Data Center at Page 2 of 13

3 Graduate Data Book The Graduate Data Book is compiled each fall by the Office of Planning, Research, and Policy Analysis each fall. The report provides program-level data for each masters, certificate, and doctoral program. See the PRPA Data Center at Data Center (Office of Planning, Research, and Policy Analysis) In addition to current and archived editions of the University FactBook and Graduate Data Book, the Data Center on the PRPA website provides access to historical enrollment and graduate records by program and sequence, credit hour reports, time-to-degree data, and information regarding university-level comparison groups. See the PRPA Data Center at Alumni and student engagement surveys In addition to assisting programs with student learning outcomes assessment plans, University Assessment Services conducts several surveys of the university community that may be useful in the self-study process. These include the annual Alumni Survey and three surveys of student engagement: National Survey of Student Engagement (NSSE), the Faculty Survey of Student Engagement (FSSE), and the Beginning College Survey of Student Engagement (BSSE). See the survey section of the University Assessment Services website at The Alumni Survey queries alumni one and five years removed from graduation. Programs wanting to use different methodology to query their alumni may contact University Assessment Services for assistance. University Assessment Services staff is available to help program faculty design and administer surveys and to compile results. Library resources and services information Program faculty is asked to work collaboratively with their Milner Library subject specialist to address integration of library resources and services with the program and with faculty research. An initial step in this process is to identify the types of quantitative and qualitative information needed for the analysis. Program review coordinators are encouraged to meet with their subject specialist early in the self-study process to begin data selection and compilation. The types of information selected for the analysis are expected to vary from one program to another based on idiosyncrasies of each discipline. Illinois comparator program information The IBHE website provides access to numerous data products regarding Illinois colleges and universities, including public, independent not-for-profit, independent for-profit, and out-of-state institutions. These products include data regarding programs sharing the same CIP code. See the IBHE website ( particularly the links on the left side of the page to Degree Program Inventory, Enrollments & Degrees, and Data & Statistics. To aid faculty discussion of comparator programs at other Illinois public universities, the Office of the Provost will issue each unit a list of programs with the same CIP code, with historical enrollment and degree data. Please note that the latest IBHE data regarding comparator programs is several years old. The IBHE data provided by the Office of the Provost will be the most recent available. CONDUCTING THE SELF-STUDY The program review process begins with review of the program s Student Learning Outcomes Assessment Plan by the Assessment Advisory Council. This review, initiated by University Assessment Services, occurs approximately two years before the program review self-study is due. The revised assessment plan is then used to compile data for use in the self-study. Page 3 of 13

4 Approximately one year prior to report submission, the academic unit will begin a thorough self-study of the academic program. The unit head (chairperson or director) typically designates a program review coordinator to facilitate the self-study process culminating in preparation of a self-study report. Faculty, students, and staff in an academic unit should have the opportunity to participate in the program review self-study process in meaningful ways. The program review coordinator should contact the appropriate Milner Library subject specialist to collaboratively examine relevant holdings, usage statistics, and library services and to assess the impact of the library on the program. The program review coordinator and subject specialist should identify contributions of the library to student learning and actions to be taken during the next review cycle to ensure integration of library resources and services with the curriculum. The serves as a liaison to the academic unit to answer questions about the review process. University Assessment Services provides guidance in matters related to the student learning outcomes assessment. Units may wish to utilize an external consultant to help identify program strengths and areas for improvement. COMPILING THE PROGRAM REVIEW REPORT The academic unit should follow Academic Program Review Self-Study Report Guidelines included in this document to compile a report for each program scheduled for review. The report consists of two sections. Overview of the Academic Unit This section includes information common to all programs in the unit, such as information about the administrative structure of the unit, facilities, faculty, and advisement. Academic units with more than one program being reviewed during the year should submit only one Overview of the Academic Unit document. Academic Program Review This section includes detailed information unique to the academic program being reviewed. Academic units with more than one program being reviewed during the year should submit a separate and unique Academic Program Review document for each program. If the program submitted a request for accreditation or reaccreditation to a specialized accreditation organization on or after October 1, 2013, the program may use information included in that request to compile the program review report. The program may copy sections of the accreditation request verbatim into the program review report but should not refer to the accreditation report by section or page. The program should not submit the accreditation report with program review documents. The program review coordinator should submit the program review report to the department chairperson or school director by September 1, The department chairperson or school director, in turn, should submit the program review report to the dean by September 15, The dean, in turn, should submit the program review report to the by October 1, In preparing the final version of the report, please avoid using color if possible (to facilitate copying). Please ask your dean s office to submit one print copy to the (Campus Box 4000) and a PDF version of the document to the. Please do not submit faculty vitae with the report. Please do not submit curricula vitae with the report. Page 4 of 13

5 As a courtesy to our colleagues at Milner Library, we ask that you send the library components from your completed self-study report to your library subject specialist. Doing so will help your librarian answer any questions the Academic Planning Committee may have regarding library support for your program and will also help the librarian implement any recommendations you have made for enhancement of that support. REVIEW BY THE ACADEMIC PLANNING COMMITTEE The Academic Planning Committee serves as the primary review committee for program reviews. The committee is an external committee of the Academic Senate. The committee has 15 members including a faculty member from each college recommended by the college dean and confirmed by the Faculty Caucus of the Academic Senate; the chairperson of the Academic Senate; a member of the Academic Affairs Committee of the Academic Senate; representatives from the Office of the Provost, University Assessment Services, and the Graduate School; an undergraduate student appointed by the Student Government Association; and a graduate student appointed by the Graduate Student Association. The Academic Planning Committee meets from October into early spring to discuss each program review selfstudy. The committee may request clarification or supplemental information from the department chairperson or school director during this time. The Academic Planning Committee will write a summative report of the program based on its review of the program review report and any additional information provided by the program. The summative report includes recommendations for program development and modification that serve to inform the department or school, college, and University on decisions regarding resource allocation, faculty staffing, program focus, admissions standards, curricular content, and other academic matters. A draft of the summative report is provided to the unit head and appropriate dean for review and comment. The and members of the Academic Planning Committee will then meet with the dean, unit head, program coordinators, and any other interested faculty members to review the summative report prior to its inclusion in the draft Academic Plan. APPROVAL OF PROGRAM REVIEW SUMMARIES AND RECOMMENDATIONS The Academic Planning Committee submits the summative reports to the Academic Senate as part of the draft Academic Plan. The Academic Plan then moves forward to the Board of Trustees for approval before being submitted to IBHE. Thereafter, the Provost s office collaborates with appropriate deans and chairpersons/directors to monitor implementation of program review recommendations. Page 5 of 13

6 PROGRAM REVIEW TIMELINE FOR GRADUATE PROGRAMS SUBMITTING SELF-STUDY REPORTS IN FALL 2015 Before conducting the program review self-study Fall 2013 Fall 2013 Spring 2014 Unit heads and program review coordinators meet with University Assessment Services staff to review the Student Learning Outcomes Assessment Plan. Program faculty works with University Assessment Services and the Assessment Advisory Council to revise the Student Learning Outcomes Plan. Data collection begins. Compiling the program review self-study report October 2014 Spring-Summer 2015 September 1, 2015 September 15, 2015 October 1, 2015 Unit heads and program review coordinators meet with the to review the self-study process. Program faculty members review data available for the selfstudy, including student learning outcomes data, and begin collecting additional data and information as needed. The Office of the Provost disseminates academic program profile data. Program faculty members review information collected and write the self-study report The Program Review Coordinator submits the program review report to the Chairperson/Director. The Chairperson/Director works with the Program Review Coordinator to revise the report and then submits it to the Dean. The Dean works with the Chairperson/Director to revise the report and then submits it to the Provost s office. After submitting the program review self-study report October 2015 February 2016 March 15, 2016 March-April 2016 April 2016 May 2016 July 2016 August 2016 The Academic Planning Committee reviews program review reports, requests additional information from Chairpersons/Directors as needed, and drafts summative reports and recommendations. Summative report drafts are distributed to Chairpersons/Directors and Deans. The and members of the Academic Planning Committee meet with the Dean, Chairperson/Director, and faculty to review the summative report draft. The summative report is finalized based on the conversation with the Dean, Chairperson/Director and faculty and is incorporated into the draft Academic Plan. The draft Academic Plan, including summative reports, is reviewed with the Academic Senate. The draft Academic Plan, including summative reports, is reviewed by the Board of Trustees. The Board approves the plan as presented or amended. The Academic Plan and the summative reports are submitted to IBHE. University Assessment Services Chairperson/Director; Program Review Coordinator Program Review Coordinator Program Review Coordinator Program Review Coordinator Chairperson/Director Dean Academic Planning Committee, Provost Page 6 of 13

7 ACADEMIC PROGRAM REVIEW SELF-STUDY REPORT GUIDELINES GRADUATE PROGRAMS The program review self-study process involves submission of two documents that together constitute the program review self-study report. The documents should be submitted simultaneously. OVERVIEW OF THE ACADEMIC UNIT This document describes aspects of the academic unit common to all programs offered by the academic unit. If more than one program in the academic unit is scheduled for program review, only one overview should be submitted for all programs. ACADEMIC PROGRAM REVIEW This document provides detailed information and analysis regarding the degree program being reviewed. If more than one program in the academic unit is scheduled for program review, a separate and unique academic program review document should be submitted for each program. Page 7 of 13

8 OVERVIEW OF THE ACADEMIC UNIT NOTE: The term unit refers to the department or school that administers the program being reviewed. 1. Describe the administrative structure of the academic unit. 2. Describe degrees and sequences offered by the academic unit. 3. Describe minors offered by the academic unit. 4. Provide the following information regarding faculty in the academic unit. Do not submit curricula vitae with the report. a. Describe qualifications required to be hired as a tenure track faculty member. Do all tenure track faculty members hold a terminal degree? If not, why have exceptions been made? b. Describe qualifications required to be hired as a non-tenure track faculty member. c. Describe typical faculty assignments for tenure track and non-tenure track faculty. d. Describe expectations for scholarly productivity of tenure track faculty members. e. Define underrepresented populations pertinent to faculty in the discipline (e.g., race/ethnicity, gender). Then describe goals for faculty diversity, actions taken toward meeting those goals, and results achieved. 5. Describe facilities used by the academic unit, including classrooms and laboratories on and off campus. 6. Describe academic and career advisement for students in the academic unit. a. How does the academic unit facilitate transition of students from University College advisement? b. How does the academic unit facilitate transition of transfer students? c. How does the academic unit support students transitioning into and out of a major? 7. Collaborate with the appropriate Milner Library subject specialist to review collections, instruction, and services provided by the library to students and faculty in the academic unit. Then provide a narrative written collaboratively with the subject specialist that responds to the following questions. a. How does the academic unit integrate library facilities, collections, and services into research and learning? b. How does the academic unit work with the subject specialist to select library resources that facilitate student learning and faculty research? c. What goals and strategies have the academic unit and library collaboratively established for improving library resources and services that support the academic unit during the next eight years? 8. Attach a copy or provide a link to the current strategic plan for the academic unit. Page 8 of 13

9 ACADEMIC PROGRAM REVIEW Section I: Self-study process Describe the process used to conduct the self-study, including faculty and student involvement, key actions taken, the project timeline, and the process used to gather feedback from key external stakeholders such as alumni and employers. Section II: Description and analysis of the academic program A. Overview 1. Identify goals of the academic program and describe how they align with Educating Illinois. 2. Describe how the academic program furthers one or more goals of The Illinois Public Agenda See 3. Discuss enrollment trends relative to enrollment targets. 4. Describe student recruitment activities including use of scholarships. 5. Describe financing of graduate students, utilization of graduate assistants, and training of graduate teaching assistants. 6. Discuss processes used by the program to track alumni. Where have program graduates been employed or, for students continuing their education, enrolled? 7. Define underrepresented populations pertinent to the discipline (e.g., race/ethnicity, gender). Then describe goals for diversity of the student population, actions taken toward meeting those goals, and results achieved. 8. Identify any accreditation this program (or program sequences) receives from an external organization. For each accreditation provide the following information: name of accrediting organization, date of last accreditation review, current accreditation status, and date of next scheduled review. B. Curriculum 1. Describe admission requirements, curriculum design, and graduation requirements. Include in your description responses to the following questions. How often are sub-waivers and exceptions requested on degree audits? Is there a pattern of common exceptions and/or substitutions? If so, does the program have any plans to address those needs (for example, through curricular changes or changes in course scheduling)? 2. The Higher Learning Commission (university accreditor) requires that at least 50 percent of courses applied to a graduate program are courses designed for graduate work, rather than undergraduate courses credited toward a graduate degree. Accordingly, all master s degree programs at Illinois State must require that at least 50 percent of the non-thesis credit hours applied to the degree must be earned in 400-level courses or higher. Provide a chart or table demonstrating compliance of the program with this requirement. Page 9 of 13

10 3. What actions does the program take to promote graduation within the catalog time limit? Of the graduates during the last eight years, how many were past the catalog time limit? Of the students currently in the program, how many are past the catalog time limit? 4. Describe student participation in research and creative activities such as presenting at research symposia, exhibitions, and professional conferences. How does the program track students participation in such activities? 5. In the Overview of the Academic Unit you have described collaboration between the academic unit and Milner Library. In this section, identify broad information fluency skills needed for success in this academic program (e.g., research, citation, finding and using data). Describe how the program collaborates with Milner Library to teach those skills. 6. Identify co-curricular opportunities available to students in the program and describe how they relate to learning objectives of the program. C. Student learning outcomes assessment Attach a copy or provide a link to the student learning outcomes assessment plan for the program. Describe the learning goals measured, assessment methods and metrics used, and stakeholders involved. Describe the process used to analyze assessment data and identify program modifications. Describe program changes made since the last program review based on assessment outcomes. If the program is delivered off-campus, is the same assessment plan used for the off-campus version? Are assessment data collected and analyzed separately for on- and off-campus program offerings? If courses are offered via distance education, are the same goals, methods, and metrics applied to them? D. Faculty 1. Have all eligible faculty members in the academic unit become members of the graduate faculty? 2. In the Overview of the Academic Unit you have provided information regarding faculty in the academic unit. In this section, identify core faculty members teaching in the academic program and provide evidence of their leadership in the discipline at the national level. Do not submit curricula vitae with the report. E. Non-traditional program delivery 1. Distance learning Definitions: Distance education course: A course in which 75 percent or more of the instruction and interaction occurs via electronic communication, correspondence, or equivalent mechanisms, with the faculty and students physically separated from each other. This includes both synchronous and asynchronous interaction. Distance education program: A program in which a student can earn a degree by taking 50 percent or more of the work in distance education courses (as defined above). Page 10 of 13

11 Based on these definitions: List distance education courses offered as part of this program. How are decisions made about which courses are taught via distance education? Is training offered to faculty members interested in teaching distance education courses? Is this a distance education program? 2. Off-campus program delivery Definition: A program is considered to be offered at an off-campus location if 50 percent or more of the credit hours required to graduate from the program are offered at one or more sites not located on the Normal campus. A situation in which one or more courses are offered at one or more off-campus locations but collectively constitute less than 50 percent of the credit hours required for graduation is considered an off-campus offering of courses but not of the program. If this program is delivered at off-campus locations, please provide the following information. At what off-campus locations is the program offered? How has the academic unit recruited students into this off-campus program? Which faculty members teach at off-campus locations? Do these faculty members teach the same courses on campus? If additional faculty members are hired to teach at off-campus locations, describe the professional credentials of new hires. If additional faculty members are hired to teach courses on-campus to allow other faculty members to teach at off-campus locations, describe professional credentials of the hires. How does the program assure appropriate oversight of instruction at off-campus locations? How do students at off-campus locations connect with advisors? How do students at off-campus locations access library and computer services? Describe reporting and feedback mechanisms used to address concerns or problems of students enrolled in courses at off-campus locations. Section III: IBHE comparator and aspirational programs A. IBHE comparator programs What other Illinois universities offer a program similar to this one? How does our program compare to those similar programs at other Illinois public universities relative to curriculum, enrollment, and degrees conferred? Page 11 of 13

12 B. Aspirational programs Identify programs at other universities to which our program aspires and identify specific actions our program might take to achieve the levels of quality in those aspirational programs. STEP 1: Identify quality indices most appropriate for the program. Identify between three and five quality indices appropriate to the program/discipline and consistent with goals described in Section II.A.1. Examples: accreditation admission standards student quality student-faculty ratio curriculum (e.g., breadth, depth, capstone experiences, internships) co-curricular opportunities faculty qualifications and accomplishments publications and presentations by students success of program graduates in obtaining employment or admission to higher education licensing examination pass rates facilities, laboratories STEP 2: Identify programs at other universities that excel relative to the selected indices. These programs may be located at any institution. They may, but need not be, located at IBHE comparator institutions or at institutions identified by Illinois State University as comparator or aspirational institutions. While you may seek data to compare institutions, this analysis need not be quantitative and need not be extensive. Use of anecdotal information or personal knowledge of institutions in the discipline is sufficient. STEP 3: Identify actions our program might take to meet or exceed quality levels of aspirational programs. Section IV: Responses to recommendations resulting from the previous program review Summarize actions taken by the program to address recommendations in the last IBHE program review eight years ago. Please list each recommendation as it appears in the program review summary you have been provided, followed by a brief description of actions taken. Section V: Changes in the academic discipline, field, societal need, and program demand Describe any changes since the last program review in external factors affecting the academic program. Address changes in the manner in which students in the discipline are taught, professions for which students are trained, the demand for program graduates, and the demand for the program by prospective students. Page 12 of 13

13 Section VI: Major findings of this program review self-study Summarize the state of the program based on findings from the self-study. What aspects of the program are working well? What aspects should be improved? Is there continued demand for the program as it is, or should the program be modified to better meet student needs and respond to changes in the discipline? Section VII: Initiatives and plans for the next program review cycle Based on major findings of this program review self-study, identify actions to be taken by program faculty during the next program review cycle to improve the program. EXECUTIVE SUMMARY Prepare a summary of the completed program review self-study report using the outline below. Your executive summary will serve as the basis for the review submitted to IBHE. Therefore, in preparing your executive summary, please assume that its readers are not familiar with the program or academic unit and will not have read the complete program review self-study report. Please use the executive summary to recap points made in the report rather than introduce new information. Please limit the summary to three pages. Self-study process Program curriculum Program or academic unit faculty Program goals and quality indices Student learning outcomes assessment plan and process Specialized accreditation Responses to recommendations resulting from the previous program review Changes in the academic discipline, field, societal need, and program demand Major findings of this program review self-study Initiatives and plans for the next program review cycle Page 13 of 13