Evaluating Graduate Education: Reviewers Perspectives A Session for Peer Reviewers

Transcription

1 Evaluating Graduate Education: Reviewers Perspectives A Session for Peer Reviewers Fritz Erickson, Gail Jensen, Joe Miller, and Roberta Teahen April 2014

2 Session Outcomes Attendees will 1. Enhance their capacity to evaluate evidence that would contribute to affirmation of an institution s capacity for offering high-quality graduate education and meeting the standards of the Higher Learning Commission. 2. Receive summary notes following session.

3 Session Overview Setting the Stage Teahen Status of graduate education Where are you now? (survey) Fireside Chat Reviewer perspectives of breadth of considerations Audience identification of the characteristics of a Graduate Culture

4 Overview (continued) Panelists contributions in some broad areas of focus elements of Quality: Mission: Miller Resources: Erickson Infrastructure: Jensen Assessment: Teahen Audience identification of evidence sources and links to HLC Standards (reference handouts)

5 CGS Findings Nearly 1.74 million students in graduate programs in fall Distribution of student enrollments Six out of ten graduate students were enrolled at public institutions, Three out of ten were at private, not-for-profit institutions, One of 10 were at private, for-profit institutions.

6 CGS Graduate Program Statistics (cont.) Two-thirds of all first-time graduate students were enrolled full-time in fall About 58% of all first-time graduate students in fall 2012 were women. For those known, 82% were U.S. citizens and permanent residents. One-quarter of all first-time graduate students were members of racial/ethnic minority groups.

7 So what is the problem? Concerns about student debt levels coupled with ability to repay, often associated with graduate education Concerns about graduate outcomes in the forms of appropriate employment Concerns about the adequacy or quality of graduate education, especially as more providers join the marketplace Concerns about the lack of a graduate culture (appropriate level of scholarship?)

8 Your Perspectives on Graduate Culture Frequently reviewers write that an institution lacks a graduate culture. Take 3 minutes to talk at your tables about what constitutes a graduate culture. What are the characteristics or artifacts that you would find? Rapid Reports will follow so annoint a spokesperson!

9 One Perspective Visit the California State University Stanislaus site for their perspective on graduate culture. You will find a reproduction of their list on your tables as well as at this website after April 15:

10 Mission Clearly stated Guides operations

11 Questions for peer reviewers: A. Mission statement allows for graduate education? - Specified in mission documents? - Separate mission statement?

12 Mission Questions (continued) B. Why? Do mission statements guide programs, allocation of resources, and assessment? Strategic Plan includes graduate education?

13 Mission Evidence C. Evidence: - Mission Documents - Supporting Documents - Strategic Plan

14 Financial Considerations When considering the financial viability and sustainability of a graduate program it is vital to consider three dominant factors - adequacy, stability, and predictability. Adequacy is reflected in meeting programmatic needs. It can be defined through specific programmatic accreditation requirements.

15 Financial Considerations (cont.) Stability is reflected in a budget model is reflected over time and not subject to rapid change. Predictability is reflected in the shared knowledge of the budget model and the reliability of the budget model to fund the graduate program.

16 Graduate program growth Undergraduate focus New expectations

17 New graduate growth Undergrad

18 HLC Criteria: Organization/Oversight 3A The institution s degree programs are appropriate to higher education 3A1 Levels of performance 3A2 Differentiate degrees 3B Demonstrate exercise of intellectual inquiry 3B5 Faculty and students contribute to scholarship, creative work and discovery of knowledge to extent appropriate to programs 4A Responsibility for quality of educational programs 4A1 Regular program review 4A6 Evaluates success of graduates 4B Assessment of student learning

19 Graduate Education: Organization/Structure The genius of American graduate education is that no one is in charge.kenneth Prewitt, 2006, pg23) NOT one size fits all.. If you have seen one Graduate School St you have seen one Graduate school Neither FISH nor FOWL Generally tuition does not flow Do not own faculty (secondary appointments) Soft power/best job in the world Interdisciplinary perspective/incubator

20 Organizational/Structural Challenges Diversity of programs Variety of expectations Faculty appointments Quality academic standards Support for students Support for research/scholarship Resistant organizational culture

21 Council of Graduate Schools (8 elements) Administrative support Faculty unit/supervision of graduate study Faculty oversee curriculum/guide researchscholarship Chief academic officer Degree granting unit Equity in standards (school or college) Graduate program director Graduate council/board Student representation

22 Assessment and Evaluation First, reviewers should distinguish between assessment data and evaluation or effectiveness data. The term assessment is reserved for the analysis of achievement of student learning outcomes at whatever level university, program, course, or module/unit graduate and undergraduate Evaluation is often broader, as characterized in typical program reviews.

23 Summary on Assessment... See sample questions. What you are attempting to establish through your questioning is: if learning outcomes are specified at appropriate levels, whether data is analyzed, the extent to which faculty and relevant others are involved, and whether improvements are crafted to enable continuous improvement of the graduate programs.

24 Probable Components of Evaluation Program Review Course Evaluations Employer Surveys Current or Recent Student Surveys Graduate/Alumni surveys (including those integral to federal and state reporting) Focus Groups with varied constituencies Advisory Committees

25

26 Table Talk Consider graduate education holistically, identifying what you believe would characterize high-quality graduate education and current thinking from Best Practices. Also consider topics not already addressed by the panelists, such as marketing, enrollment, needs assessments, graduate assistants, rank and tenure, pressures for online, policies, scholarship, faculty roles, etc. Note: For your use, we have provided a template of the most relevant HLC Standards.

27 Report Out In 1-2 minutes per table: 1. Name the topic 2. Identify the appropriate HLC Standard 3. Suggest best practices 4. List possible evidence sources

28 Quick Survey

29 HLC Perspectives Bob Appleson and Jeff Rosen will share perspectives from the Commission staff

30 To be continued... The vital next step is the writing of evidentiary statements that contain specific information that supports your conclusions... One piece of evidence does not a pattern make! Writing effective evidentiary statements is the topic of other and upcoming sessions!

31 Panelists Contact Information Fritz Erickson, Provost and VPAA, Ferris tate University, Gail Jensen, Creighton University, Joe Miller, John Carroll University, Roberta Teahen, Associate Provost, Ferris State University,