Using the Degree Qualifications Profile: Institutional Examples of Promising Practices

Transcription

1 Using the Degree Qualifications Profile: Institutional Examples of Promising Practices Jillian Kinzie, Indiana University & NILOA Pat Hutchings, Gonzaga University & NILOA Margaret Bailey, Point Loma Nazarene University Jennifer Helm, American Public University System Karan Powell, American Public University System

2 The Plan Brief Introduction to DQP NILOA s role and overview of findings Three campus examples > Point Loma Nazarene University > American Public University System > Georgia State University and Georgia Perimeter College Conversation and Questions

3 What is the DQP? What have you heard? Are you using the DQP on your campus?

4 What is the DQP? Framework that illustrates what students should be expected to know and do -- regardless of major Specifies five learning outcomes critical for all academic programs Applied learning Intellectual skills Specialized knowledge Broad knowledge Civic learning Knowledge, skills and abilities in 5 areas grow, like the spiral threads of a spider web

5 NILOA NILOA s mission is to discover and disseminate effective use of assessment data to strengthen undergraduate education and support institutions in their assessment efforts. SURVEYS WEB SCANS CASE STUDIES FOCUS GROUPS OCCASIONAL PAPERS WEBSITE RESOURCES NEWSLETTER LISTSERV PRESENTATIONS TRANSPARENCY FRAMEWORK FEATURED WEBSITES ACCREDITATION RESOURCES ASSESSMENT EVENT CALENDAR ASSESSMENT NEWS MEASURING QUALITY INVENTORY POLICY ANALYSIS ENVIRONMENTAL SCAN DEGREE QUALIFICATIONS PROFILE

6 NILOA s partnership with the DQP NILOA is serving as information harvester for all of the funded and unfunded work currently going on with the DQP

7 NILOA and the DQP Document what is being done, by whom, and lessons learned; Identify synergies and cross-cutting issues; Provide support to campuses; Gather information to help guide the upcoming revision of the DQP.

8

9 DQP Corner

10 NILOA s DQP Corner Resources New to the DQP DQP in Practice DQP Resource Kit DQP Event Calendar DQP Community Forum DQP Webinar Series

11

12 What We re Learning

13 Institutional Control of Institutions Working with the DQP Public Private not-for-profit Private for-profit N/A

14 403 institutions working with the DQP: 238 institutions are unfunded 165 institutions in Lumina funded projects What is the nature of this work? Discussion of DQP Outcome Review Curriculum Mapping Transfer Program Development Accreditation Strategic Planning Assessment Other No. of institutions

15 DQP as a conversation starter, and a common vocabulary for talking about outcomes Curricular mapping where are we teaching these things, where are the gaps? The importance of faculty engagement, which takes time. The challenge of a new vision of assessment beyond exo-skeleton, built into classroom assignments, for all students.

16 DQP has compelled faculty to think collectively about the course sequence and program as well as degree in ways we never had before.

17 Point Loma Nazarene University

18 Degree Qualification Profile CIC & WASC pilot program Point Loma Nazarene University, San Diego DQP Team Leaders: Dr. Kerry Fulcher, Provost and Chief Academic Officer Dr. Maggie Bailey, Vice Provost, Program Development and Accreditation Dr. Maria Zack, Chair Mathematical, Information and Computer Sciences

19 PLNU Scope of the pilot program Piloted in undergraduate academic programs with a culminating experience for the major (e.g. capstone, senior seminar, action research, senior exhibition, etc.). The DQP Task Force surveyed department chairs to identify what majors had strong culminating experiences used for degree program summative assessment. Seven program directors volunteered their capstone courses for participation in the DQP pilot in Spring DQP Task Force, identified a subset of DQP learning outcomes they would assess during the pilot. The Task Force agreed to use AAC&U Value Rubrics for assessment (e.g. written communication, quantitative literacy, critical thinking, information literacy). Compared the results of the DQP pilot with institutional results from the ETS Proficiency Profile and NSSE.

20 PLNU Scope of the pilot program PLNU academic programs participating in the pilot: Art & Design (dropped out of pilot) School of Business School of Education Literature, Journalism and Modern Languages (2 programs) Mathematical, Information, and Computer Sciences School of Theology and Christian Ministry All DQP reports, assignments, surveys and supporting materials are on PLNU s public website:

21 DQP pilot program Goals (original design) Develop a set of GE learning outcomes that can be externally benchmarked and also incorporate the unique aspects of institutional mission. (NOTE: shifted to Program Learning Outcomes) Provide additional accountability for progress on student learning outcomes Provide a record of the challenges a that a mission driven university faces when engaging in a national conversation around basic learning expectations for an undergraduate education while maintain the mission aspects in its learning outcomes Provide a clear set of reference points for continuous improvement of the General Education program Develop a common language around student learning for curricular improvements in the institution Expedite the academic programmatic approval and implementation process Form a basis for developing a system of external benchmarking with participating institutions with similar faith-based mission

22 Accomplishments & findings Compared three sets of data: 1. NSSE 2. ETS Proficiency Profile 3. Pilot DQP outcomes and assessments Movement towards culminating experience in all majors Use of AAC&U Value Rubrics (adopted these for WASC Core Competencies) DQP Intellectual Skills include the WASC Core Competencies Meaning, value, and integrity of the degree Key conversation about linking Program and GE Student Learning Outcomes (begins with the mapping of the core competencies)

23 Challenges 1. The complex language of the DQP outcomes 2. Cross disciplinary comparisons 3. Faculty engagement was not uniform 4. In some cases faculty viewed DQP as an add on rather than an enhancement 5. Academic leadership in the participating programs was key

24 Going forward During the DQP pilot program PLNU was engaged in important curricular conversations and work to reframe the PLNU vision around the meaning and value of the degree in the context of student success. These conversations will continue as the university commits to curricular and co-curricular continuous improvement in the midst of disruptive changes. Examination of DQP Pilot, NSSE, and ETS results for continuous improvement GE revision of Student Learning Outcomes and program review Culminating experience for every major (an issue addressed in program review) DQP Learning areas for culminating/ capstone experience Cross-disciplinary comparisons and linking to GE SLOs WASC core competencies (Intellectual Skills), AAC&U Value Rubrics, and mapping that links GE and Program Outcomes

25 American Public University System

26 American Public University System American Public University System has been dedicated to educating those who serve since our doors first opened in The American Public University System (APUS) is a fully online system, encompassing both American Military University and American Public University. We are regionally accredited by the Higher Learning Commission (HLC) of the North Central Association.

27 About APUS AMU was founded in 1991 to serve active duty and retired military members, became APUS in 2001, comprising AMU and APU. Initially accredited by DETC in 1995, now regionally accredited through NCA/HLC since 2006 Currently over 100,000 students worldwide Approximately 60% military, average age 35, open enrollment Over 90 degree programs; 21 associate s, 40 bachelor s, and 30 master s level, plus certificates Monthly course starts of 8 or 16 weeks Courses offered are 100% online in an asynchronous environment Headquarters and executive offices Charles Town, WV Administrative offices Manassas, VA

28 Implementing the DQP Framework at APUS Strengthen the overall university and discipline level curriculum. Further enhance the curricular mapping of courses and programs. Enhance our readiness to implement revisions to our core learning curriculum from a strategic perspective. Ensure that APUS curriculum is current and relevant with industry standards. Assist in our focus on the discipline and the integration of the field of study. Provide transparency to the student by establishing expectations of the student life cycle, helping them learn what leads to a career and the next level of study. Assist students with taking responsibility and ownership of their own learning. Ensure that our students are equipped with the knowledge, skills, and abilities they need for success in work, citizenship, global participation, and life.

29 Dialogue and Planning Implementation Assembled leadership team Paul Gaston visit Introduced DQP framework Stakeholder input DQP resources developed DQP review meetings conducted Program Directors mapped programs to the DQP framework to determine alignment and identify gaps Working sessions Core learning mapping Incorporate into new program and review processes Develop learning communities to share DQP related assignments and assessments The DQP leadership team sought input from stakeholders, defined assumptions, developed a timeline of action, defined project goals, benchmarked schools, and created sample maps Decisions on initial program changes recommended by year end 2013 for implementation in 2014 Six new institutional learning outcomes developed five DQP areas of learning plus digital information literacy. Student learning data evaluated to determine impact of DQP framework on student learning and success

30 Support and Resources Important to Success APUS DQP Leadership Committee and Review Teams

31 Trends and Themes Across Programs The current process of identifying gaps in the programs is serving as a tool/framework for a deep dive analysis of the curriculum and providing a roadmap for future program development. The overall curriculum is being enhanced to address the civic skill area of learning which encourages students to take an active role in the community (work, service, co-curricular activities) and examines civic issues encountered and insights gained. The overall curriculum is being enhanced to address the broad integrative skill area of learning which encourages instructional methods to expand beyond the discipline or field of study to ensure an inter-disciplinary approach for students. Instructions on assignments and syllabi are being clarified by instructors to increase student understanding of expectations.

32 Trends and Themes Across Programs Program objectives are being modified and enhanced to ensure they align with core course learning objectives and assignments. Course assignments are being revised and enhanced to ensure the assessment methods accommodate various learning styles to enhance the student learning experience. Programs are continuing to recognize the need to communicate in multiple media formats in providing students with enhanced options for responding to forums and creating assignments. The process is providing a means for faculty to gain a holistic picture on how their courses fit into the overall progression and sequencing of curriculum. The process is providing a solid framework for the progression and sequencing of courses through the AA, BA, and MA levels from the student perspective.

33 Georgia State University and Georgia Perimeter College

34 AASCU DQP Project Exploring student transfer - AA to BA Leveraging characteristics of the DQP to provide a better means of predicting and aiding student success in the transition from Associate s- to Bachelor s level education than traditional achievement markers such as course grade and cumulative GPA

35 Institutions Involved in AASCU DQP Project Georgia Perimeter College (GPC) 2-year, Associates granting Highly Diverse (60% non-white) 25,000 students on 6 campuses Georgia State University (GSU) Research One institution Highly Diverse (60% non-white) 32,000 students IMPORTANT: Between 6,000 and 7,000 current GSU students began their studies at GPC

36 AASCU DQP Project: Developing the DQP Step One Establish a DQP for both Associate s and Bachelor s students in each of the three targeted disciplines (Bio, Psych, Criminal Justice). How would faculty members in specific disciplines across institutions go about developing a Degree Qualification Profile for their fields? What are the obstacles? What are the advantages? 36

37 AASCU DQP Example -- Degree Profile Matrix: Psychology Learning Goals I. Knowledge, Skills, and Values Consistent With the Science and Application of Psychology Goal 1: Knowledge Base of Psychology Students will demonstrate familiarity with the major concepts, theoretical perspectives, empirical findings, and historical trends in psychology. Goal 2: Research Methods in Psychology Students will understand and apply basic research methods in psychology, including research design, data analysis, and Interpretation. 37

38 AASCU DQP : Degree Profile Matrix: Psychology GSU and GPC Specialized Knowledge (APA Goals 1, 10: Knowledge Base, Career Planning & Development) Civic Learning (APA Goals 5, 8, 9: Values, Sociocultural and Interna onal Awareness, Personal Development) GSU GPC Broad, Integra ve Knowledge (APA Goal 6: Informa on & Technological Literacy Applied Learning (APA Goals 2, 4: Research Methods, Applica on of Psychology) Intellectual Skills (APA Goals 3, 7: Cri cal Thinking, Communica on Skills) Figure 1: Degree Qualifications Profile for Psychology at Georgia Perimeter College and Georgia State University. Lines indicate the degree to which APA-defined goals within the five cores are pursued within the psychology major at the associate s level (GPC, yellow) and the bachelor s level (GSU, blue). 38

39 AASCU DQP: Measuring DQP Competencies Step Two The Assessment Question!! Establish a means of determining student DQP attainment in each of the selected disciplines. How to assess DQP competencies fairly and accurately? Can these assessments reasonably be used on an individual student level to gauge past achievement and to predict future performance? Can they be used on a program level? 39

40 AASCU DQP: Degree Profile Matrix: Psychology First Day Assessment Instructions. Without using a calculator, please find the solution to each of the problems below. Show and label all of your work on a separate piece of paper; put only the answers in the space provided below. Your performance on this assessment will NOT affect your grade in this course; it will not be graded Estimating Answers (Estimate whole-number answers to the problems) 1. (7.02) ,989 X 8.01 Rounding Numbers (Round numbers to the nearest tenth) Decimals Add: Subtract: Multiply: * Divide: / Fractions Add: Subtract: Multiply: * 5 4 Divide:

41 AASCU DQP Some Preliminary Lessons The DQP is a positive disrupter in faculty curricular discussions. The DQP is effective in raising for faculty members issues central to successful transfer. The usefulness of the DQP to the transfer process hinges on developing accurate, trusted and easily admissible ways of assessment. (Parallels to prior learning) Contact Tim Renick, Vice Provost, Chief Enrollment Officer at Georgia State University for more information. 41

42 Discussion 1. Familiar themes? 2. Questions? 3. Challenges you re facing?