Predictive Analytics

Transcription

1 Using Predictive Analytics For Student Success at the University of Hawai i Pearl Iboshi, Director, Institutional Research & Analysis Office SHEEO July

2 Who we are 2

3 Who we are State s single public higher education system, founded in campuses 1 research/doctoral campus (Mānoa) 2 baccalaureate campuses (Hilo and West O ahu) 7 community colleges (on four islands) Education centers throughout the islands Headcount enrollment: approximately 55,000 students Primary commuter campuses No majority ethnic/racial group Every campus is Native Hawaiian-serving 3

4 Basic Infrastructure UH MĀNOA Banner for student data, People-soft for HR data, Kuali for financial data All on one system UH MAUI COLLEGE WINDWARD CC Operational Data Stores UH HILO UH WEST O AHU Common data and generally consistent definitions LEEWARD CC BANNER PEOPLE-SOFT KUALI HAWAI I CC KAUA I CC HONOLULU CC KAPI OLANI CC 4

5 The Move to a Data Driven Culture Focused on Student Success Hawai i Graduation Initiative Strategic Directions measures Graduation Scorecards 15 to Finish Reverse credit transfer GPS and the Right 15 Performance based funding Adoption of early alert system by all 10 campuses Examining course scheduling by all 10 campuses 5

6 Moving the Needle on Student Success Exceeded degree completion goals: overall, PELL, STEM and Native Hawaiians over the period and narrowed degree gaps Highest number of reverse credit transfers degrees awarded in the nation in the past two years (approx. 1,600 AA degrees) Dramatic increase in the number of students taking 15 credits, especially in their first semester Improvement in on-time graduation rates, especially at our flagship (10 percentage point improvement) Increase in the percentage of freshmen coming in with 6 or more early college credits 6

7 Using Predictive Analytics: Started with Internal Studies How can we improve first year retention? How can we identify at risk students? Is the Compass placement test an indicator of success? What are early indicators of risk or probable success? 7

8 Key Findings Semester hours taken had the strongest effect on firsttime freshmen retention. High School GPA was the second most important variable. Asians 58% more likely to be retained than Native Hawaiians, and students coming directly from High School were 67% more likely to be retained. Students using STAR 3.5 times or more during the 2 month period were 75% more likely to be retained. Students not withdrawing from any class in the first 2- months were 55% more likely to be retained 8

9 Joining the Predictive Analytics Reporting Framework (PAR): A Community of Practice

10 PAR Model: Predicting Success of UH CC Transfer Student to UH Four-Year Outcomes considered in the transfer student model: 1. First Term GPA at the UH Four-Year (above/below 2.0) 2. One year retention at the UH Four-Year Key Findings: 1. Positive impacts: AA completed, College level and DevEd English & Math Completed at CC, Full time at first semester, CC GPA. 2. Negative impacts: DevEd taken, Courses repeated 10

11 PAR Course Explorer Heat Map Honolulu Community College Associate in Science Natural Sciences Major 11

12 PAR Course Explorer Dashboard Honolulu Community College Associate in Science Overall Course Pass Average: 72% PSY100 Course Pass Average: 47% 12

13 PAR Course Explorer Dashboard Honolulu Community College Associate in Science 13

14 PAR Student Explorer: Predictors Characteristics (15) Preparation ( 7 ) Entering Date (4) Remediation (6) Age Gender Race Pell eligible Enter with a GED Enter with prior credits High School GPA Level Quarter start DevEd attempted Degree Sought (6) Education Level (4) Academic Performance (14) Course Taking Online/Offline (2) Level of degree sought Non-degree student STEM major Predictors 58 total Education level Ongoing GPA Progress toward degree Transfer credits brought in after entry Credit ratio Number of Course withdrawals Fully onground student (Online Ratio=0) Fully online student (Online Ratio=1)

15 PAR Student Watch List Honolulu Community College Associate in Science Selected Students 15

16 Concerns Expressed at Annual Advisors Conference, October 2015 Inaccuracies due to limited data; non-academic factors, e.g., hours worked, outside responsibilities) that may affect risk are not included How was the score developed and how do we know that the score predicts retention? Who has access to the score? Potential negative impact (labeling or stereo-typing) due to seeing the score If we use it for automatic interventions, how will this impact student services? 16

17 How do we address or mitigate these concerns: as expressed at Annual Advisors Conference, October 2015 Use score as one of several factors, as a tool and not the whole tool box Keep the scores confidential from students Set procedures/checklists of required interventions based on the scores Provide clear explanation on how score is developed Promote buy-in at all levels, and provide training and resources Communication, transparency, and accountability 17

18 IPASS Project to Link PAR Data into Starfish: Honolulu Community College Vision: By being Student Centered, Student Focussed, HCC recognizes that student success and retention is the responsibility of every faculty, staff, and administrator. Students will be proactively empowered to successfully complete their academic journey through intentional connections with faculty, student services and academic support programs. These intentional connections will be supported through the robust use of integrated technology solutions. 18

19 IPASS Project to Link PAR Data into Starfish: Honolulu Community College Specific tasks: 1.Identify at risk students and create success plans for them 2.Focus on students that we can help 3.Create the workflow needed to provide and assess the interventions 4.Change the culture of advising 19

20 Honolulu Community College IPASS Team Plan Step 1 Step 2 Step 3 Step 4 Setting a shared vision: completion and success is everyone s responsibility Considering workflow: does every high risk student get contacted? Designing the display based on campus input: Simple display of red, yellow, green Rollout: It s not about the tool, focus on the student 20

21 PAR Retention Score Level 40% 5,629 5, % 30% 25% 20% 3, % 38.0% 37.7% 36.5% % 10% 5% 0% 20.1% 20.6% % 5.2% UHCC Honolulu CC Retention Level 1 Retention Level 2 Retention Level 3 Retention Level 4 21

22 100% Actual Spring-to-Fall Retention Rate for Estimated PAR Retention Score Level 90% 80% 70% 60% 50% 40% 30% 87.8% 81.4% 83.4% 62.0% 64.2% 90.6% 20% 10% 22.4% 24.5% 0% UHCC Honolulu CC 22 Retention Level 1 Retention Level 2 Retention Level 3 Retention Level 4

23 Home Campus: Honolulu Community College Program: HON-Natural Sciences Pre-Major: Pre-Medicine PAR Level 1 PAR Factors: #1 Associates student, #2 Enter with no prior credits, #3 Low cred... Registered for: 15 Credits at any UH institution Fall 2016 Registered for: 13 Credits at any UH institution Spring 2016 Registered for: 12 Credits at any UH institution Fall 2015 Semester Entered: Fall 2014 Applications: Applicant Accepted at Honolulu, Accepted at Leeward COMPASS Math: 28 1 COMPASS Reading: 27 High School: Central High School 5/2014 Career Interest: Health Science (medicine, dentistry, pharmacy, nursing, physical t. Immediate Ed Goal: Take courses to transfer to another college Highest Ed Goal: Earn a Medical Degree Highest Ed Goal Institution: University of Hawaii Manoa ORG_MEMBERSHIP: HON-ALL-STUDENTS-FA2015, HON-FINANCIALAID-FA2015

24 Working with campus teams to implement Step 1 Step 2 Step 3 Step 4 Setting a shared vision: completion and success is everyone s responsibility Considering workflow: does every high risk student get contacted? Designing the display based on campus input Rollout: It s not about the tool, focus on the student Status: Phase 1 complete Status: Completed Status: Finalizing Status: August, 2016 planned 24

25 Improvement via Interventions Interventions 25

26 Predictive Analytics What s the goal? 26