Redefining At-Risk Through Predictive Analytics: A Targeted Approach to Enhancing Student Success

Transcription

1 Redefining At-Risk Through Predictive Analytics: A Targeted Approach to Enhancing Student Success Amilcah Gomes Assistant Director, Academic Services Center Eastern Connecticut State University

2 Predictive Analytics Using real-time data to plan for the future Predictive analytics helps your organization predict with confidence what will happen next so that you can make smarter decisions and improve business outcomes. IBM

3 Institutions That Use Predictive Analytics Predictive Analysis Reporting (PAR) Framework Institutions American Public University System Ashford University Broward College Capella University Colorado Community College System Lone Star College System Penn State World Campus Rio Salado College Sinclair Community College Troy University University of Central Florida University of Hawaii System University of Illinois Springfield University of Maryland University College University of Phoenix Western Governors University Purdue University Rutgers University University of South Florida Eastern Connecticut State University Others

4 Eastern Connecticut State University Location: Willimantic, Connecticut Institution Type: Public, Liberal Arts University Total Enrollment (2012): 5,237 students, 4,506 undergraduate FTE Nearly 2/3 residential, with 93% of first-time students living on campus 2011 FTFT cohort (N = 931): Female 53.4%; Students of Color 22% Other Characteristics First Gen Students (32.5%), Compass Cohort (50.1%), STEP/CAP students (6.8%), Honors Scholars (2.4%), Athletes (9.7%) 2012 FTFT cohort (N = 979) 2013 FTFT cohort (N = 960)

5 Institutional Changes Strategic Plan: Student Success Initiative ( ) Dual Advising Model Project Compass ( ) Communities of Practice Early Identification of At-Risk Students Enhancing First-Year Advising Services Targeting tutoring services to high-risk subjects Title III Student Support Services ( ) Strategic Plan & Targeted Student Success Initiatives

6 Development of Prediction Models 2008 Multivariate model developed for predicting withdrawal prior to second year Original assumption: students withdraw due to poor academic performance Problem: Original model failed to differentiate between students who left for academic and nonacademic reasons Did not account for financial reasons, motivational variables, engagement factors, etc. Withdrawal risk quintiles were difficult to interpret Professional advisors were unable to develop advising strategies specific to a student s needs 2011 Developed additional multivariate model for academic risk; implemented two-model approach

7 Withdrawal Model, 2011 Cohort (2008, 2009 Data) Variable B S.E. Wald df Sig. Exp(B) Base male female black * white hisp oth_race not_east commuter Pell_yr first_gen Athletics HsGpa_quint HsGpa_quint HsGpa_quint HsGpa_quint admit_rating_le_ admit_rating_ge_ Vsat_quin Vsat_quin Vsat_quin Vsat_quin Stem PreEd Undec ERG_none ERG_ABC ERG_GHI East CT River Campus Not Pell Not FGEN Not Athlete Quintile 3 Rate 5, 6, or 7 VSAT Quint 3 Not STEM Not PreED Declared major ERG DEF got_schol_yr No schol got_fedloan_yr No Fed Loan Choice Not #1/No FAFSA Constant *Factors that are significant at the 0.10 level have been highlighted.

8 Academic Risk Model, 2011 Cohort (2008, 2009 Data) Model 2: for classifying 2011 Cohort into Academic-Risk Quintiles, Based on Data from 2008 and 2009 Cohorts. B S.E. Wald df Sig. Exp(B) HsGpa ERG_DEF ERG_GHI ERG_none black hisp oth_race Pell_yr first_gen male Stem PreEd Athletics admit_rating_le_ admit_rating_ge_ Constant

9 Interpreting the Models: What We Saw Factors that had a significant impact (0.10 level): Factors Significant in Both Models Commuter Not East Federal Loans FAFSA Choice Only Significant in Withdrawal Model HS GPA ERG/DRG (CT HS districts) Athlete (negative, both) African American identity (negative withdrawal, positive academic risk) Admissions Rating 8 (negative, both) Males STEM Majors Only Significant in Academic Risk Model

10 Interpreting the Models: What We Didn t See Factors that did not have a significant impact (0.10 level): Pell eligibility (less significant in GPA model) 1 st Generation Status Undeclared Majors (significant for 2 nd to 3 rd year retention) Multiracial Identities (added to 2012 model) Math SAT (good predictor of GPA > 3.0)

11 Classifying Withdrawal & Academic Risk Cross-classification of 2011 cohort of entering first-time, full time students by the two types of risk: withdrawal and low academic performance: Academic Risk Quintile Total Q_ Total TAC 1 = Intensive 172 TAC 2 = Tutoring 205 TAC 3 = Engaged 232 TAC 4 = Monitor 322

12 Targeted Advising Cohorts (TAC) TAC 1 = Intensive (high risk withdrawal & high risk GPA < 2.3) TAC 2 = Tutoring (low risk withdrawal & high risk GPA < 2.3) TAC 3 = Engaged (high risk withdrawal & low risk GPA < 2.3) TAC 4 = Monitor (low risk withdrawal & low risk GPA < 2.3)

13 Targeted Advising Cohorts (TAC) High academic risk TAC 2 TAC 1 Tutoring Intensive Low risk withdrawal High risk withdrawal High risk GPA < 2.3 High risk GPA < 2.3 Low withdrawal risk TAC 4 Monitor Low risk withdrawal TAC 3 Engaged High risk withdrawal High withdrawal risk Low risk GPA < 2.3 Low risk GPA < 2.3 Low academic risk

14 Fall 2011 Cohort TAC Assignments* By using the two-model approach it was determined that 18.5% of the FTFT students entering were at risk of withdrawal and low academic performance (TAC1), and another 25.5% were at risk of low academic performance (TAC2). 33.2% 18.5% TAC1=Intensive TAC2=Tutoring TAC3=Engaged TAC4=Monitor 25.5% * 35 students originally in TACs 3 and 4 who did not participate in the library assessment and orientation were reassigned to TAC %

15 Observations Gender Males made up less than half (46.6%) of 2011 cohort, but represented 67% of TAC 1 & 2 students Race/Ethnicity Students of color, particularly African American/Black students, were overrepresented in TAC 2 (46.3% vs. 23.3% overall) Home of Record 80.2% of TAC 1 students were Not East (international, out of state, and students living west of the CT river) students (56.9% overall) Other Characteristics 68.8% of Pell eligible and first generation students were classified in TACs 2 & 4

16 Summary of Outcomes FTFT 2011 General Characteristics Outcomes (Average) N % Female % 1st Gen % Students of Color # Credits Att. # Credits Earned 1st Yr. GPA Lib.Score Retention TAC TAC TAC TAC There is very little variation in credits attempted across the four TACs Students in TAC 1 and TAC 2 on average were slightly less likely to achieve the same level of academic momentum as their classmates in terms of credits earned In addition, the average GPAs for students in TAC 1 and TAC 2 are significantly lower than those in TAC 3 and TAC 4 TAC assignment is clearly related to retention TAC 3 and TAC 4 more likely to utilize tutoring services and devote more hours with math and writing tutors on average

17 Data-Informed Approach Reassess assumptions about student withdrawal patterns and ALANA student performance on campus Align institutional priorities to significantly enhance student success Better allocate limited staffing and financial resources to support high-impact practices Address persistence and performance issues among high-risk groups

18 Developing Targeted Interventions Freshman Preference Registration Developmental Advising Focus Leading Indicators Project (Library Orientation Score) Collaboration with Campus Departments Registration Holds and Financial Review Diversity Scholars Program Strategies for High-Performing Students Recognition of ALANA Student Success

19 Intervention: Major Course Selection TAC FTFT 2011 Cohort Mean CGPA First-Semester GPA for FTFT 2011 & 2012 cohorts by TAC N FTFT 2012 Cohort Major Course CGPA N FTT 2012 Cohort No Major Course CGPA A Total * N * 23 students did not complete the Fall 2011 semester and were not included in the first semester GPA analysis

20 Intervention: Major Course Selection TAC First-Semester Persistence for FTFT 2011 & 2012 cohorts by TAC FTFT 2011 Cohort Returned Spring 2012 FTFT 2012 Cohort Returned Spring 2013 FTFT 2012 Cohort Major Course Returned Spring 2013 FTFT 2012 Cohort No Major Course Returned Spring % 90.0% 93.0% 84.9% % 93.0% 95.2% 90.7% 2A 82.9% 93.3% 94.4% 75.0% % 94.0% 95.0% 91.2% % 95.0% 95.3% 94.9% Total 90.9% 93.3% 94.7% 90.7%

21 Intervention: Major Course Selection Year 1 Persistence to Year 2 Year 1 Academic Performance Outcomes (Average) No Maj Cume No Maj ECO 200 Cume GPA grade in 1 st Cume GPA Cohort Overall Crse ECO 200 BUS 201 GPA Crse ECO 200 Semester BUS 201 BUS 201 grade in 1 st Semester FTFT % 72.5% 77.5% 82.4% (C+/B-) (B) FTFT % 70.0% 77.2% 80.9% (C+/B-) (B/B+) Year 2 Persistence to Year 3 Year 2 Academic Performance Outcomes (Average) No Maj Cume No Maj ECO 200 Cume GPA grade in 1 st Cume GPA BUS 201 grade in 1 st Cohort Overall Crse ECO 200 BUS 201 GPA Crse ECO 200 Semester BUS 201 Semester FTFT % 52.5% 65.0% 76.5% (C+/B-) (B) BUAD majors taking BUS 201 and ECO 200 during first semester via FPR Slight improvement in cumulative GPA for students in BUS 201 over ECO 200 Students consistently performed better in BUS 201, regardless of TAC, etc. Some students may be at a disadvantage with ECO 200, particularly African American, Hispanic/Latino, FGEN, PELL, and TAC 1 & TAC 2 students Intervention for both at-risk and high-performing students

22 Intervention: Registration Holds 36.8% of 2012 cohort had holds just prior to Spring 2013 advising and registration, mostly financial 44% of TAC 1 and 48% of TAC 2 students had holds 45.7% of TAC 2A students had holds 76.2% of STEP/CAP students and 51.1% of nonwhite, non-step/cap students had holds vs. nearly 30% of white students Average midterm GPA with holds: 2.53 (2.81 without holds) Collaboration with Enrollment Management, Fiscal Affairs, and Student Affairs Financial Review Day Intrusive Advising Strategies Individual Day Passes Average cumulative GPA with holds: 2.74 (3.07 without holds) TAC Returned Spring 2013 N Did Not Return Total Not Returned with Holds % (52.4%) % 12 9 (75.0%) 171 2A 93.3% 3 2 (66.6%) % 14 8 (57.1%) % 16 8 (50.0%) 320 Total 93.3% (57.6%) 979 N

23 Redefining Perceptions of At-Risk Shifting from deficitbased assumptions Cognitive deficit assumptions mischaracterization marginalization toward a successbased model! Targeted initiatives based on student needs relevant student support

24 Diversity Scholars Program Spring 2012 Pilot group for FTFT SOC, TAC 1-2, < 2.0 GPA (N = 28) Fall 2012 services for all FTFT ALANA students (N = 133) Developmental advising services had positive impact on GPA for TAC 1, limited impact for TAC 2 when given alone ( high-relational groups) Peer mentoring: participants (2.79) vs. non-participants (2.47) Hispanic/Latino student participants (3.03) vs. non-participants (2.13) Academic & student support interventions (e.g., taking major course in first semester, financial review) FTFT students of color had 3-4 times higher increase in GPA from previous year than white students, except TAC 4 African American students Reduction in GPA gap experienced across all ethnicities Fall 2013 added targeted developmental advising outcomes and major-related opportunities/workshops (N = 138) TAC 1-2 DSP participants (2.94) vs. ALANA non-participants (2.34) TAC 1-2 Hispanic/Latino students outperformed white students during first semester (2.72 vs. 2.59) TAC 3-4 African American students still underperform (2.89 vs. 3.10)

25 Recommendations for First-Year Success Use multiple approaches to assist students, based on individual needs and types of risk Consider non-academic barriers when advising students (finances, registration holds, campus integration, family issues, etc.) Encourage use of early alert system (APN) and sharing of information across departments Connect students to academic departments and major-related opportunities, particularly high performing students

26 Where do we go from here? Explore other non-cognitive measures and/or instruments and administer during the fall semester Revisit second-year persistence models developed during Project Compass Develop advising strategies for second-year students Communicate data findings with more faculty Increase access to high-impact practices, particularly for underrepresented students

27 ANY QUESTIONS?