Assessing Course Effectiveness Learning Analytics. Jere Turner,PhD Director of Institutional Research Manchester Community College

Transcription

1 Assessing Course Effectiveness Learning Analytics Jere Turner,PhD Director of Institutional Research Manchester Community College

2 Background MCC is one of seven CC in NH Data warehouse built in 2004 DI software Veera adopted in 2008 Learning Analytics grant 2012 Noel-Levitz College Student Inventory CSI incorporated into the College Success Seminar fall 2013

3 Research Questions 1. What are our students academic goals and how prepared are they to meet their goals? 2. How can analytics techniques be used to create a predictive model to determine student success?

4 Definitions Student Success: student transfers to another college or graduates from MCC with associate degree or certificate. Learning Analytics: measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs. (SoLAR) The Society for Learning Analytics Research (SoLAR) is an inter-disciplinary network of leading international researchers who are exploring the role and impact of analytics on teaching, learning, training and development.

5 Methodology Banner Oracle Database Data Warehouse DI Software Data Sources Veera National Student Clearinghouse Noel-Levitz Survey Accuplacer placement Analytics Blackboard Digital Data Action Plan?

6 Retention Predictors (Literature) Achievement High school GPA* SAT/ACT* First Semester/year GPA Gateway courses Developmental courses* Demographic Gender* Race/ethnicity Household income First generation college Academic major* Predicting College Student Retention (2011) Hanover Research, Washington, DC. Financial Loan aid Work study aid Student need Social/Situational Transfer student Commuter Psychological Personality traits Locus of control Self-esteem/self-efficacy Student readiness inventory * Predictive Modeling to Identify High Risk Students Taylor & McAleese, Paul Smith s College

7 Variables considered in the MCC Model Achievement First semester GPA Persistence Accuplacer test scores Demographic Age Gender Race/ethnicity Enrollment status Financial Pell grant Financial aid Noel-Levitz Educational goals Parents education Family support Work plans

8 College Student Inventory The Noel-Levitz College Student Inventory (CSI) Form B was administered to 1,257 MCC students between September, 2007 and January, The inventory is designed especially for incoming firstyear students and is comprised of 100 items subsumed in 17 different scales. These scales are organized under three main categories: 1. Academic Motivation 2. General Coping Skills 3. Receptivity to Support Services

9 Frequency Frequency Sample 1257 incoming freshmen students fall 2007 to fall 2009 AGE Years First Semester GPA GPA

10 College Student Inventory Dropout Proneness The MCC sample is above average (69%) on the DP scale (the norm is 50%). Twenty-six percent of the students have a high inclination (stanines 8 and 9) to drop out of school before finishing a degree. 139 (43%) of the students were successful. Predicted Academic Difficulty MCC students have a higher propensity to have a predicted academic difficulty (63% above the mean). The norm for this scale is 50%. Nineteen percent of the students were stanine 8 & 9, of those 233 students 94 (40%) were successful.

11 Relationship found between 'success' and 'Desire to Transfer' College Student Inventory Relationship found between 'success' and 'Family Emotional Support'

12 College Student Inventory Relationship found between 'success' and 'Highest degree sought' Relationship found between 'success' and 'Plans to work during school year'

13 Data Integration with Veera

14 Analysis MODEL 3: Logistic Regression Model Maximum Likelihood Estimates Variable Coef S.E. Wald chi-sqr p-value Intercept persistence GPA Cube(Desire to Transfer) Cube(Family Emotional Support) Binary(FULL_PART_IND,F) LOGe(Highest degree sought) Binary(Plans to work 40+ hrs/wk) SquareRoot(TEST_SCORE) Decile Analysis on Model Diagnostics -2 Log L AIC SC N Intercept only Intercept and Covariates Association of Predicted Probabilities and Actual Responses Percent Concordant 76.37% Somers' D Percent Discordant 23.51% G-K Gamma 0.529

15 Next Steps Students Prob_quartile 1st 2nd 3rd 4th Sum Successful Unsuccessful # Student # Student % % % % 100 % % % % % 100 % Determining intervention threshold Determining interventions Determining intervention results Creating a model for short-term goals (persistence to second semester)

16 Conclusion 1. The Noel-Levitz College Success Inventory provides a proactive approach to a comprehensive student retention plan designed to enable the college to: a) Assess students needs and strengths b) Identify students who may be at risk of leaving college c) Recognize students motivation patterns, such as receptivity to assistance 2. Data Analytics techniques have the ability to create predictive models that can assist higher education administers with action plans that will support student success. 3. Additional student variables are needed to strengthen the predictive model.

17 Thank you!