Predictive Analytics: The Postsecondary Use Case The Association Conference August 2, 2013 Heidi Hiemstra, Ph.D. Associate Director, Research Heidi.Hiemstra@parframework.org
What is Predictive Analytics? Technology that learns from experience (data) to predict the future behavior of individuals in order to drive better decisions. - Eric Seigel, Predictive Analytics: The power to predict who will click, buy, like or die (2013: John Wiley and Sons).
Machine Learning Data Warehouses and the Cloud make it possible to collect, manage and maintain massive numbers of records. Sophisticated technology platforms provide computing power necessary for grinding through calculations and turning the mass of numbers into meaningful patterns. Data mining uses descriptive and inferential statistics moving averages, correlations, and regressions, graph analysis, market basket analysis to look inside those patterns for actionable information. Predictive techniques, such as neural networks and decision trees, help anticipate behavior and events.
Predicting Behavior: Observation and modeling associations
Predicting Behavior: Estimating Individual Risk 62% 8% 43% 87%
Predictive Analytics is not Social Science Contribute back to generalizable knowledge Start with generalizable knowledge (theory driven, seeking causation) Prove/ disprove hypothesis Social Science Propose hypothesis Collect and analyze data Develop method to test the hypothesis
Predictive Analytics is not Social Science Treat the individual differently based on risk score Start by defining a business problem Assign a risk score to individuals Predictive Analytics Gather related data (structured/ unstructured) Apply predictive associations from model to current individuals Develop a model that predicts historical behavior
Making Better Decisions Treat the individual differently based on risk score Would this student do better in developmental mathematics or in a supplemented college-level class? Will this student succeed in our nursing program? Might this student need a referral for co-curricular support?
Decision Support, not Automation
Predictive Analytics Reporting (PAR) Framework Funded by Bill & Melinda Gates Foundation in 2011, 2012 Managed by WICHE Cooperative for Educational Technologies, operated by WCET core project team 16 institutional partners 7 4-year schools 5 community colleges 4 for-profit institutions 12.5M course level records 1.7M student level records In-kind donations to date Blackboard idata Starfish
Institutional Partners American Public University System* Ashford University Broward College Capella University Colorado Community College System* Lone Star College System Penn State World Campus Rio Salado College* Sinclair Community College Troy University University of Central Florida University of Hawaii System* University of Illinois Springfield* University of Maryland University College University of Phoenix* Western Governors University * Original 6 partners
Common data definitions = reusable predictive models and meaningful comparisons. Openly published via a cc license @ https://public.datacook book.com/public/institu tions/par Structured Data
Data Inputs Student Demographics & Descriptive Gender Race Prior Credits Perm Res Zip Code HS Information Transfer GPA Student Type Student Course Information Course Location Subject Course Number Section Start/End Dates Initial/Final Grade Delivery Mode Instructor Status Course Credit Student Financial Information FAFSA on File Date Pell Received/Awarded Date Student Academic Progress Curent Major/CIP Earned Credential/CIP Course Catalog Subject Course Number Subject Long Course Title Course Description Credit Range Lookup Tables Credential Types Offered Course Enrollment Periods Student Types Instructor Status Delivery Modes Grade Codes Institution Characteristics Possible Additional ** Placement Tests NSC Information SES Information Satisfaction Surveys College Readiness Surveys Intervention Measures ** Future
Actionable Predictive Models Gateway Course Demonstration
Now What?
Student Intervention Exercise List five student support programs or policies at your institution (2 min): 1. 2. 3. 4. 5. Timer Bar
PAR Student Success Matrix (SSM x ) Literature-based tool for benchmarking student services and interventions https://par.datacookbook.com/public/institutions/par
Predictor Categories
Academic Cycle Phases Academic Career Connection: application to enrollment Entry: first weeks/ course/semester Progress: continuation toward ed. obj. Completion: achieve educational objective Course Connection: advising to enrollment Entry: first days/weeks, withdrawal Progress: midterm and beyond Completion: Pass/excell
Intervention Focus General vs. Targeted
Iwannabegreat Community College SSM x Completion
Student Intervention Exercise At your table, work as a team to place all of the interventions previously listed into the large Student Success Matrix provided (10 min). If the same intervention was listed by multiple people, negotiate a combined response. Only list an intervention once per row. Use arrows to draw across the academic cycles if needed. If the program is for all students, label it G; if it is for a targeted group of students, label it T.
Food for thought Looking at the distribution of interventions across rows and columns? Where are programs concentrated? Where are there gaps? How well does this reflect the supports you believe your students need to succeed? Are you measuring the effectiveness of any of these programs?
SSM x Early Results 15 institutions 659 interventions contributed across predictor categories 554 unique within institution Actual unique TBD 1,307 entries in cells Released July 15, 2013 Over 1,000 downloads to date
Quantification of Intervention Effectiveness Quantified intervention effectiveness results
Use of Aggregated Data Benchmark dashboards currently in development Research on use of predictive analytics in postsecondary ed. Research on the effectiveness of interventions Generalizable knowledge for theory and practice
Predictive Analytics: The Postsecondary Use Case The Association Conference August 2, 2013 Heidi Hiemstra, Ph.D. Associate Director, Research Heidi.Hiemstra@parframework.org