Predictive Analytics: The Postsecondary Use Case

Transcription

1 Predictive Analytics: The Postsecondary Use Case The Association Conference August 2, 2013 Heidi Hiemstra, Ph.D. Associate Director, Research

2 What is Predictive Analytics? Technology that learns from experience (data) to predict the future behavior of individuals in order to drive better decisions. - Eric Seigel, Predictive Analytics: The power to predict who will click, buy, like or die (2013: John Wiley and Sons).

3 Machine Learning Data Warehouses and the Cloud make it possible to collect, manage and maintain massive numbers of records. Sophisticated technology platforms provide computing power necessary for grinding through calculations and turning the mass of numbers into meaningful patterns. Data mining uses descriptive and inferential statistics moving averages, correlations, and regressions, graph analysis, market basket analysis to look inside those patterns for actionable information. Predictive techniques, such as neural networks and decision trees, help anticipate behavior and events.

4 Predicting Behavior: Observation and modeling associations

5 Predicting Behavior: Estimating Individual Risk 62% 8% 43% 87%

6 Predictive Analytics is not Social Science Contribute back to generalizable knowledge Start with generalizable knowledge (theory driven, seeking causation) Prove/ disprove hypothesis Social Science Propose hypothesis Collect and analyze data Develop method to test the hypothesis

7 Predictive Analytics is not Social Science Treat the individual differently based on risk score Start by defining a business problem Assign a risk score to individuals Predictive Analytics Gather related data (structured/ unstructured) Apply predictive associations from model to current individuals Develop a model that predicts historical behavior

8 Making Better Decisions Treat the individual differently based on risk score Would this student do better in developmental mathematics or in a supplemented college-level class? Will this student succeed in our nursing program? Might this student need a referral for co-curricular support?

9 Decision Support, not Automation

10 Predictive Analytics Reporting (PAR) Framework Funded by Bill & Melinda Gates Foundation in 2011, 2012 Managed by WICHE Cooperative for Educational Technologies, operated by WCET core project team 16 institutional partners 7 4-year schools 5 community colleges 4 for-profit institutions 12.5M course level records 1.7M student level records In-kind donations to date Blackboard idata Starfish

11 Institutional Partners American Public University System* Ashford University Broward College Capella University Colorado Community College System* Lone Star College System Penn State World Campus Rio Salado College* Sinclair Community College Troy University University of Central Florida University of Hawaii System* University of Illinois Springfield* University of Maryland University College University of Phoenix* Western Governors University * Original 6 partners

12 Common data definitions = reusable predictive models and meaningful comparisons. Openly published via a cc book.com/public/institu tions/par Structured Data

13 Data Inputs Student Demographics & Descriptive Gender Race Prior Credits Perm Res Zip Code HS Information Transfer GPA Student Type Student Course Information Course Location Subject Course Number Section Start/End Dates Initial/Final Grade Delivery Mode Instructor Status Course Credit Student Financial Information FAFSA on File Date Pell Received/Awarded Date Student Academic Progress Curent Major/CIP Earned Credential/CIP Course Catalog Subject Course Number Subject Long Course Title Course Description Credit Range Lookup Tables Credential Types Offered Course Enrollment Periods Student Types Instructor Status Delivery Modes Grade Codes Institution Characteristics Possible Additional ** Placement Tests NSC Information SES Information Satisfaction Surveys College Readiness Surveys Intervention Measures ** Future

14 Actionable Predictive Models Gateway Course Demonstration

15 Now What?

16 Student Intervention Exercise List five student support programs or policies at your institution (2 min): Timer Bar

17 PAR Student Success Matrix (SSM x ) Literature-based tool for benchmarking student services and interventions

18 Predictor Categories

19 Academic Cycle Phases Academic Career Connection: application to enrollment Entry: first weeks/ course/semester Progress: continuation toward ed. obj. Completion: achieve educational objective Course Connection: advising to enrollment Entry: first days/weeks, withdrawal Progress: midterm and beyond Completion: Pass/excell

20 Intervention Focus General vs. Targeted

21 Iwannabegreat Community College SSM x Completion

22 Student Intervention Exercise At your table, work as a team to place all of the interventions previously listed into the large Student Success Matrix provided (10 min). If the same intervention was listed by multiple people, negotiate a combined response. Only list an intervention once per row. Use arrows to draw across the academic cycles if needed. If the program is for all students, label it G; if it is for a targeted group of students, label it T.

23 Food for thought Looking at the distribution of interventions across rows and columns? Where are programs concentrated? Where are there gaps? How well does this reflect the supports you believe your students need to succeed? Are you measuring the effectiveness of any of these programs?

24 SSM x Early Results 15 institutions 659 interventions contributed across predictor categories 554 unique within institution Actual unique TBD 1,307 entries in cells Released July 15, 2013 Over 1,000 downloads to date

25 Quantification of Intervention Effectiveness Quantified intervention effectiveness results

26 Use of Aggregated Data Benchmark dashboards currently in development Research on use of predictive analytics in postsecondary ed. Research on the effectiveness of interventions Generalizable knowledge for theory and practice

27 Predictive Analytics: The Postsecondary Use Case The Association Conference August 2, 2013 Heidi Hiemstra, Ph.D. Associate Director, Research