Predictive Analytics Reporting (PAR) Framework Ellen Wagner WICHE Cooperative for Educational Technologies
The Predictive Analytics Reporting (PAR) Framework A big data analysis effort iden*fy drivers related to loss and momentum and to inform student loss preven*on WCET member ins*tu*ons voluntarily contribute de- iden6fied student records to create a single federated database which is then analysed using descrip*ve, inferen*al and predic*ve methods.
PAR Objectives Iden*fy common variables likely to influence student reten*on and progression, and measure the degree of influence Determine if measures and defini*ons of reten*on, progression, and comple*on differ materially among various types of postsecondary ins*tu*ons Begin to iden6fy the highest impact interven6ons with targeted pools of at risk- students
Costs and Completion Rates Gradua6on rates at 150% of 6me 70 60 2- yr colleges 4- yr colleges 50 40 30 20 10 0 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 Cohort year Source: New York Times; NCES
Performance Based Funding htp://www.ncsl.org/issues- research/educ/performance- funding.aspx
Are You Data-Ready? htp://www.whitehouse.gov/issues/educa*on/higher- educa*on/college- score- card
PAR Framework background Funded by Bill & Melinda Gates Founda*on in 2011, 2012, 2013 Managed by WICHE Coopera*ve for Educa*onal Technologies, operated by WCET core project team 16 grant- funded ins*tu*onal partners 7 4- year schools 5 community colleges 4 for- profit ins*tu*ons New self- funding members Grant extensions for new work In- kind dona*ons to date Blackboard idata Starfish
PAR FRAMEWORK GRANT PHASES 1. Iden6fy poten6al for mul6- ins6tu6onal data mining 2 Expand sample, variables and measurement period 3 Deliver models, frameworks, benchmarks and alerts to individual campuses 4. Perform field tests to analyze interven6ons across ins)tu)ons 5. Scale and self- sufficiency 8
Institutional Partners Founding Partners (since 2011): American Public University System* Colorado Community College System* Rio Salado College* University of Hawaii System* University of Illinois Springfield* University of Phoenix* Implementa6on Partners (since 2012): Ashford University Broward College Capella University Lone Star College System Penn State World Campus Sinclair Community College Troy University University of Central Florida University of Maryland University College Western Governors University New Members (as of Oct 2013): Northern Arizona University Kaplan University Excelsior College University of North Dakota
DATA STATISTICS Total Counts Time Frame August 2009 May 2013 13,090,351 course records 1,842,917 student records
The PAR Framework Value Proposition ACT PREDICT Reusable predic*ve models Student level watch lists for targeted interven*ons Measurable results RESULTS Common Defini*ons of terms Common Defini*ons of interven*ons Mul*- Ins*tu*onal collabora*on Scalable cross- ins6tu6onal improvements
Structured, Readily Available Data Common data defini*ons = reusable predic*ve models and meaningful comparisons. Openly published via a cc license @ htps:// public.datacookbook.co m/public/ins*tu*ons/ par
PAR Data Inputs Student Demographics & Descrip*ve Gender Race Prior Credits Perm Res Zip Code HS Informa*on Transfer GPA Student Type Student Course Informa*on Course Loca*on Subject Course Number Sec*on Start/End Dates Ini*al/Final Grade Delivery Mode Instructor Status Course Credit Student Financial Informa*on FAFSA on File Date Pell Received/Awarded Date Student Academic Progress Curent Major/CIP Earned Creden*al/CIP Course Catalog Subject Course Number Subject Long Course Title Course Descrip*on Credit Range Lookup Tables Creden*al Types Offered Course Enrollment Periods Student Types Instructor Status Delivery Modes Grade Codes Ins*tu*on Characteris*cs Possible Addi*onal ** Placement Tests NSC Informa*on SES Informa*on Sa*sfac*on Surveys College Readiness Surveys Interven*on Measures ** Future
Some of PAR s Outputs Validated Mul*- Ins*tu*onal Dataset Reflec*ve Ins*tu*onal Reports Cross- Ins*tu*onal Benchmarks Aggregate Models Ins*tu*onal Models Student Watch List Policy Local Interven*on Compara*ve Interven*ons
Actionable Predictive Models
Sample benchmark framework
Once we know what we are talking about its easier to deal with the Now, What? problem.
PAR Student Success Matrix (SSMx) Literature- based tool for benchmarking student services and interven*ons 600+ total interventions submitted Ability to compare among all 16 PAR institutions Basis for institutional intervention field tests Publically available, over 1,000 downloads since June 2013 htps://par.datacookbook.com/public/ins*tu*ons/par
Testing Intervention Effectiveness 4 Year Ins6tu6ons: Does Peer Mentoring (social integra*on focus) improve success as measured by % of students earning a C or beter? Community Colleges: Do the same or similar developmental educa*on approaches offered at mul*ple ins*tu*ons yield the same or similar results at all each ins*tu*on?
Without common diagnoses, it is difficult to agree on treatments to address diagnosed problems. Without agreed- upon treatments, it is difficult to measure the efficacy of treatments. If one cannon measure efficacy it is impossible to scale. Perhaps worse of all, we con*nue to guess about what really works in student success.
Delivering on the Promise of Data Helping Students Succeed. Scaling Student Success
Fin