PESC Spring 2015 Summit: Community Best Practices Defining Outcomes Eduventures, Inc.

Transcription

1 PESC Spring 2015 Summit: Community Best Practices Defining Outcomes

2 Agenda Defining Outcomes Data Collection and Sharing Trends Data Sources and Methods Partners and Trusted Sources 2

3 Defining Outcomes Outcomes v. Outputs

4 Defining Outcomes Stakeholders Prospects Learners and alumni Employers Regulators Accreditors Professional associations and networks Investors 4

5 Defining Outcomes What data should institutions be collecting? Academic Metrics Graduation Rate Time to graduate Time to employment Retention rates Students services received Career advising services (resume, placement, career fairs) Workforce Metrics Employer Career classification (SOC code) Salary/ Time in Job Currently unemployed? Geography Unique IDs / Debt load (absolute amount and as % of income) 5

6 Defining Outcomes Outcomes Data Collection Maturity Less Mature More Mature Some repeatable processes Defined and proactive procedures Explicit collection and reporting objectives Some feedback and coordination among business units Systematic Aligned Processes designed for continuous improvement Knowledge sharing and communication among units Alignment with strategic goals Awareness of the value of outcomes data becomes more distinct Embedded improvement processes supported through continuous monitoring Efficiencies identified and shared among units, networks and exchange partners Collaboration and innovation in support of strategic goals Strategic and operational metrics validate system success to external constituents Integrated

7 Data Collection and Sharing Trends The Catchment Area Model Identifying major feeders and destinations to maximize data sharing ROI Traditional model follows students from High Schools Community Colleges Employers What is your institutional equivalent opportunity? 7

8 Data Collection and Sharing Trends The Sharing Economy Market of transparency and accountability Smart business in the education economy requires data sharing - You must give into the system to get back what you need Be prepared to share your data on student enrollment with data stewards of outcomes data. In turn, partners will be more likely to share available data in return through approved systems, interfaces and agreements SLDS PK12 HE W EMP APIs NSC 8

9 Data Collection and Sharing Trends Share more, Not less True partnerships will be built upon a foundation of sharing for research purposes, fully in accordance with FERPA (revised regulations, December 2011): States have been working to establish longitudinal data systems to help answer these questions, and to determine what works and what doesn t when it comes to our children s education. The FERPA regulations amend certain provisions to help facilitate effective research and evaluation of Federal- and State-supported education programs through the use of State longitudinal data systems. The new regulations clarify who may receive student information to conduct evaluations of education programs, and under what circumstances these types of disclosures may occur. The regulations also provide best practices for written agreements that cover the circumstances of these disclosures, as well as best practices for reasonable methods an entity disclosing student information must take to ensure to the greatest extent practicable that those conducting an evaluation using student information protect privacy and comply with FERPA. From: 9

10 Data Collection and Sharing Trends What you have to offer 10

11 Data Collection and Sharing Trends What you could receive in kind 11

12 Data Collection and Sharing Trends In Pursuit of a Master Person Index Commonly used in healthcare: Enterprise Master Patient Index The objective is to ensure that each patient is represented only once across all the software systems used within the organization. To SSN or not SSN Have you adopted State unique IDs? If so, how many? Workforce (Unemployment) uses SSN K12 is SSN-averse Matching Services provided by 3 rd parties and APIs attached to data warehouses and SLDS The match engine must be configured and tuned for each implementation to minimize the false matches and non-matches. Based upon which sources have which data elements available 12

13 Data Sources and Methods Data Sources, Formats and Standards National Student Clearinghouse (NSC) [NSLDS & IPEDS] Employer Networks, Trade Associations and Professional Organizations Establish data sharing agreements with strategic, large, national-scale employers Alumni / Graduate Surveys Optionally tie completion of a survey to the distribution of transcripts, credentials or certificates Obtain waivers at the same time as other FERPA agreements Common Education Data Standards (CEDS) Ensure data elements conform to standards to maximize potential for matching and taxonomy/glossary consistency P20 Education Standards Council (PESC) Participate in Common Identity and Trust (CommIT) and other student-centric data sharing and exchange projects APIs (LinkedIn, Facebook, others) Single source of data, direct from alumni, most likely to be up to date 13

14 Data Sources and Methods When do you need the data? What is your time horizon? Segment elements by immediate, short and long term needs Group elements by sources and availability (available, requires effort, not yet available) When do data sources become available? Time from first outreach of a data owner to exchange of records could take 6-12 months Understand the timetable for each source and create a data publication calendar Don t let method of transfer become a barrier to acquisition SFTP vs. Web Services How will the data be used? Marketing using outcomes data effectively is more than just talking about outputs Accountability Gainful Employment Regulatory Compliance Institutional Research 14

15 Partners and Trusted Sources Student Longitudinal Data Systems In 2012 The DOE issued 24 state-level grants for the design and implementation of state longitudinal data systems. Of the 24 grants, 14 were awarded to states pursuing linking K12 data with postsecondary and workforce data. Previous rounds of grants were awarded in 2006, 2007, and twice in # of Grants Priority Description States (Grant Award) 9 Priority 1 The design, development, and implementation of a statewide, longitudinal kindergarten through grade 12 (K-12) data system; 1 Priority 2 The development and linking of early childhood data with the State s K-12 data system 14 Priority 3 The development and linking of postsecondary and/or workforce data with the State s K-12 data system Total Awarded: $41.4 M Arizona ($5M); Delaware ($4.6M); Nebraska ($4.4M); New Hampshire ($5M); Oklahoma ($5M); Puerto Rico ($4.7M); South Dakota ($3M); Vermont ($4.9); West Virginia ($4.8M) The Virgin Islands (2.6M) Total Awarded: $53.3 M Alaska ($4M); District of Columbia ($4M); Hawaii (3.4M); Idaho ($3.1M); Indiana ($4M); Iowa ($3.7M); Kentucky ($3.6M); Maryland ($4M); Montana ($4M); Nevada ($4M); New Jersey ($4M); North Carolina ($3.6M); North Dakota ($4M); Rhode Island ($4M) 15

16 Partners and Trusted Sources Student Longitudinal Data Systems 47 states, the District of Columbia, Puerto Rico, and Virgin Islands have received a SLDS Grant from the DOE. 16

17 Partners and Trusted Sources Data Quality Campaign Best Practices 1. Link Data Systems Ten State Actions 2. Create stable, sustained support 3. Develop governance structures 4. Build state data repositories 5. Provide timely, role-based access to data 6. Create progress reports with student-level data for educators, students, and parents 7. Create reports with longitudinal statistics to guide system-level change. 8. Develop a purposeful research agenda and collaborate with universities, researchers, or intermediary groups to explore the data for useful information. 9. Implement policies and promote practices, including professional development and credentialing, to ensure educators know how to access and use data appropriately. 10. Promote strategies to raise awareness of available data Ten Essential Elements 1. Statewide student identifier 2. Student-level enrollment, demographic, and program participation information 3. The ability to match individual students' test records from year to year 4. Information on untested students and the reasons they were not tested. 5. A teacher identifier system with the ability to match teachers to students by classroom and subject. 6. Collect student-level transcript information, including information on courses completed and grades earned. 7. Student-level college readiness test scores 8. Student-level graduation and dropout data. 9. The ability to match student records between the P 12 and higher education systems. 10. A state data audit system that assesses data quality, validity, and reliability. 17

18 Partners and Trusted Sources Data Quality Campaign - 43 states link K12 and postsecondary data while just 19 states link K12 with workforce data (shaded in green). 18

19 Partners and Trusted Sources Workforce Data Quality Initiative (BLS initiative) 19

20 Partners and Trusted Sources Maximizing Other Data Sources National Student Clearinghouse: Gainful Employment Reporting Service Free to institutions Simplifies GE compliance reporting Offers the flexibility to review your reports before the final ED submission FERPA compliant Employer Partnerships Bi-lateral data sharing agreements with your major employers of graduates Define which data is exchanged in each direction, per partner Leverage the competitive nature of larger employers Feature data-connected employer partners on your 20

21 Partners and Trusted Sources LinkedIn University Pages 21

22 Partners and Trusted Sources LinkedIn Alumni Tool 22

23 More information on Defining Educational Outcomes is available at: 23