Summer 2012 American Institutes for Research

Transcription

1 Summer 2012 American Institutes for Research 1

2 Overview of State LDS Essential Questions and Data Elements Gap Analysis Models of State LDS State Presentation Force Field Analysis Questions from Day One? 2

3 Using Longitudinal Data to Empower Your Research Addressing Common Challenges Planning Logistics State Presentation Communicating Your State LDS Efforts Getting Support 3

4 4

5 Administrators can understand students performance better and manage more effectively and efficiently; Teachers can tailor instruction to help each student improve; and Policymakers can evaluate which policy initiatives show the best evidence of increasing student achievement. 5

6 To improve the effectiveness of adult education programs, we need to monitor student academic progress over time. What kind of data set will we need? Student longitudinal data set 6

7 Student Name Initial (2005) Raw Score 2006 Raw Score 2007 Raw Score 2008 Raw Score 2009 Raw Score Jane David Kate Matt

8 8

9 Estimate the overall effectiveness of individual teachers. What kind of data set will we need? Teacher longitudinal data set Student longitudinal data set 9

10 Test Score Test Score Year Student Student Student Teacher Teacher Teacher Teacher ID Student Name Pretest (2005) Posttest (2006) Gender Age Race Gender Age Experience A Jane Female 23 White Female B David Male 34 Black Male 30 2 A Kate Female 23 White Female B Matt Male 34 Black Male

11 Teacher A is more effective in terms of improving student test scores. 11

12 Evaluate the effectiveness of a specific policy What kind of data set will we need? For policies to improve student achievement: Preand Post-intervention student longitudinal data. For policies to improve teacher performance: Preand Post-intervention teacher and student longitudinal data. 12

13 Year student test scores Program Pre-intervention Pre-intervention Pre-intervention Pre-intervention Pre-intervention Pre-intervention Post-intervention Post-intervention Post-intervention Post-intervention Post-intervention 13

14 14

15 Another advantage of building a longitudinal data system is to follow students across sectors. K-12 Adult Education Labor Market To conduct an evaluation study of the effectiveness of Career Counseling services in adult education. We need to follow students after they exit AE and enter employment 15

16 Student ID 2008 Employment Status 2009 Employment Status Career-counseling participation in employed employed YES 2 employed employed YES 3 unemployed employed YES 4 unemployed employed YES 5 unemployed employed YES 6 employed employed YES 7 employed employed YES 8 unemployed employed YES 9 unemployed employed YES 10 unemployed employed YES 11 employed employed NO 12 unemployed unemployed NO 13 unemployed unemployed NO 14 employed unemployed NO 15 unemployed unemployed NO 16 unemployed unemployed NO 17 employed employed NO 18 unemployed unemployed NO 19 unemployed unemployed NO 20 employed unemployed NO 21 unemployed unemployed NO 22 unemployed unemployed NO 16

17 17

18 How would this information help your programs? What questions do you have about research and your state LDS? Take five minutes to write down some potential questions that your state may want to answer with LDS data. 18

19 19

20 20

21 Case Studies Lack of common IDs and privacy issues Sustainability Poor interagency coordination Limited resources See Handout 5: Case Studies 21

22 Share Case Studies Share the challenges and solutions discussed in your case studies Do any of these apply to challenges identified in your planner? State Challenges What challenges did you identify in your planner? What process will your team go through to identify and choose solutions to these challenges? Begin your process 22

23 23

24 What is our end goal? How will data be collected? What will AE do with the longitudinal data? What is needed to make each of these things happen? Training? Resources? Support and collaboration? Budget? 24

25 If development of your State LDS is further along: What changes would you make to improve the LDS for use by adult education? How will you focus on research pursuits? What are the next steps with your data? 25

26 What are our priorities for 2012? What can be done by 2013? What are our next steps? 26

27 What is our end goal? What is our timeline? Who will be responsible? What is the biggest challenge that we are facing? What strategies will we use to address that challenge? 27

28 28

29 29

30 What is our end goal? What is our timeline? Who will be responsible? What is the biggest challenge that we are facing? What strategies can we use to address that challenge? 30

31 31

32 32

33 33

34 How to make connections with the state LDS leaders active in your state, or to develop a state LDS if nothing else exists Possibly how to collaborate across states to leverage funding/interest/etc. regionally Whom to partner with What s in it for them What does adult education have to contribute to the planning and conversation 34

35 Creating a communication plan will help you to: Identify your potential partners and other stakeholders or influencers; Determine your objectives, depending on how far along the state is in development of an LDS; Identify the information that will inform your message; Select the person who can take the message forward; and Determine how to maintain the collaboration into the future. 35

36 Who will I partner with in my state LDS? Which stakeholders will advocate for adult education to join the LDS system? Is there a state LDS? If so, which agency is leading the effort? Who else is involved? If not, what agencies would you like to collaborate with? Is there more opportunity with K-12, postsecondary, or workforce? 36

37 Other potential stakeholders Local program directors, especially those in politically important areas of the state or other politically influential advocates for adult education. The Data Quality Campaign (DQC) focuses on K-12, but IES requires that adult education be involved. Make your interests known to the DQC at the national level. 37

38 What is your goal for collaborating with other agencies? These objectives should include: Helping other agencies to understand your value Building relationships with these agencies to further your mutual interests 38

39 What do I want to say to those potential partners? How will I frame my message? What information do my stakeholders need to ensure that adult education is engaged? What data will help potential partners to understand my adult education program? What message do I want to send? Identify what s in it for you and what s in it for them Identify key data to support that message 39

40 Objective 1: Join the collaboration Key Messages: Adult education is capturing learners as they exit the P-20 pipeline We have data on your learners and we need data about our shared learners Supporting Information: Thirty-five percent of our learners are and most come within 1-2 years after dropping out of K-12 Data we need: What courses have they completed in HS? What are their most recent test scores in English, Math? 40

41 Objective 2: Create long-term relationship with Agencies A, B, & C to collaborate on future initiatives Key Messages: Adult education has lots of data Adult education has experience with data use, data training Supporting Information: Our learners include: 35% dropping out of K-12; 56% looking to improve job skills; 37% set postsecondary as a goal Our program staff and teachers: 82% have received training on data use; 76% of programs have a report card for monitoring or stakeholders 41

42 Who in my office will be the representative for adult education in the LDS conversations at the state level? State Director? Data System Developer? Other? One person? Two people? Who will support the LDS lead? What kinds of support will this person need from the AE office? 42

43 How frequently will our staff meet with partners? What kinds of updates will we provide them to keep adult education in mind? 43

44 Identify your Partners/Stakeholders/Influencers (Audience) Objectives Key Messages that support your objectives and inform your audiences Supporting information for those messages (data from your MIS, other) Frequency of meetings & updates Representative/relationship-builder (who is responsible, may be more than one person taking on different roles) 44

45 If you are already engaged in your state LDS, develop a communication plan for locals What do locals already know about your LDS collaboration & how is it communicated? What are your plans to communicate this information to locals throughout the process? Locals will want to know: what an LDS is and why adult education would want to be part of that system, benefits to the adult education system, benefits to program staff, teachers, and students, and impact or any additional burden on them for data collection or reporting. 45

46 If you are already engaged in your state LDS consider where will your state LDS be in the future (next 3-5 years) What other agencies would you like to add or connect with? What additional data sets would you include? What changes would you make to the process? (e.g. host meetings at the adult education office) What change Benefits of this change and for whom Timeline for change Resource needed Who is responsible 46

47 47

48 U.S. Department of Education Data Quality Campaign Websites Publications Webinars Conferences On-site Technical Assistance Nonprofits and Foundations 48

49 State LDS Best Practice Conference Public Domain Clearinghouse (PDC) Best Practice Briefs NCES National Forum on Education Statistics Topical Working Groups Monthly Webinars Listserv and Correspondence NCES Personnel Exchange Network (PEN) State Support Team (SST) 49

50

51 Midwest Education Information Consortium (MEIC) MN, NE, IA, MO, ND, SD, KS Working to track students across state lines New England Secondary School Consortium (NESSC) CT, NH, ME, RI, VT Setting data standards and gathering metrics Western Interstate Commission for Higher Education (WICHE) WA, OR, ID, HI Multistate Longitudinal Data Exchange 51

52 IES Grants Workforce Data Quality Initiative (WDQI) Other Federal Funding Sources Title I Grants to Local Education Agencies (LEAs) Workforce Innovation Fund Workforce Investment Act Foundations 52

53 What kinds of support can the NRS project offer? What kinds of support can OVAE offer? 53

54 54

55 55