Taking Stock of the California Linked Learning District Initiative

Transcription

1 SRI INTERNATIONAL Taking Stock of the California Linked Learning District Initiative Fifth-Year Evaluation Report DECEMBER 2014 PREPARED FOR THE JAMES IRVINE FOUNDATION

2 SRI INTERNATIONAL

3 Taking Stock of the California Linked Learning District Initiative Fifth-Year Evaluation Report December 2014 Prepared by: SRI International Center for Education Policy Suggested citation: Guha, R., Caspary, K., Stites, R., Padilla, C., Arshan, N., Park, C., Tse, V., Astudillo, S., Black, A., & Adelman, N. (2014). Taking stock of the California Linked Learning District Initiative. Fifth-year evaluation report. Menlo Park, CA: SRI International. 1

4 2

5 Contents Exhibits... iii Acknowledgments... iv Executive Summary... v Chapter 1: Introduction... 1 Chapter 2: Student Access and Equity... 9 Chapter 3: Curriculum, Instruction, and Assessment Chapter 4: Work-Based Learning Chapter 5: Perceptions of Skills Gained in Pathways Chapter 6: Student Engagement and Achievement Chapter 7: Student Postsecondary Plans and Supports Chapter 8: Sustaining Linked Learning Systems Chapter 9: Conclusions References Appendix: Research Methods... A-1 i

6 ii

7 Exhibits Exhibit 1-1 Demographic and Achievement Profile of Linked Learning Districts, Exhibit 1-2 Linked Learning Pathways Meeting Certification Criteria as of Exhibit 2-1 Certified Pathways by California Career Pathway Industry Sectors Exhibit 2-2 Student Subgroup Enrollment in Certified Pathways Exhibit 2-3 Female Enrollment in Certified Pathways by Career Theme Compared with District Average Exhibit 2-4 Retention to the 11th Grade in Certified Pathways Overall and by Student Subgroup Exhibit 2-5 Pathway Students in the Most Common Class for Their Pathway in Each Subject Exhibit 3-1 Students Reporting Feeling Challenged by Their Teachers Exhibit 3-2 Students Reporting At Least One Teacher Supporting Application of Classroom Learning Exhibit 3-3 Students Reporting Outside Application of Classroom Learning Exhibit 4-1 Pathway and Comparison Students Participating in Specific Work-Based Learning Activities in Exhibit 4-2 Students Reporting on Their Most Meaningful Work-Based Learning Experience Exhibit 5-1 Skills Needed for Postsecondary Success Exhibit 5-2 Students Reporting That High School Had Improved Their Communication and Collaboration Skills Exhibit 5-3 Students Reporting That High School Had Improved Their Sense of Self-Efficacy and Self-Management Skills Exhibit 6-1 Framework for How Linked Learning Affects Student Academic Achievement Exhibit 6-2 Pathway Students Were More Likely to Remain in the Same District Exhibit 6-3 Pathway Students Earned More Credits Exhibit 6-4 Pathway Students Made Greater Progress Toward a g Completion in 10th Grade Exhibit 6-5 Pathway Students Scored Higher on the ELA CAHSEE Exhibit 6-6 Credit Accumulation and ELA and Math Content Knowledge by Student Subgroup Exhibit 7-1 Framework for How Linked Learning Affects Student Transition to Postsecondary Experiences Exhibit 7-2 Students Reporting Opportunities to Explore Future Education Plans Exhibit 7-3 Students Reporting Extent of Adult Guidance for Postsecondary Planning Exhibit 7-4 Students Reporting Extent of Adult Support with College Application Process Exhibit 7-5 Students Reporting Taking College Entrance Tests Exhibit 7-6 Students Reporting Submitting College Applications iii

8 Acknowledgments Many individuals contributed to the completion of this report. We are indebted to the district- and schoollevel staff who took time out of their busy schedules to participate in this independent evaluation and assisted us with data collection. In particular, we thank the following for serving as our primary liaisons: Robin Schmitt and Maria Villanueva, Antioch Unified School District; Esther Soliman and Karin Kroener- Valdivia, Los Angeles Unified School District; Cynthia Bater, Long Beach Unified School District; Ayele Dodoo, Montebello Unified School District; Gretchen Livesey and Susan Benz, Oakland Unified School District; Marisa Sarian, Pasadena Unified School District; Cynthia Brown, Porterville Unified School District; Theresa McEwen, Sacramento Unified School District; and Cecilia Mendoza and Michael Aaronian, West Contra Costa Unified School District. We also thank district and school staff who worked with us to coordinate student survey activities. In particular, we thank Maria Villanueva, Antioch Unified School District; Karin Kroener-Valdivia, Los Angeles Unified School District; Crystal Howard, Long Beach Unified School District; Krystal Diaz, Montebello Unified School District; Claire Mueller, Oakland Unified School District; Rosa Valdez, Pasadena Unified School District; Larry Gray, Porterville Unified School District; Lily Liemthongsamout, Sacramento City Unified School District; and Michael Aaronian, West Contra Costa Unified School District. We greatly appreciate the students participation in our study; they provided valuable information on their pathway experiences. We recognize the assistance of individuals and organizations that provided data for our analysis of student outcomes. Many thanks, in particular, to Lauren Sosenko, Victor Manchik, and April Haagenson at the Institute for Evidence-Based Change. We also thank Cynthia Lim, Kathy Hayes, and Joshua Klarin of the Los Angeles Unified School District. We extend our appreciation to the staff at ConnectEd: The California Center for College and Career for their ongoing assistance with evaluation activities, with special thanks to Gary Hoachlander, Brad Stam, Roman Stearns, Kathy Harris, Rob Atterbury, Tameka McGlawn, Arlene LaPlante, and Anna Salomone. We are also grateful to the district and pathway coaches for the Linked Learning District Initiative and members of the various Linked Learning partner organizations for their insights during the evaluation. We acknowledge the thoughtful contributions of the members of the evaluation advisory group in reviewing study materials and prioritizing issues to investigate. Our advisors are Beverly Farr of MPR Associates, Nancy Hoffman of Jobs for the Future, Sean Reardon of Stanford University, Russ Rumberger of the University of California, Santa Barbara, and John Rogers of the University of California, Los Angeles. The report is the culmination of five years of evaluation research by a large team of SRI researchers. The writing team for this report was led by Nancy Adelman, Kyra Caspary, and Roneeta Guha, who provided intellectual leadership and guidance to individual chapter authors and reviewed and edited multiple drafts of the report. The primary chapter authors were Nicole Arshan, Samantha Astudillo, Adam Black, Christine Padilla, CJ Park, Regie Stites, and Victoria Tse. Members of our extended research team provided crucial support. We are indebted to our colleagues, Francine Biscocho, Jennifer Bland, Kristin Bosetti, Ashley Campbell, Erica Harbatkin, Erin Harless, Paul Hu, Nyema Mitchell, Paul Petit, Naomi Tyler, and Miya Warner, for their contributions to data collection, analysis, editing, and report production. We are grateful to Larry Gallagher, Harold Javitz, and Haiwen Wang, who consulted on technical matters. We also appreciate the contributions of Eileen Behr, Mimi Campbell, and Klaus Krause to the editing and production of the report. This evaluation is supported by a grant from The James Irvine Foundation. The opinions expressed in this report are those of the authors and do not necessarily reflect the view of The James Irvine Foundation. We thank the foundation staff, especially Kevin Rafter, who provided valuable substantive guidance and support throughout the design, implementation, and reporting phases of this study. We are also grateful to Anne Stanton and Aaron Pick for their support during various phases of the work. iv

9 Executive Summary Since 2006, The James Irvine Foundation has invested more than $100 million in Linked Learning, a promising approach to transforming education in California. In 2009, the Foundation launched the California Linked Learning District Initiative ( the initiative) to demonstrate this approach in nine districts. The multiyear evaluation for this large initiative has a two-fold purpose: to document the work, results, and lessons from districts that are applying Linked Learning systemically; and to measure the effect of this comprehensive implementation on student outcomes. SRI International s fifth annual evaluation report on the progress of the initiative comes at a time when Linked Learning is gaining momentum among K 12 and postsecondary educators, policymakers, and business leaders as a promising approach for preparing all students for college, career, and life. In early 2013, 63 districts and county offices of education were selected to participate in the California Linked Learning Pilot Program, which serves as a test of how Linked Learning can be expanded across the state. In June 2014, 39 partnerships received a total of $250 million through the California Career Pathways Trust, a competitive grant designed to develop work-based learning infrastructure, create regional partnerships, and improve and expand career pathway programs statewide. In 2015, a second round of grants will provide an additional $250 million to district and community college partnerships across the state. About Linked Learning Linked Learning integrates rigorous academics with real-world experiences. This approach aims to transform education into a personally relevant, wholly engaging experience and open students to career and college opportunities they never imagined. Linked Learning builds on more than four decades of experience gained by California schools that combine academic and technical content to raise student achievement. It seeks to improve high school graduation rates and increase successful transitions to a full range of postsecondary education opportunities, particularly for low-income and disadvantaged youth. Linked Learning is delivered through career pathways, comprehensive programs of study that connect learning in the classroom with real-world applications outside of school. It is within this context of increased funding and policy support for Linked Learning that we present this fifth-year evaluation report. Previous evaluation reports have focused on the development of district systems and structures to support new and existing Linked Learning pathways. As we close out the fifth year of our evaluation, we turn our primary attention to the students who participate in these pathways to ask the following questions: Who enrolls in pathways? Who stays? How do students feel about their experiences? What are their perceptions of the skills they are gaining? What effect does participation in a Linked Learning pathway have on students high school outcomes? To answer these and other questions, this report offers updated findings on student engagement and achievement outcomes from the nine districts participating in the initiative. Additionally, for the first time, our report takes an in-depth look at the issue of student equity and access to pathways through an analysis of student enrollment patterns across pathway career themes and of pathway retention among student subgroup populations. Finally, it assesses pathway students experiences with academic and technical curricula and work-based learning, their perceptions of the skills they are gaining as a result of their pathway experiences, and their plans for the future. Lessons from the experiences of the nine initiative districts are highly instructive for those that are just beginning to engage with or scale up Linked Learning. As context for understanding students experiences in pathways and their outcomes, this report provides an update on the nine districts efforts to develop and improve systems and structures to support Linked Learning and their initial plans to use new funding sources and regional partnerships to sustain Linked Learning. v

10 Student Equity and Access A central goal of the initiative is to provide all students with equitable access and opportunities for full participation in a variety of high-quality career-themed pathways. Evidence of such choice and access includes the percentage of students participating in pathways, as well as how representative these students are of each district s high school student population (in terms of prior achievement, socioeconomic status, gender, race/ethnicity, etc.); the absence of tracking by pathway (clustering students with low prior achievement in one set of pathways and students with high prior achievement in others); and retention in pathways, particularly for students with special learning needs, such as special education students and English learners. In the nine districts, district- and site-level leaders have been working to increase the numbers and variety of open-access pathways available to students. Through efforts to communicate information about pathway options to students and their parents, as well as targeted recruitment and outreach, district leaders have made progress in opening access to pathways to all students, including English learners, special education students, and students with low prior academic achievement. However, some districts have had more success than others in making pathways accessible to all student subgroups. Our findings suggest that: Patterns of student subgroup enrollment in certified pathways vary from district to district and by career theme. The only consistent patterns are disproportionately low female enrollment in certified pathways with an engineering career theme and disproportionately high female enrollment in certified pathways with a health sciences career theme. Within any given certified pathway, student course-taking patterns can differ greatly from student to student. Through an in-depth analysis of course-taking in two pathways, we found that diminishing percentages of pathway students enroll in the same core classes in each successive year of the pathway. Students in the same pathway tend to take different math and science courses, especially in the upper grades. Nearly 80 percent of students who start out in a certified pathway in its lowest grade level were still in the same pathway by the time they reached 11th grade, but students with special learning needs have lower than average rates of retention in certified pathways. Student Outcomes For the third consecutive year, we examined indicators of pathway students engagement in school, their progress toward high school graduation and college eligibility, and their gains in knowledge, statistically adjusting for their background characteristics and prior achievement. 1 These results are estimated across districts, rather than provided separately for each individual district, and compare outcomes for Linked Learning students to similar peers enrolled in traditional high school programs in each district. This single cross-district estimate is conceptually appropriate given that Linked Learning is an approach, not a series of individual initiatives implemented separately in each district. 2 Students in certified pathways are more likely than similar peers to remain in the district through the 12th grade and outperform them in in credit accumulation in the 9th 11th grades. 1 2 Student outcomes findings are based on data available from eight of the nine districts involved in the initiative. One district did not have any certified pathways at the time of analysis. To examine student enrollment and retention patterns within pathways, as well as outcomes for students in certified pathways compared with similar peers in traditional high school programs, we used student-level demographic and achievement data from the districts. For the analysis of student outcomes, we assigned students their pathway status based on the academic program in which they enrolled in the 9th or 10th grade, whichever was the lowest grade level served by the pathway. This is in contrast to pathway outcomes reported from the student survey, which represent all 12th graders enrolled in certified pathways across the districts in spring 2014, regardless of the grade level in which the students first enrolled in the pathway. vi

11 Percentage Points Taking Stock of the California Linked Learning District Initiative Fifth-Year Evaluation Report Our results reinforce the strongest and most consistent findings from our earlier reports: students in certified pathways are more likely than similar peers to remain in the district through the 12th grade and outperform their peers in credit accumulation in the 9th 11th grades. Findings around completion of the suggested a g college preparatory coursework by pathway students are also promising, though equivocal. Students in certified pathways are as likely as their peers to be on track to complete the a g coursework even though these students also have the demands of completing a career technical course sequence in high school, and they are more likely to be on track to complete these requirements at the end of 10th grade. Student subgroups who enrolled in certified pathways perform at last as well as (if not better) on credit accumulation and test score outcomes compared with their peers. As in previous years, we did not find evidence that Linked Learning leads to higher scores on most standardized achievement tests. We did find, however, that student subgroups most frequently underserved by traditional schools such as English learners, underachieving students, African American and Latino students who enrolled in certified pathways perform at least as well as (if not better) on credit accumulation and test score outcomes compared with their peers in the same subgroup in traditional high school programs. Overall, the results from this year s analyses suggest that Linked Learning may be leading to greater student engagement and moderately greater success in school. Engagement in School Because Linked Learning aims to make school more relevant for students, the core components of a pathway have the potential to increase students engagement in school. We used two measures to assess student engagement: attendance and retention within the district (a proxy for dropout prevention). We found: Pathway Students Were More Likely to Remain in the Same District In general, average attendance rates for pathway and non-pathway students were high. We did not find evidence that students enrolled in certified pathways had better attendance than similar peers in traditional high school programs th*** 11th*** 12th** On average, students enrolled in certified Grade pathways were 2.2 percentage points more likely to stay within their district from 9th to 10th grade, 4.6 percentage points Source: District-provided student data. more likely to stay through 11th grade, and **Statistically significant at p < percentage points more likely to ***Statistically significant at p <.001. remain through 12th grade, compared with similar peers in traditional high school programs. These differences likely occur because students continue to attend school instead of dropping out before graduation These findings indicate that students in certified pathways may be more engaged than similar peers such that they are motivated to remain in school. vii

12 Credits Taking Stock of the California Linked Learning District Initiative Fifth-Year Evaluation Report Success in School Even students who regularly attend school cannot progress through high school and toward college or career without successfully completing the necessary coursework. We examined students progress toward high school graduation, as measured by credits accumulated and course failures. We also assessed students progress toward college eligibility, as measured by completion of the coursework necessary to enter the California State University system. For the first time, we examined Algebra II completion by the end of the 11th grade; research suggests that students who take advanced mathematics courses during high school have better odds of attaining a bachelor s degree (Adelman, 1999; Gamoran & Hannigan, 2000; Horn, Kojaku, & Carroll, 2001). We found that students in certified pathways are accumulating more credits in the 9th 11th grades than similar peers: On average, 9th-grade pathway students earned 7.3 more credits than similar peers in traditional high school programs, while 10th-grade pathway students earned 6.9 more credits and 11thgrade pathway students earned 3.3 more credits. These differences are meaningful because the average student in each district accumulated about 55 credits (roughly 25% of the credits needed to graduate) in each of these grades. Extra credits in these early grades may provide pathway students with a buffer against later failures. The pathway and non-pathway student groups did not differ on course failures. In prior evaluation reports, we found that students in the majority of initiative districts were more likely than similar peers to be on track to complete the suggested a g college preparatory coursework at the end of the 9th and 10th grades. In estimating a single cross-district effect of Linked Learning this year, we found this result in the 10th grade only. Although the results point in the right direction in grades 9 and 11, the differences are not statistically significant in these grades. 3 More specifically, we found: On average, students in certified pathways were 7.9 percentage points more likely to be on track to complete the suggested a g requirements at the end of 10th grade than similar peers in traditional high school programs. Pathway Students Earned More Credits The weaker findings for the 9th and 11th grades may be due in part to our use of an a g on-track indicator rather than number of a g credits earned. The on-track indicator does not capture incremental differences in the number of a g courses taken unless these differences move students from not being on track to complete a g to being on track. In future years, we will look at the cumulative a g credits earned by pathway students and similar peers by the end of 12th grade to assess whether pathway students complete more college preparatory requirements throughout high school. In addition to analyzing course credits, course failures, and progress toward a g completion, we examined, for the first time, whether pathway students were more likely to complete Algebra II by the end Source: District-provided student data. **Statistically significant at p <.01. ***Statistically significant at p < th*** 10th*** 11th** Grade 3 The results for a g completion in the 9th and 11th grades have p-values of p >.05 and p <.1. We consider a p-value of <.05 to be statistically significant. viii

13 of 11th grade. This would allow them to take more advanced mathematics while still in high school, a critical determinant of postsecondary preparation and success (Adelman, 2006). However, we found no statistically significant difference between pathway students and similar peers in traditional high school programs in their likelihood of taking Algebra II by the end of 11th grade. Collectively, our analyses find limited but promising evidence that pathway students are more successful in high school than their peers: pathway students earned a full semester-long course s worth of credits or more above what their peers earned in grades 9 and 10, and pathway students were more likely than similar peers to be on track to complete the a g coursework at the end of their 10th grade year. Students Experiences in Pathways The Linked Learning approach strives to impact these student outcomes through students direct experiences with the academic and technical programs and work-based learning. During the school year, districts work around curriculum and instruction focused largely on implementing the Common Core State Standards; some districts more successful than others in their efforts to align Linked Learning with the rollout of these new standards. According to our survey of twelfth-grade students, early efforts to improve curriculum and instruction in pathways appear to be paying off, as a greater share of pathway students than comparison students reported experiencing rigorous, integrated, and relevant instruction in the following ways: At least one teacher challenged them to understand a difficult topic (75% versus 61%), asked difficult questions in class (79% versus 69%), and asked difficult questions on tests (83% versus 74%) about once a month or more. A teacher discussed how to apply what they were learning in class to the real world (66% versus 51%), explained how what they learned in class could be applied to what they might do after finishing high school (70% versus 58%), and asked them to use tools or equipment (69% versus 49%) about once a month or more. Used ideas or skills learned in class outside of school (68% versus 59%) and saw connections between what they learned in class and the real world (70% versus 60%) about once a month or more. In addition to offering a challenging academic program integrated with a demanding technical sequence of courses, Linked Learning pathways should provide all students with access to a continuum of highquality work-based learning opportunities that help them connect classroom learning to the skills and knowledge needed in a particular industry sector. This integration of the academic and technical curricula with work-based learning makes the Linked Learning experience unique. All districts are making progress on expanding the number and variety of work-based learning opportunities at the career exploration level (job shadows and mentoring), but student access to work-based learning experiences at the career preparation (internships and practicum) and training (work experience and certification) levels remains limited. Further, in all but a few exemplary pathways, leaders and teachers still have work to do to integrate work-based learning with classroom learning and pathway outcomes. On our survey of twelfthgrade students, we found: The vast majority of pathway students across all districts (ranging from 80% in Antioch to 93% in Oakland) reported participating in at least one work-based learning experience during the school year, with 87% of pathway students participating overall. Across the range of experiences, pathway students most frequently reported engaging in activities on the earlier end of the work-based learning continuum (career awareness and career exploration). Only 34% of pathway students reported participating in an internship during the school year. When asked to report how often students tie their work-based learning experiences back to the schoolwork, 28% of pathway students reported doing so most of the time or always. ix

14 One institutional barrier to developing more robust work-based learning systems is district staff capacity to generate and communicate opportunities for internships to students. Student-level barriers to participation in career preparation and training opportunities include competing demands on students time, limited transportation options, the need or desire to earn money (very few internships are paid), the need for credit recovery, and students lack of interest in the internship opportunities offered. Students Perceptions of Skills Gained Today s students need 21st century skills to succeed in any postsecondary endeavor. The Linked Learning College and Career Readiness Framework defines these skills as the range of cross-cutting cognitive processes and applications of knowledge needed to succeed in postsecondary education and future careers (ConnectEd, 2012, p. 2). We asked 12th-graders to report on the extent to which they felt high school had helped them improve a range of skills and behaviors. On our student survey, pathway students were more likely than comparison students to report that high school has helped them develop the following skills: Collaboration: Develop the skills necessary to interact effectively with people from different backgrounds (59% versus 49%), with adults outside of their family (40% versus 29%), and in professional settings (54% versus 33%), as well as to collaborate in a group to achieve a shared goal (56% versus 36%). Communication: Improve their ability to present information to an audience, whether by making a public presentation or performing in front of a group (52% versus 30%), or by speaking in public (43% versus 27%). Judgment: Develop their ability to use information to make good decisions (55% versus 38%), conduct online searches to answer a question (52% versus 36%), summarize information from multiple sources (45% versus 32%), and judge whether they can trust the results of an online search (42% versus 25%). Perseverance: Improve their ability to accept responsibility for the quality of their work (63% versus 51%), to believe they can reach their goals through hard work (55% versus 45%), and to believe they can learn something really difficult if they try (44% versus 28%). Organization: Develop useful self-management skills, such as setting goals for doing well in their classes (35% versus 27%), developing a system for organizing schoolwork (31% versus 25%), and managing their time in order to get all their work done (25% versus 21%). Students Postsecondary Plans and Supports Pathway students are developing productive dispositions and behaviors necessary for success in school and postsecondary endeavors. Ultimately, Linked Learning should foster students awareness of and readiness for college and career, and support their successful transitions their future undertakings. Our findings suggest that pathway participation has helped students identify career interests and enhanced their understanding of the education and training necessary to prepare for postsecondary opportunities. Greater percentages of twelfth-grade pathway students than of comparison students reported on our survey that they received: Opportunities to visit a college or technical school campus (75% versus 59%) and to speak with a college or trade school representative (74% versus 65%). A lot of help to understand high school graduation requirements (79% versus 68%), what they wanted to do after they graduated (47% versus 35%), the high school courses needed to get into college (64% versus 51%), how to choose a two- or four-year college (55% versus 44%), how to pay for college or training (49% versus 37%), what kind of education or training is needed to prepare for a possible career (44% versus 32%), and how to choose a career training or trade school (31% versus 23%). x

15 Pathway students were somewhat more likely than comparison students to report that they planned to continue their education full time (74% versus 68%) and work part time (66% versus 63%) after they graduated. Although the magnitude of these differences is small, the differences are statistically significant and unlikely due to chance. Similarly, pathway students were more likely than comparison students to report taking college entrance examples and submitting college applications, including taking the SAT (74% versus 62%) and PSAT (74% versus 66%), and submitting an application to a California State University campus (86% versus 79%) or a University of California campus (60% versus 54%). Looking Ahead Three years of student outcome analysis point to the promise of the Linked Learning approach. This year s results reinforce previous findings that Linked Learning participation is related to student engagement and success in school. Students in certified pathways are more likely than similar peers to stay in their district, and they accumulate more credits, putting them on track to graduate from high school. They also are just as likely as their peers to be on track to complete a g requirements at the end of 9th and 11th grades, and even more likely at the end of 10th grade. Yet there is still work to be done. Even when students are more engaged in school and complete more course credits, these positive outcomes do not consistently translate into improved achievement outcomes, as measured by standardized test scores. As districts continue to develop and expand pathways, Linked Learning practitioners must be vigilant about improving the quality of instruction and providing all pathway students the supports necessary to ensure that they succeed in their classes. Implementation of the Common Core provides a real opportunity for changes in classroom practice that may lead to improved student achievement results. As we look ahead to the future of Linked Learning in California and in other states across the country, pathway expansion plans have raised some concerns among Linked Learning administrators and our research team regarding fidelity to the Linked Learning approach that is, the extent to which a district can adapt the pathway approach before it is no longer appropriate to call the pathway a Linked Learning pathway. Within the nine districts, there is already a range of such approaches from districts with academies that do not meet Linked Learning pathway criteria to districts that adhere closely to ConnectEd s definition of a high-quality Linked Learning pathway, supported and sustained through strong centralized control of implementation by the district Linked Learning office. Beyond the districts in the initiative, a whole new crop of Linked Learning districts, through the California Linked Learning Pilot Program and the California Career Pathways Trust, will not receive the intense technical assistance or encouragement that the original districts received to adhere to the Linked Learning approach. If new Linked Learning pathways developed under these efforts fail to deliver, there is a danger that Linked Learning s positive image could be damaged and, as one district administrator shared, the whole brand will suffer. Additionally, the rapid growth of Linked Learning districts could test the scalability of the Linked Learning approach without extensive external supports. An essential element in district implementation of Linked Learning has been knowledge, expertise, prior experience, and other support from external partners. In particular, district leaders have found district-level coaching to be critical in supporting their ability to navigate initial planning and systems-building activities (e.g., support for district staff to understand and spread foundational knowledge of Linked Learning, getting key leaders on board, helping shift educators and other stakeholders mindsets to align priorities and supports with Linked Learning). New districts will have a much more limited support system, which could impact their approach to implementing Linked Learning pathways, specifically their focus on systems building. Moving forward, Linked Learning funders, technical assistance providers, and the broader field will need to continue discussing this critical question of fidelity to the Linked Learning approach in terms of the essential elements of pathways and a district s system of support for Linked Learning implementation. The evaluation of the initiative will continue for two more years. During this time, we will report on the progress of the nine districts as they transition to additional funding sources beyond The James Irvine xi

16 Foundation and ConnectEd to support and sustain Linked Learning implementation (districts will receive one final round of grant funding from the Foundation through ConnectEd for the school year). We will look into districts plans for sustaining and scaling Linked Learning, including the use of funds from the Local Control Funding Formula, the California Career Pathways Trust, and other resources to support Linked Learning. We also will examine the role of new regional partnerships in expanding workbased learning opportunities. Most importantly, during the next two years of the evaluation, we will provide new data on how well Linked Learning graduates fare compare with similar peers as they transition to postsecondary endeavors. xii

17 Chapter 1: Introduction SRI International presents its fifth annual evaluation report on the progress of the California Linked Learning District Initiative ( the initiative ). This report comes at a time when Linked Learning is gaining momentum among K 12 and postsecondary educators, policymakers, and business leaders as a promising approach for preparing all students for college, career, and life. In early 2013, 63 districts and county offices of education were selected to participate in the California Linked Learning Pilot Program, which serves as a test of how Linked Learning can be expanded across the state. 4 In June 2014, 39 partnerships received a total of $250 million through the California Career Pathways Trust, a competitive grant designed to develop work-based learning infrastructure, create regional partnerships, and improve and expand career pathways programs statewide. 5 In 2015, a second round of grants will provide an additional $250 million to district and community college partnerships across the state. Core Components of the Linked Learning Approach Linked Learning combines four elements designed to advance student success: Rigorous academics An academic core that includes college preparatory English, mathematics, science, history, and foreign language courses for all students. Career-based learning in the classroom A challenging career-based component of three or more courses to help students gain the knowledge and skills that can give them a head start on a successful career. Work-based learning in real-world workplaces A series of work-based learning opportunities that begin with mentoring and job shadowing and evolve into intensive internships, school-based enterprises, or virtual apprenticeships. Integrated student supports Services including counseling and supplemental instruction in reading, writing, and mathematics that help students master academic and technical learning. It is within this context of increased funding and policy support for Linked Learning that we present this fifth-year report. This report offers updated findings on student engagement and achievement outcomes from the nine districts participating in the initiative. Additionally, for the first time, our report takes an in-depth look at the issue of student equity and access to pathways through an analysis of student enrollment patterns across pathway career themes and of pathway retention among student subgroup populations. Finally, it assesses pathway students experiences with academic and technical curricula and work-based learning, their perceptions of the skills they are gaining as a result of their pathway experiences, and their plans for the future. Lessons from the experiences of the nine initiative districts are highly instructive for those that are just beginning to engage with or scale up Linked Learning. As context for understanding students experiences in pathways and their outcomes, this report provides an update on the nine districts efforts to develop and improve systems and structures to support Linked Learning and their initial plans to use new funding sources and regional partnerships to sustain Linked Learning. 4 5 The Linked Learning Pilot Program was authorized by Assembly Bill 790 (Furutani), passed by the California State Legislature in The program is managed by the California Department of Education. The California Career Pathways Trust was authorized by Assembly Bill 86 (Budget Act of 2013), passed by the California Legislature in The program is managed by the California Department of Education. The California State Budget included $250 million to be allocated by the California Department of Education through a one-time competitive grant process. 1

18 About Linked Learning and the District Initiative Since 2006, The James Irvine Foundation ( the Foundation ) has invested more than $100 million in Linked Learning, a promising approach to transforming education in California. Linked Learning integrates rigorous academics with real-world experiences to provide high school students with a personally relevant, wholly engaging experience and open them to college and career opportunities they never imagined. The Linked Learning approach builds on the more than four decades of experience gained by California schools that combine academic and technical content to raise student achievement. The objectives are to improve high school graduation rates and increase successful transitions to a full range of postsecondary education opportunities, particularly Districts Participating in the Linked Learning District Initiative Antioch Unified Long Beach Unified Los Angeles Unified Montebello Unified Oakland Unified Pasadena Unified Porterville Unified Sacramento City Unified West Contra Costa Unified for low-income and disadvantaged youth. Linked Learning is delivered through career pathways, comprehensive programs of study that connect learning in the classroom with real-world applications outside school. In 2009, the Irvine Foundation launched the California Linked Learning District Initiative, a demonstration of Linked Learning in nine California school districts. ConnectEd: The California Center for College and Career, established by the Foundation in 2006, is the primary intermediary and technical assistance provider. Numerous other partners support the initiative, including the Stanford Center for Opportunity Policy in Education, the Center for Powerful Public Schools (formerly the Los Angeles Small Schools Center), the National Academy Foundation, the College & Career Academy Support Network, and The Education Trust West. The Foundation is supporting the nine demonstration districts in developing systems of career pathways that are available to all high school students, with students selecting their pathway. The initiative serves as a vehicle for the Foundation and its partners to develop and refine the Linked Learning approach, to determine what makes Linked Learning successful at a systemic level, and to demonstrate the viability of Linked Learning as a comprehensive approach for high school reform. In this fifth annual evaluation report, we look at districts progress in developing the core components of Linked Learning pathways, focusing on curriculum, instruction, and work-based learning, and their efforts to sustain Linked Learning beyond their participation in the initiative. We examine students experiences with the core pathway components, their perceptions of the skills they are gaining as a result of their experiences, and their plans beyond high school. We also assess the factors that influence students access to and participation in pathways and the impact of pathway participation on student engagement, success, and knowledge gains. Status of the District Initiative The nine districts participating in the Linked Learning District Initiative vary in size, from close to 19,000 to more than 650,000 students, and represent a variety of geographic regions across California. All have a high proportion of disadvantaged students. Collectively, the nine districts serve approximately 286,000 high school students, or 15% of the almost 2 million high school students enrolled in California public schools. All have below-average student achievement as measured by California s Academic Performance Index (API), ranging from 715 to 784 compared with a statewide average of More than three-quarters of the high school students in each of these districts are nonwhite, and more than half of the students are socioeconomically disadvantaged, with district poverty rates ranging from 60% to 81%. 7 Exhibit 1-1 summarizes student demographic and achievement data for the nine districts Base API. The source for all demographic and achievement data cited here is the California Department of Education. Based on the percentage of students who qualified for free or reduced-price meals in

19 Exhibit 1-1 Demographic and Achievement Profile of Linked Learning Districts, District High School Enrollment a Minority b (%) English Language Learner (%) Poverty c (%) Graduation Rate (%) Math CAHSEE Pass Rate d (%) 2013 ELA 2014 Math 2014 ELA Certified Pathways Number Percentage Enrolled e Antioch Unified 5, Long Beach Unified 26, Los Angeles Unified f 198, Montebello Unified 10, Oakland Unified 12, Pasadena Unified 5, Porterville Unified 6, Sacramento City Unified West Contra Costa Unified 13, , Source: California Department of Education (CDE). a Includes enrollment at charter and noncharter schools classified by the CDE as high schools (public) and continuation high schools with active/pending status. b Percentage of all students who do not identify as White, not Hispanic, including students whose ethnic designation is listed as not reported. c Based on the percentage of students who qualified for free or reduced-price meals in in the whole district (not just high school students). d The California High School Exit Examination (CAHSEE) passing rates are based on the March exam date for 10th-grade students for and for all districts except Long Beach, Oakland, Pasadena, Porterville, and West Contra Costa. CAHSEE passing rates for Porterville, and West Contra Costa are based on a February exam date for 10thgrade students for and and Oakland and Pasadena for CAHSEE passing rates for Long Beach were averaged between the February and March exams. ELA is English language arts. e Percentage of high school students in the district enrolled in certified pathways, as provided by ConnectEd on July 1, f Profile is for all of LAUSD. The initial Linked Learning grant was made to Local District 4, but the district restructured beginning with the school year, dissolving the local district structures. Linked Learning is now a full districtwide initiative. 3

20 In 2010, ConnectEd developed and began using a tool and process to certify the quality of individual career pathways along the dimensions of design, engaged learning, system support, and evaluation and accountability. ConnectEd and the Linked Learning partners are using the certification process to establish and support examples of programs that implement Linked Learning with high quality and fidelity, whether they are part of the district initiative or individual schools or programs outside of the initiative. Exhibit 1-2 lists the 37 pathways ConnectEd had certified as of July 2014 in the nine districts. District Exhibit 1-2 Linked Learning Pathways Meeting Certification Criteria as of Certified Pathways School Types a Certification Year Pathway Enrollment Antioch Unified Dozier-Libbey Medical High School Small school Engineering and Designing Green Environments (EDGE) SLC b Law & Justice Academy (DVLJA) SLC c Long Beach Unified Architecture, Construction, and Engineering Academy (ACE) California Academy of Mathematics and Science (CAMS) Community of Musicians, Performers, Artists, and Social Scientists (COMPASS) SLC c Small school d SLC PEACE Academy SLC Media and Communications (JMAC) SLC Pacific Rim Business Academy SLC c Los Angeles Unified Los Angeles High School of the Arts (LAHSA) Small school Los Angeles School of Global Studies SLC New Media Academy SLC c STEM Academy of Hollywood Small school b Oakland Unified Life Academy of Health and Bioscience Small school c Pasadena Unified Media College Preparatory Small school c Education Academy SLC c Arts, Entertainment, and Media Academy (AEM) SLC c Business and Entrepreneurship Academy (BE) SLC c Creative Arts, Media, and Design Academy (CAMAD) Engineering and Environmental Science Academy (EESA) SLC SLC b Health Careers Academy (HCA) SLC b,c

21 Exhibit 1-2 Linked Learning Pathways Meeting Certification Criteria as of (concluded) District Porterville Unified Certified Pathways School Types a Certification Year Pathway Enrollment Partnership Academy of Business (PAB) SLC b,c Academy of Engineering (AOE) SLC b Multimedia Technology Academy (MTA) SLC b,c Partnership Academy of Health Sciences (PAHS) SLC b,c Academy of Performing Arts (APA) SLC Academy of Digital Communication and Design (DDC) SLC b AERO Academy SLC b,c Sacramento Unified Health Professions High School Small school b,d New Technology High School Small school Johnson Corporate Business Academy (JCBA) SLC b,c The MET Small school School of Engineering and Sciences Small school b West Contra Costa Unified Multimedia Academy SLC c Law Academy SLC c Engineering Partnership Academy SLC c Health Academy SLC c Source: Communication from ConnectEd (July 1, 2014). There are no certified pathways in Montebello. a SLC refers to a small learning community within a comprehensive high school, not necessarily supported by a federal Smaller Learning Communities program grant. Small school refers to a small stand-alone school. b Pathway is supported by the National Academy Foundation (NAF). c Pathway is a California Partnership Academy (CPA). d Magnet school. Fifth-Year Evaluation Activities In 2009, the Foundation commissioned the Center for Education Policy at SRI to conduct a rigorous multiyear evaluation of the initiative. SRI is assessing the nine districts implementation of the Linked Learning pathways and analyzing outcomes for students participating in them. SRI is using a multimethod research design that includes qualitative and quantitative data collection and analysis. The following key research questions guide the evaluation: What structures, policies, and supports facilitate the implementation and institutionalization of a districtwide system of high-quality pathways, and what challenges do districts face in implementing such systems? How do districts support the implementation of pathways, and what challenges do pathways face in implementation? What are the educational experiences and outcomes for students participating in pathways? 5