Mathematics and Science Partnerships Program Highlights. Analytic and Technical Support for Mathematics and Science Partnerships

Transcription

1 Matheatics and Science Partnerships Progra Highights Anaytic and Technica Support for Matheatics and Science Partnerships Contract # ED-04-CO-00 Task Order # 00 Apri 0 Prepared for: Pat O Conne Johnson U.S. Departent of Education OESE/ Matheatics and Science Partnerships 400 Maryand Ave, SW Washington, DC 008 Subitted by: Wheeer Street Cabridge, MA 08 In Partnership with: Daphne Minner, Ph.D. Een Bobronnikov Nathanie Donoghue Sione Fried Sean Morris T

2 Tabe of Contents Matheatics and Science Partnerships Progra Overview... Research on Effective Professiona Deveopent... Part : A Coser Look at the MSP Progra Coponents... Matheatics / Science Content and Teaching Skis...4 Professiona Deveopent Deivery Forats...6 Part : MSP Progra Outcoes...9 MSP Progra Augenting State-wide Refor Efforts...9 MSP Progra Encouraging Rigorous Project Evauation Designs... Trends in Teacher and Student Learning... 4 Suary of Sucesses... 6 References... 8 Appendix A: Snapshot of Matheatics and Science Partnerships Progras by State... 9 Appendix B: Detais on Professiona Deveopent (PD) and Participating Educators (Perforance Period 009)... 7 Appendix C: Copeted MSP Projects with Rigorous Evauation Designs... 8 Contents pg. i

3 Matheatics and Science Partnerships Progra Overview The Matheatics and Science Partnerships (MSP) Progra cae into being in 00 as a part of the passage of the No Chid Left Behind Act of 00. The MSP Progra adinistered at the U.S. Departent of Education is designed to iprove the atheatics and science knowedge of cassroo whie encouraging the to increase their use of teaching techniques associated with iproved student earning. One unique aspect of this progra is that it requires projects to be conducted through oca partnerships between high-need schoo districts and science, technoogy, engineering and/or atheatics (STEM) departents in coeges and universities. Projects reporting in the three ost recent fisca years ( ) had an average of 0 partner organizations per project. The U.S. Departent of Education s MSP Progra has been adinistered as a forua grant progra to the states since 00. The aount of oney given to individua states is based on the size of the student popuation and poverty rates within that state. Each state then issues a request for proposas, specifying priorities for projects and inviting educationa institutions to appy for sub-grants. A 0 states, the District of Coubia, and Puerto Rico have received MSP forua grants. Exhibit shows the funding disbursed by the U.S. Departent of Education between 00 and 0. Exhibit : MSP Progra Funding, Fisca Years 00 0 Federa MSP Funding (Miions of Doars) Federa Fisca Years Source: U.S. Departent of Education state budget tabes. A substantia nuber of educators and students have received the benefits of iproved science and atheatics instruction, reative to the size of this investent. Since 004, an average of,000 educators have annuay received professiona deveopent services in atheatics and science through this progra a tota of approxiatey 0,000 participants. In the ast three years, educators have reached approxiatey.6 iion students annuay, or over 7.8 iion students in tota. Many MSP projects train instructiona eaders, who then provide training to other in their schoos and districts, further expanding the professiona deveopent reach. Most of the who participate in MSP projects are reguar cassroo responsibe for atheatics or science content, though principas and district personne aso participate in soe projects. In the eary years of the MSP Progra ore projects served idde schoo educators, but in the ast three years there has been a shift towards eeentary grade educators. The next section wi expore the aignent between MSP project coponents and research-based best practices. pg.

4 Research on Effective Professiona Deveopent A nuber of research studies have identified coponents of in-service teacher professiona deveopent (PD) progras that have an effect on practice and student earning. The first coponent is the substantia tie that needs to be invested in the PD experience for it to have an effect on practice and utiatey student earning. A review (Yoon, Duncan, Lee, Scaross, & Shapey, 007) of research studies with rigorous evauation designs found that who received an average of 49 hours of PD, spread over 6 to onths, boosted their students achieveent by about percentie points on standardized achieveent tests. PD that offered to 4 hours had no significant effect on student atheatics and science achieveent. The tota nuber of PD hours is ony part of the story depth of the substantive focus is aso iportant. This substantive focus can occur in PD with reativey few hours if carefuy designed, but PD with ore hours increases the opportunity for deep content exporation (Cohen & Hi, 000), thus increasing the ikeihood of enhanced teacher content knowedge. Unfortunatey, extended-duration contentspecific PD is uncoon across our nation. Data fro the Schoo and Staffing Survey indicated that ony 4 percent of the nation s participated in ore than hours of PD in the content they teach (Wei, Daring-Haond, & Adason, 00). Key Features of Effective PD Substantia nuber of hours (~0) Intensive and foow-up experiences Faciitative of professiona coaboration Science or atheatics content focused with active earning opportunities to transfer into teaching practices and curricuu The way PD hours are distributed across tie is aso iportant. Having a concentrated earning opportunity through either workshops or institutes (typicay hed during the suer), with foow-up sessions to reinforce the earning fro the intensive experience, has been shown to be particuary supportive of teacher earning (Saxe, Gearheart, & Nasir, 00). This kind of structure aso provides an opportunity for professiona networks to for aong the participants and, if conducted within a schoo setting, can ead to the foration of counities of practice (Wei, Daring- Haond, Andree, Richardson, & Orphanos, 009) that can undertake activities such as peer observations and coaching, and anaysis of student work or assessent data to iprove the curricuu and instructiona practices in a given schoo. Studies aso show that the content of the PD progra is reated to student earning. In 998, Kennedy found that subject-specific, contentbased professiona deveopent that targets the curricuu as we as how students earn is ore effective than focusing on genera pedagogy, and further studies continue to corroborate this finding (Yoon et a., 007). Further, professiona deveopent experiences shoud be structured to refect how earn. Opportunities for active earning or sense-aking are iportant because they aow to engage in transforing their practice rather than ayering new strategies on top of the od (Wei et. a., 009; U.S. Departent of Education, 0; Nationa Staff Deveopent Counci, 00). pg.

5 Part : A Coser Look at the MSP Progra Coponents A of the aforeentioned key features of effective PD progras were incorporated into the egisated aowabe activities of the No Chid Left Behind Act (Tite II, Part B, Sections 0-0), which guides the MSP Progra ( Exhibit beow dispays these and other activities that states were aowed to fund through their requests for proposas. Exhibit : MSP Aowabe Activities in No Chid Left Behind, Part B, Sec. 0 Key Goas Increasing science / ath teacher content knowedge Prooting teaching skis Conducting suer institutes and foow-up Recruiting ath, engineering, science ajors into teaching* Deveoping curricua or aigning to state standards Estabishing distance earning Peer entoring by Exposure to STEM professionas K 8 science / ath expertise-buiding* No Chid Left Behind Aowabe Activities Creating opportunities for enhanced and ongoing professiona deveopent of atheatics and science that iproves the subject atter knowedge of such. Prooting strong teaching skis for atheatics and science and teacher educators, incuding integrating scientificay-based and technoogy-based teaching ethods. Estabishing and operating atheatics and science suer institutes or workshops, incuding foow-up training, for eeentary and secondary schoo atheatics and science. (A suer institute consists of at east weeks of work. Foow-up shoud be at east days.) Recruiting atheatics, engineering, and science ajors into teaching through the use of: (A) signing and perforance incentives; (B) stipends for certification through aternative routes; (C) schoarships to pursue advanced course work in STEM; and (D) other progras that the State educationa agency deterines to be effective in recruiting and retaining individuas. Deveoping or redesigning ore rigorous atheatics and science curricua that are aigned with chaenging State and oca acadeic content standards and with the standards expected for postsecondary study in atheatics and science. Estabishing distance earning progras for ath and science using curricua that are innovative, content-based, and grounded in current scientificay-based research. Designing progras to prepare a atheatics or science teacher at a schoo to provide professiona deveopent to other at the schoo and to assist beginning and other at the schoo, incuding echaniss to integrate the teacher's experiences fro a suer workshop or institute into the provision of professiona deveopent and assistance. Estabishing and operating progras to bring atheatics and science into contact with working scientists, atheaticians, and engineers, to expand such ' subject atter knowedge of and research experience in science and atheatics. Designing progras to identify and deveop exepary atheatics and science in the kindergarten through grade 8 cassroos. Encouraging underrepresented individuas into STEM* Training atheatics and science and deveoping progras to encourage young woen and other underrepresented individuas in atheatics and science careers (incuding engineering and technoogy) to pursue postsecondary degrees in ajors eading to such careers. *A reativey sa nuber of states (,, and 7 respectivey) entioned these activities in their RFPs so they are not presented in this report. For this report, a thorough review of each state s RFP, covering the tie period of , was undertaken to deterine the eve of proinence that these PD activities were given nationay. Appendix A provides a snapshot of MSP projects by state, based on data fro three sources: (a) the state-issued request for proposa for each of the 0 states, DC and Puerto Rico ; (b) annua perforance reports subitted during Perforance Period 009 ; and (c) suppeenta inforation fro state MSP coordinators. A review of state RFPs, project annua reports, and inforation provided by state coordinators aso reveas soe interesting trends about the science and atheatics content covered in For sipicity, throughout the reainder of the report the ter state incudes Washington DC and Puerto Rico. Perforance Period 009 (PP09) incudes projects where the ajority of activities took pace between October, 009 and Septeber 0, 00, but soe PP08 reports for which teacher and/or student data were not avaiabe in tie to subit during the previous year were aso incuded in PP09. pg.

6 MSP projects and the professiona deveopent depoyed through the MSP Progra. An overview of these is provided beow, and additiona detai is contained in Appendix A. Matheatics / Science Content and Teaching Skis A states requested that projects increase teacher content knowedge in science and/or atheatics. Science and atheatics higher education facuty ebers are required to serve as PD providers for each MSP project. However, haf of a states ( percent) additionay caed for partnerships that woud expose to STEM professionas such as atheaticians and research scientists. The MSP Progra s focus on increasing content knowedge through direct instruction by content experts is an effective strategy soidy backed by existing research. Nationay, 44 percent 4 of states fund MSP projects that have siiar ephases on science and atheatics content, percent focus ore on atheatics, and 4 percent focus ore on science (see Appendix A). Exhibit iustrates that, aong the types of atheatics content being taught, probesoving is the ost coon topic across grade eves. For projects serving eeentary and idde schoo atheatics, other coon topics are nuber and operations and agebra. For projects serving high schoo atheatics, the ost coon topics incude agebra and technoogy use for atheatics instruction. For science-specific PD, scientific inquiry was taught by ost projects, aong with topics in physica and earth sciences, as we as technoogy use in science instruction (see Exhibit 4). Projects provide PD on utipe atheatics or science concepts, but ost typicay focus on those concepts that are part of the core state curricuu. Exhibit : Projects Deivering Specific Matheatics Content by Grade Leve Projects Serving Eeentary Teachers (N=0) Projects Serving Midde Schoo Teachers (N=) Projects Serving High Schoo Teachers (N=8) Matheatics Content Probe-soving 87% 88% 84% Nuber and operations Agebra Geoetry 6 6 Measureent Probabiity and statistics 4 6 Reasoning and proof 8 60 Cacuus 7 8 Technoogy Other 7 Source: Annua Perforance Report ite VI.A. for 009. Tota nuber of projects reporting in PP09 was 90. The tota nuber of projects that provided professiona deveopent in atheatics content areas or processes in PP09 was 406. Percents tota ore than 00 because respondents coud check ore than one category. Projects coud serve one or utipe schoo eves. 4 was used as the denoinator for percentages reated to data fro state RFPs. 0 was used as the denoinator for percentages reated to active projects within a state in because two states did not report during this period. pg. 4

7 Exhibit 4: Projects Deivering Specific Science Content by Grade Leve Projects Serving Eeentary Teachers (N=6) Projects Serving Midde Schoo Teachers (N=7) Projects Serving High Schoo Teachers (N=9) Science Content Scientific inquiry 9% 9% 9% Physica science/physics Life science/bioogy Earth science Cheistry 4 0 Technoogy Other Source: Annua Perforance Report ite VI.B. The tota nuber of projects that provided professiona deveopent in science content areas or processes in PP09 was 4. The non-response rate was 0. Percents tota ore than 00 percent because respondents coud check ore than one category. Projects coud serve one or utipe schoo eves. In 8 percent of the state RFPs, there was aso an ephasis on providing PD on teaching skis that enhance student earning of science and atheatics content. Most states (8 percent) ephasized in their RFP that the science and atheatics content covered in the PD needed to be aigned with the state educationa standards, and enhance abiity to understand and use these standards. Professiona deveopent focused on substantive and research-supported teaching skis, as ephasized in the MSP Progra, has been shown to be an effective strategy for iproving both student and teacher earning. Across a active projects in the reporting period, neary two-thirds of students of MSP, scored at the proficient eve or above (64 percent of students in atheatics and 6 percent in science). This is an increase over prior reporting periods in 008, 8 percent in both atheatics and science; and in 007, 4 percent in atheatics and 49 percent in science. Aso in the reporting period, 7 percent of the MSP showed significant gains in science content knowedge and 6 percent in atheatics on pre-post PD coparisons adinistered by the individua MSP projects. pg.

8 Professiona Deveopent Deivery Forats Whie the Matheatics and Science Partnerships enabing egisation provided guidance on aowabe activities, it did not stipuate how these activities shoud be configured within projects to construct a cohesive professiona deveopent experience for. As we wi see in Part of this report, soe states provided further specifications for projects so that they fit into a state-wide refor effort, whie any states aowed project partners to construct and ipeent their own coherent PD odes. In this section we wi describe the siiarities and differences aong states and projects in the ways PD content was deivered to. Suer Workshops / Institutes and Schoo Year Foow-up Neary a states (4 states, 8 percent) requested that PD be deivered through suer institutes or workshops (or an equivaent intensive experience) with schooyear foow-up. A ajority of states (8 percent) aso stipuated the required iniu nuber of PD hours. In the 009 reporting period, the edian nuber of PD hours deivered across projects ranged fro to 89 hours per participant (see Appendices A and B for ore detais). This eets, and often exceeds, the recoended iniu nuber of PD hours (4 to 49) that research has shown to increase student achieveent in atheatics and science. It is aso iportant to note that the range in hours refects varying content knowedge and teaching ski goas of the states. For exape, in Aabaa (edian= hours) the MSP projects are designed to support a specific and ore iited aount of content whereas Texas (00 hours) has broad-reaching goas (see Part for state profies). Therefore the intensity of the experience for a teacher within any particuar topic or ski area ay be coparabe in these two states even though the tota nuber of PD hours varies greaty to cover ore breadth of topics. The PD activities varied fro project to project; however, there were soe coon activities, incuding exporation of atheatics and science education content standards, curricuu apping, esson and curricuu deveopent, cassroo odeing and deonstration, cassroo observation with feedback, and inquiry activities. The benefit of the suer intensive and schoo year foow-up PD forat is that it is easier to estabish a counity of practice aong the participants. Research supports activities that specificay encourage coaboration aong teaching coeagues, but as we wi see in the next section there are severa ways to estabish coaboration. Train-the-Trainer Forat In 9 state RFPs (6 percent), there was a specific ca for projects to prepare atheatics or science to provide PD to other at their schoo. The train-the-trainer PD forat can be hepfu in broadening the reach of the PD knowedge throughout a schoo syste, and soe MSP projects have found it usefu. pg. 6

9 Professiona Learning Counities Twenty-one percent of the 009 projects reported having a professiona earning counity (PLC) as part of their PD experience. At the ost basic eve a PLC consists of a group of participants (generay ) who eet over an extended period of tie to focus on iproving one or ore aspects of teaching and earning. The PLC serves as a foru for to share and refect on their experiences and participate in generative discussion with their coeagues. Within these guideines, PLCs take different fors, and vary in the nuber and types of participants, frequency of eetings, focus, and activities. Many PLCs within the MSP portfoio are coprised of within one subject area. This arrangeent aows for vertica aignent across grades within that subject, or for deving deepy into a particuar content strand within a state standard. Soe PLCs integrated across subjects to gain an interdiscipinary approach to content or to address students that they teach in coon. Sti other projects reported that PLCs were district-wide, which yieded diverse participant perspectives. For exape, one Indiana project estabished a districtwide PLC to support teacher eaders in creating schoo-eve PLCs around atheatics teaching strategies. Invoving fro different grade eves with varying experience eves aowed the to adapt activities to different grades, thus enhancing the vertica integration in the curricuu (Garrity, 00). In addition to content, PLCs addressed pedagogica issues pertaining to onitoring student thinking and designing ore effective instruction. Coony reported topics incuded strategies for inquiry-based earning, forative assessent, addressing student isconceptions, questioning, and scaffoding student understanding. The tie dedicated to participating in the PLCs aowed to deve deepy into instructiona nuances that typicay go unexained. Participants had the opportunity to conduct peer observation of instruction, which is aso not typicay avaiabe. PLCs served as support networks where ebers coud bring particuar issues fro their cassroos for discussion and probe-soving with the rest of the group. One specific type of PLC, utiized in about percent of MSP projects reporting in 009, is esson study. Lesson study is a type of cassroo practice-driven action research, in which a group of co-research, pan, ipeent, refect on, revise, and potentiay re-ipeent a esson. Though there were variations in how projects conducted their esson study, ost took a researchdriven approach to coaborative esson design. Lesson study aows to deve deepy into the research, data coection and observation, refection, and redesign of instruction. This was based on a text search of open-ended responses on APRs and ay not incude projects that either did not report on using a PLC in the open-ended responses or that used other ters to describe their PLCs. pg. 7

10 One particuary unique and copeing aspect of esson study is the use and evauation of evidence of student earning fro the esson being studied. Often student work is used for this purpose and coectivey deveop tangibe, specific indicators of earning that shoud be present in student work. This concentrated focus on observabe features of earning aows in the group to discuss and ake expicit their ideas on how they know earning has or has not occurred beyond just a student providing a correct answer. For exape, in one Ohio project, used state content standards and district pacing guides to identify the research esson s goas. With the hep of university facuty as faciitators, constructed earning hierarchies, expored instructiona strategies such as Think-PairShare and Socratic questioning, and used project-deveoped protocos for pre- and post-esson discussions on esson effectiveness for student earning (Appova, 0). Onine Learning Counities The professiona earning counities presented thus far priariy entai face-to-face interaction around shared instructiona events. However, for any educators this is not an option due to scheduing constraints or reote ocation. Professiona deveopent in rura counities poses a unique set of chaenges for MSP projects, such as iited resources stretched over vast areas, feeings of professiona isoation, and difficuty accessing new technoogies, research findings, or content experts. In these schoo settings, it is not uncoon for to take on utipe roes at both schoo and district eves, aking it a chaenge for the to focus on any one responsibiity (such as targeted professiona deveopent in STEM content). Many rura MSP projects epoyed traveing PD providers and instructiona coaches. In this way, a sa group of experts coud spread the PD benefits to a uch arger and ore disparate group of. However for a nuber of projects this approach was not feasibe, so instead they ipeented creative uses of onine and other counication technoogy to overcoe barriers to professiona deveopent. Whie ony percent of projects identify onine earning as their priary focus, any ore (at east 4 percent) use this echanis as an iportant part of their overa PD approach. In fact, 9 states (7 percent) expicity recognized the iportance of estabishing distance earning opportunities in their proposa soicitations. MSP projects use various kinds of technoogy to for and faciitate onine teacher earning counities videoconferencing, forus, discussion boards, bogging, chat roos, and fie sharing. These echaniss aow participants to engage in productive conversations in spite of distance or scheduing constraints, and have the added benefit of archiving participants ideas for ater access. Specific software appications that support onine counities such as Backboard or WebCT are used by projects to encourage coaboration and counication aong and STEM experts. Teachers who woud otherwise trave ong distances to eet their counterparts or university facuty were abe to for counities and/or entoring reationships through the use of counication technoogies. pg. 8

11 Part : MSP Progra Outcoes The MSP Progra coponents, as estabished by the No Chid Left Behind Act, are we-aigned with best practices for professiona deveopent that research has inked to student earning. Part of this report provided exapes of how states enacted these progra coponents within the projects they funded, and soe of the specific kinds of professiona earning counities that have been enabed through the MSP Progra. This section of the report wi discuss the effects of the MSP Progra in three areas: augenting state-wide science and atheatics refor efforts; encouraging rigorous project evauation designs to better understand the effect of PD interventions; and changing individua and students science and atheatics knowedge and cassroo experience. By ooking at progra effects in these three areas, a ore copete picture eerges of how the progra has infuenced changes at the state eve, within the education evauation counity, and on individua earning. MSP Progra Augenting State-wide Refor Efforts Since the MSP Progra disburses funds through forua grants, each state is responsibe for deterining how to use these funds to iprove STEM education within the andscape of other state progras or initiatives, and for anaging the projects given the existing state adinistrative structure. Each state handes these chaenges in different ways. Here we highight five states Texas, Aabaa, Indiana, Massachusetts, and Arizona that provide different exapes of integrating and everaging forua grant funding into the arger andscape of state-wide refor efforts to iprove STEM education. Texas, Aabaa and Indiana had STEM PD initiatives and adinistrative structures to support these initiatives, independent of the MSP Progra. Therefore, their chaenge was to figure out how to use the MSP funds to fi in gaps within their state-wide refor efforts and coordinate various activities across initiatives to copeent each other rather than create redundancy. Massachusetts has used MSP funds to shape an evoving strategic effort and Arizona has used the to estabish a specific state-wide PD progra (Inte Math). Beow, state profies provide additiona detais on how the MSP funds were used to further statewide refor efforts. Though these strategies are unique to each state, they do provide exapes for other states to consider in oving their STEM education efforts forward.6 TEXAS 6 In 967 the Texas Legisature estabished Education Service Centers to hep districts () iprove student perforance; () operate efficienty and effectivey, and () carry out the andates of the Texas Education Agency (TEA). These 0 Centers serve as iaisons between the TEA and schoo districts by disseinating inforation, conducting training, and providing consutation for federa and state progras. Whie the Centers assist the TEA in carrying out andates, they reain The inforation for these profies was provided by MSP state coordinators, and gathered fro existing state Departent of Education web docuents. pg. 9

12 independent and focus services on the needs of their schoo districts. Texas deiberatey ais to coordinate a professiona deveopent to effectivey everage resources. The State Board of Education recenty approved new Texas Essentia Knowedge and Skis science standards and is revising the atheatics standards. The TEA has training to support the ipeentation of these standards, but the progras estabished with MSP funding TX Statewide MSP Professiona Deveopent Network and TSTEM Centers provide PD in areas not covered by the state training, thus creating a ore coprehensive PD network. TX Statewide MSP Professiona Deveopent Network The University of Texas at Austin oversees this network, and uses the MSP funds to award Regiona Coaboratives (approxiatey $0,000 each, depending on the nuber of being served). Each Coaborative ust have an institution of higher education and a high need schoo district as partners, and address the specific PD needs of the coaborative ebers. The University of Texas provides train-the-trainer sessions for the coaboratives Project Directors on a variety of content area topics. The Project Directors return to their coaboratives and train Mentor Teachers. Each coaborative has a required nuber of PD hours required for the Mentor Teachers (80 00 hours). After training, each Mentor Teacher ust go back to their capus to train and entor Cadre ebers. For any years, the coaboratives worked with individua, but recenty Texas has shifted to an iersion ode where the coaboratives work with entire grade eves and/or capuses rather than individua. For federa fisca year 00, there were atheatics coaboratives and 9 science coaboratives supported. T-STEM Centers and Secondary Matheatics Teacher Supports The seven funded T-STEM Centers provide professiona deveopent to secondary schoos across the state in speciaized areas such as rocketry, robotics, and biotechnoogy. They assess the needs of the schoo, then design and deiver innovative training in conjunction with their partners. Unike the ore traditionay focused TX Statewide MSP Professiona Deveopent Networks, T-STEM Centers provide innovative professiona deveopent based on the instructiona and econoic needs of the area in which they serve ( S. Avery, persona counication, October, 0). The Aabaa Math, Science, and Technoogy Initiative (AMSTI) is the Aabaa Departent of Education s initiative to iprove atheatics and science statewide by providing () Suer Institutes (whoy supported by MSP funding); () equipent and aterias needed for effective atheatics and science instruction through Regiona AMSTI sites where thousands of kits are stored, shipped, and restocked (aterias kept by the teacher are purchased with state AMSTI funds, not MSP funding); and () entoring in the cassroo setting and foow-up with a participants. The MSP funds aso suppeent AMSTI efforts by fiing gaps in professiona deveopent that resut fro teacher obiity. MSP supports Copressed Training for who had been forery trained in the AMSTI progra but changed grade eves within their schoo, and newy hired in AMSTI schoos who have not previousy received training. By fiing these gaps with MSP funds, and in a few instances state and oca funds as we, ath and science in 46 percent of the schoos in the state have been trained. MSP funds are aso used to provide ebedded coaching to high need schoos by supporting fu-tie atheatics and science speciaist positions during the 00 0 funding period. The coaching and professiona deveopent supports provided by these personne have heped schoos to deveop structures to sustain the essons pg. 0

13 earned via Suer Institutes. As schoos buid capacity to support atheatics and science instruction, the need for speciaist services diinishes ( C. Jones, persona counication, October 4, 0; Hois & Howard, 0). Indiana has everaged the business counity to for a sustained effort to iprove science and atheatics education throughout the state. The Indiana Science Initiative (ISI) ( is a partnership between Ei Liy and Copany, the Indiana Departent of Education, and the I-STEM Resource Network ( working to refor K 8 science education. ISI started with the creation of Indiana s Strategic Pan for Science Education Refor in Deceber 008 by a coittee of K educators, scientists, governent agents, and higher education facuty, ed by the Nationa Science Resources Center. The ISI is anaged by the I-STEM Resource Network, a partnership of pubic and private higher education institutions, K schoos, business, and governent in 0 regions throughout the state. I-STEM supports K and eaders in ipeenting high acadeic standards for STEM iteracy for a students. I-STEM is abe to offer MSP districts, and those Indiana districts that have eected to join the ISI, a reiabe, stabe source of research-based professiona deveopent and anageent of science aterias. Districts and schoos that want to participate in ISI coaborate with one of the regiona I-STEM networks to deveop partnerships eigibe for MSP funding. Since the schoo year, the Indiana Departent of Education has supported over 0 schoo partnerships in deveoping innovative atheatics and science progras through the federa MSP Progra (K. Linz Neson, persona counication, October 8, 0). Massachusetts has used MSP funds to strategicay expand the reach of professiona deveopent and boster the rigor with MASSACHUSETTS which the ipact of these efforts can be assessed. The MSP Progra has aowed the state to regionaize and scae up severa successfu professiona deveopent odes such as the 80 hour Massachusetts Inte Math Initiative (MIMI) ( course and professiona earning counity foow-up. The origina MIMI ode was a partnership between district atheatics eadership and higher education professors, and was expanded by the Matheatics Iproveent Project to incude three institutions of higher education and over 8 districts in the Cape Cod and North River Coaboratives. Teachers in a of Southeastern Massachusetts were afforded the opportunity to participate in high quaity atheatics professiona deveopent; 60 copeted the MIMI course and foow-up during the grant period. This ode can be used to regionaize other professiona deveopent progras. The Race to the Top grant issued via the Aerican Recovery and Reinvestent Act of 009 has aowed for the expansion of ode courses deveoped in MSP partnerships by scaing up atheatics PD offerings in six regiona targeted support centers across the state. Teachers in the highest need districts now have the opportunity for quaity PD deveoped by MSP, wherever in the state they ive and work. The expansion of PD has aso taken pace at the university eve. Eeven of the courses deveoped by partnerships during have been approved as offerings by institutions of higher education. Two of the MSP-created courses are now avaiabe onine for pre-service. Others have been approved pg.

14 and continue to be offered to educators across Massachusetts for graduate credit through the state universities. Cuuativey, three degree progras in ath and science have been deveoped and are currenty offered to educators as a resut of the MSP Progra. The MSP Progra has aso provided the ipetus for iproving the toos used to assess the effects of PD progras in the state. The Professiona Deveopent Observation Too that was deveoped under MSP funding is currenty used in Massachusetts, not ony for MSP project courses, but aso for ost other suer institutes and workshops. MSP funds have aowed Massachusetts to iprove its easures of PD quaity. The MSP Progra chaenged the state s abiity to coect student achieveent data inked to, in order to investigate student outcoes of receiving PD courses. As a resut, the state s data warehouse was redesigned to provide a fraework that aows the state to seect and aggregate student data inked directy to specific, thus aowing for a ore robust assessent of the ipact of the PD interventions. Lasty, the MSP funds have contributed to better aignent of the U.S. Departent of Education and Massachusetts State STEM Pan goas. The Governor s STEM Advisory Counci, which represents a key stakehoders in the state, has deveoped the first ever state-eve STEM pan. This pan has been recognized by the Nationa Governor s Association as a ode for other states. The five goas of this pan are supported by the Race to the Top grant, state, and other federa funds. MSP supports goa five: to increase the percentage of STEM casses ed by effective educators fro Pre-K to. The MSP funds represent a significant source of targeted funding for STEM professiona deveopent provided to districts. Since the progra s inception in 004, MSP has reached,67 and fied,7 seats in 98 avaiabe courses ( C. Lach, persona counication, October 8, 0). Arizona has used MSP funds to adopt and disseinate a specific professiona deveopent ode, Inte Math, throughout the state because K 8 atheatics scores on the state assessent showed a rea need for iproveent. In the ost recent request for proposas, it was specified that ARIZONA projects ust incude a iniu of 04 contact hours during the ife of a project. The Inte Math training provides a one-week suer workshop and 7 PD days during the schoo year, for a tota of days of training. Inte Math is grounded in a probe-soving approach to topics such as integer arithetic, the decia nuber syste, pace vaue, rationa nuber arithetic, rates, inear equations, and functions. Connections are ade throughout the course as utipe representations of soutions are exained with each probe. Pedagogy coprises approxiatey 0 percent of the course content and cassroo transfer is addressed priariy through opportunities to anayze student work and instructor odeing. By restricting the type of professiona deveopent ode that can be supported with MSP funds, Arizona is aking targeted iproveents in the professiona deveopent of its ( S. Mast, persona counication, Noveber, 0). pg.

15 MSP Progra Encouraging Rigorous Project Evauation Designs Every MSP project is required to design and ipeent an evauation and accountabiity pan that aows for an assessent of its effectiveness. Projects are required to report annuay to the U.S. Departent of Education on two aspects of their evauation findings: ) gains in teacher content knowedge based on pre- and post-testing; and ) proficiency eves on state-eve assessents of students of who received professiona deveopent. In addition, states (40 percent) require participation in a statewide MSP Progra evauation that woud gather data beyond the annua reporting requireent (see Appendix A) to deterine the cuuative state ipact of the MSP Progra. Through their RFPs, states have the opportunity to pace stipuations on the kinds of designs and instruents that projects are eigibe to use for their individua project evauations. Sighty ore than haf of the states ( percent)7 specificay stated that either experienta8 or quasi-experienta designs were required ( states) or ( states). The reaining states (4 percent) did not specify an evauation design preference (see Appendix A). Across a funded projects for the ast reporting period, approxiatey haf ( percent) reported using an experienta or quasi-experienta design. This is on a par with the 008 reporting period, in which percent of the projects reported using a coparison group design, and shows an increase fro 007 when it was 44 percent of projects. As part of the portfoio review, Abt Associates conducted a review of fina-year MSP projects that reported using an experienta or quasi-experienta coparison-group design to evauate their MSP projects. The goa of the review was to deterine which projects successfuy conducted rigorous evauations yieding findings that coud be considered reiabe and vaid. To this end, Abt Associates conducted detaied reviews of projects evauations to assess whether they et criteria estabished for rigorous evauations of interventions.9 Of the fina-year projects, 6 were eigibe to have their evauation designs and findings reviewed for rigor. Exhibit beow depicts the process by which eigibiity for assessing progra effects was deterined. Exhibit : MSP Projects Seected for Review of Evauation Rigor Projects subitting fina report 4 Fina-year projects reporting using experienta or quasi-experienta design 6 Projects reporting using experienta or quasiexperienta design with appropriate coparison group Within each project, the evauation design to deterine the ipact of an outcoe teacher content knowedge, cassroo practice, or student achieveent was reviewed separatey for ethodoogica rigor. Of the 6 fina-year MSP projects reviewed for 009, 6 projects ( percent) successfuy 7 Coorado s RFP was ony for existing projects, so the denoinator for this section is rather than states. 8 Experienta design participants assigned at rando to MSP and non-msp professiona deveopent experience; Quasi-experienta design atched coparison groups 9 More inforation on the review and criteria is avaiabe in the fu Annua Report (Bobronnikov, Donoghue, Fried, & Mendez, 0). pg.

16 ipeented rigorous evauation designs to deterine the ipacts of their progras beyond what was captured in the annua reporting statistics (see Appendix C for descriptions). This represents a five-fod increase fro the previous year. Since haf of projects ( percent) in 009 reported using an experienta or quasi-experienta design in their evauation of ongoing projects, it is anticipated that in future years there wi continue to be an increase in the nuber of copeted projects that generate ipact findings via rigorous evauation designs. With regard to outcoes of interest, we found that the evauation design and instruentation to easure the effect of PD on teacher practice is in ost need of iproveent. In the review of projects, 9 of the studies that assessed this outcoe were deterined to be of ow ethodoogica rigor. The ack of reiabe and vaidated instruents for assessing the effect of an intervention on teaching practice is not an issue specific to the MSP Progra, but is rather a pervasive probe in the fied of STEM research and evauation. Trends in Teacher and Student Learning Earier, this report noted that, based on required annua reporting inforation, 6 percent of participating showed significant gains in atheatics content knowedge and 7 percent showed significant gains in science on pre-post professiona deveopent coparisons of content knowedge. Additionay, the students of these showed substantia increases in their science and atheatics proficiency neary two-thirds scored at the proficient eve or above (64 percent of students in atheatics and 6 percent in science). In ooking at the 6 projects whose evauations passed the rigor review, found significant gains in teacher content knowedge, found significant changes in practice, and found significant gains in student content knowedge. We identify beow soe of the specific findings within the PD projects focusing on science, ath, and ath and science cobined. Six projects offered professiona deveopent focused on science content for eeentary or idde schoo, and three presented evidence of effects two on teacher content knowedge and one on student achieveent. One of the studies showed that participating knowedge of earth and space science increased.-fod over that of a coparison group of non-participating. Another study deonstrated a significant gain in participating knowedge about atter and about otion and forces, whereas the non-participating showed an overa decrease in their knowedge of physica science. The third study deonstrating effects of science-focused professiona deveopent showed that students of the who had participated in the MSP suer institute and foow-up sessions iproved significanty in their understanding of physica science, atheatics and engineering. This iproveent was ore than that shown by students of non-participating. Additionay, the ore professiona deveopent activities a teacher perfored, the higher the students post-test scores. Seven projects offered professiona deveopent to eeentary or idde schoo atheatics. Three of these studies identified significant ipacts of the progra on student atheatics achieveent scores on state assessents, and three studies were not abe to identify a significant ipact. An additiona study showed that 94 percent of participating increased their atheatica knowedge reated to nubers, agebra, and geoetry an average of six ties ore than the non-participating. A pg. 4

17 significant ipact on cassroo practice was aso identified in this study, with 86 percent of the treatent copared to 60 percent of the contro showing a change in their practice. The who participated in the professiona deveopent ade substantia iproveent in the student/teacher reationship as a resut of the professiona deveopent experience. Aong the three projects that offered professiona deveopent in a cobination of topics (e.g., atheatics, science, and engineering), two deonstrated effects on teacher science content knowedge, and one on teacher atheatics and engineering content knowedge. Increased student achieveent in atheatics and science content for students whose participated in the professiona deveopent experience was deonstrated in one of the projects and not in the others. In coparing the distribution of findings, we see a cearer trend for the positive effect of these MSP project interventions on teacher science/atheatics knowedge than on practice or student achieveent within this set of 6 projects that conducted rigorous evauations. Since the ain goa of the MSP Progra is to increase teacher content knowedge, this trend is encouraging. As noted above, any other projects identified positive effects of their professiona deveopent on and students, and the 6 projects described in this section just represent the subset of projects that easured ipacts in a rigorous way. pg.

18 Suary of Sucesses Partnerships Between Loca Education Agencies and Facuty at Institutions of Higher Education Have Been Estabished in A 0 States, Puerto Rico, and Washington DC. These partnerships provide science and atheatics professiona deveopent to to increase their content knowedge. Forty-four percent of states fund a baanced portfoio of projects with reativey equa ephasis on science and atheatics content. Neary one-third of states have ore PD opportunities focused on atheatics, and 4 percent focus ore on science. Most states expicity require that the content covered in the PD be aigned with state educationa standards, and that PD be provided on teaching skis that research has shown to enhance student earning of science and atheatics content. Large Majorities of Teachers Show Significant Gains in Science and Matheatics Content Knowedge. Across a active projects in the ast annua reporting period (009 00), the ajority of MSP participating showed significant gains in content knowedge 6 percent in atheatics and 7 percent in science. These gains in atheatics were found in 8 percent of states reporting, and in science, 8 percent of states reporting representing widespread iproveent in the science and atheatics knowedge of our nation s K. Teacher Content Knowedge Buit through a Range of Professiona Deveopent Strategies. The ost coon strategy for deivering professiona deveopent was that which was recoended by the egisation suer institutes with schoo-year foow-up. Another strategy, used to reach ore within a district, was the train-the-trainer approach. In 9 state RFPs, there was a specific ca for projects to prepare atheatics or science to provide PD to other at their schoo. Most of these states had soe reote, substantiay rura districts that woud benefit fro oca PD capacitybuiding because externa support is often iited. Another coony used strategy to deveop oca capacity was the estabishent of a professiona earning counity, either through intensive in-person interactions aong or via onine earning counities (especiay for reote ocaes). Students Becoe More Proficient in Science and Matheatics. In the reporting period, neary two-thirds of the students of MSP scored at or above the proficient eve on state assessents (64 percent of students in atheatics and 6 percent in science), copared to fewer than haf in the previous reporting period. MSP Funding Augents State-Wide Efforts to Produce More Systeic Refor. States are integrating and everaging MSP funding into the arger andscape of state-wide refor efforts to iprove STEM education. Soe states had existing state-eve STEM PD initiatives that pre-dated the NCLB egisation. Therefore, they are using MSP funds to fi in gaps within their state-wide refor efforts. Other states are using MSP doars to estabish a specific state-wide PD progra that woud provide a coon educationa experience across a districts in the state, in areas that assessents have shown to be particuary weak for students. In soe states, MSP funds are used as the soe source of funding for atheatics and/or science PD. MSP Encourages Rigorous State-Leve Progra Evauation. Through request for proposas, states have the opportunity to pace stipuations on the kinds of designs and instruents that projects are eigibe to use for their individua project evauations. Sighty ore than haf of the states specificay stated that either experienta or quasi-experienta designs were required pg. 6

19 or. Across a funded projects for the ast reporting period, approxiatey haf reported using an experienta or quasi-experienta design. This is on a par with the 008 reporting period, but shows an increase fro 007. Whie there reains substantia roo for iproveent in ipeenting these rigorous designs, there has been a substantia increase in the percentage of projects generating reiabe and vaid findings over the past two reporting periods. MSP Is Aigned with Research on Effective Professiona Deveopent. Research has identified four key features of effective PD: Substantia nuber of hours (~0) Intensive and foow-up experiences Faciitative of professiona coaboration Science or atheatics content focused with active earning opportunities to transfer into teaching practices and curricuu This report has shown that the MSP Progra has supported the ipeentation of these key features across the projects funded, whie aowing for a range of approaches to eet identified needs within each state. pg. 7

20 References Aerican Recovery and Reinvestent Act of 009 (ARRA), Section 400-6, Tite XIV (Pubic Law ). Retrieved fro Appova, A. (0). Ipeenting esson study and easuring its ipact on teacher deveopent, annua report. Dayton, OH: Wright State University. Bobronnikov, E., Donoghue, N., Fried, S., & Mendez, J. (0). Matheatics and Science Partnerships: Suary of perforance period 009 annua reports. Cabridge, MA: Abt Associates, Inc. Retrieved fro Cohen, D., & Hi, H. (000). Instructiona poicy and cassroo perforance: The atheatics refor in Caifornia. Teachers Coege Record, 0(), Garet, M., Porter, A., Desione, L., Biran, B., & Yoon, K. S. (00). What akes professiona deveopent effective? Resuts fro a nationa sape of. Aerican Educationa Research Journa, 8(4), Garrity, K. (00). Barthooew Consoidated Schoo Corporation (BCSC) Math MSP, annua report. Coubus, IN: Barthooew Consoidated Schoo Corporation. Hois, S. & Howard, V. (0). AMSTI sustainabiity action pan. Conference Presentation, MSP Regiona Conference, Batiore, MD, February 6. Kennedy, M. (998). For and substance in inservice teacher education (Research Monograph No. ). Madison: Nationa Institute for Science Education, University of Wisconsin-Madison. Middeton, B. (00). Teachers Epowering a Students in Math and Science (TEAMS) Partnership, Annua report. Burington, NC: Aance-Burington Schoo Syste. Nationa Staff Deveopent Counci. (00). Standards for staff deveopent. Retrieved fro No Chid Left Behind Act of 00 (P.L. 07-0); 0 U.S.C aendent to the Eeentary and Secondary Education Act of 96. Saxe, G., Gearhart, M., & Nasir, N. (00). Enhancing students' understanding of atheatics: A study of three contrasting approaches to professiona support. Journa for Research in Teacher Education, 4, 79. U.S. Departent of Education. (0). Great and great eaders. A deeper dive into the adinistration s Bueprint for Refor. Retrieved fro Wei, R. C., Daring-Haond, L., & Adason, F. (00). Professiona deveopent in the United States: Trends and chaenges. Daas, TX. Nationa Staff Deveopent Counci. Retrieved fro Wei, R., Daring-Haond, L., Andree, A., Richardson, N., & Orphanos, S. (009). Professiona earning in the earning profession: A status report on teacher deveopent in the United States and abroad. Daas, TX. Nationa Staff Deveopent Counci. Retrieved fro Yoon, K. S., Duncan, T., Lee, S. W.-Y., Scaross, B., & Shapey, K. (007). Reviewing the evidence on how teacher professiona deveopent affects student achieveent (Issues & Answers Report, REL 007 No. 0). Washington, DC: U.S. Departent of Education, Institute of Education Sciences, Nationa Center for Education Evauation and Regiona Assistance, Regiona Educationa Laboratory Southwest. Retrieved fro pg. 8