An Independent Evaluation Conducted by: for the: Directorate for Education and Human Resources National Science Foundation

Transcription

1 Evaluation of NSF s Math and Science Partnership Program An Independent Evaluation Conducted by: for the: Directorate for Education and Human Resources National Science Foundation

2 MSP PE Page ii Table of Contents Section Page Glossary... iii The Evaluation of the MSP Program... 1 The Math and Science Partnership (MSP) Program... 1 Brief Summary of Evaluation Findings... 3 Education Outcomes... 3 Related Partnership Endeavors... 5 Challenges of Working within U.S. Education Systems... 8 References About the Evaluation Team The program evaluation team is a partnership led by COSMOS Corporation, with Dr. Robert K. Yin of COSMOS serving as principal investigator and Dr. Darnella Davis, also of COSMOS, serving as one of three co-principal investigators. Additional co-principal investigators and partnering institutions have been Prof. Patricia Moyer-Packenham at Utah State University and Prof. Kenneth Wong at Brown University. Prof. Dimiter Dimitrov of George Mason University has led the analysis of the student achievement trends. Drs. Yin and Davis authored the present report. This report is based on research supported by the National Science Foundation under Contract No. EHR Any opinions, findings, and conclusions or recommendations in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

3 MSP PE Page iii Glossary Evaluation of the MSP Program: An ongoing evaluation that started in 2004 and whose findings are based on 43 substudies that are part of the evaluation. Inservice PD [also see professional development]: The provision of education and related collaborative opportunities, to enrich the knowledge of existing K 12 teachers. Institution of higher education (IHE): A university, a four-year college, a two-year college, or any other institution offering formal education degrees beyond the K 12 level. K 20 pathway: The entire range of grades, from pre-kindergarten through undergraduate and graduate programs, through which students advance in their formal education. Math and Science Partnership (MSP) Program: A program of partnership and related grant awards, administered by the National Science Foundation. National Science Foundation (NSF): A U.S. government agency devoted to supporting scientific research and development as well as related education initiatives. Partnership: The main type of award made by the MSP Program, calling for collaboration by an institution of higher education (IHE) and a K 12 school system. Peer-reviewed academic journals: Research publications composed of articles whose acceptance only occurs following critical reviews by relevant experts. Preservice PD [also see professional development]: The provision of education and related collaborative opportunities, to enrich the knowledge of prospective K 12 teachers. Principal investigator (PI): The leader of a partnership or research project. Professional development (PD) [also see inservice PD and preservice PD]: The provision of educational and related collaborative opportunities, to enrich the knowledge of either existing or prospective K 12 teachers. Research and development (R&D): Scientific activities aimed at contributing new ideas (research) or their new applications (development). Research, evaluation, or technical assistance (RETA) project: A type of project supported by the MSP Program to provide evaluation-related assistance to the partnerships and conduct research on relevant partnership topics. Science, technology, engineering, and mathematics (STEM) discipline faculty: Professors whose primary field of research falls within one of these named disciplinary areas.

4 MSP PE Page 1 THE EVALUATION OF THE MSP PROGRAM The findings in this report come from a series of substudies conducted as part of an ongoing and independent evaluation of the MSP Program. 1 The findings are based on data collected from the program s initial set of 48 partnerships, who received their initial awards from 2002 to These partnerships then operated for an average of five to seven years, with NSF-MSP funds ending around The data collection consisted of site visits to each of these partnerships as well as reviews of their annual reports, evaluation reports, and submissions of data into the MSP Program s management information system. The evaluation design and results have been published in 19 separate research articles and 24 other unpublished reports. The evaluation substudies relevant to the present report have been cited throughout the text so that they can be consulted by readers wishing to learn 2 additional details about the findings. THE MATH AND SCIENCE PARTNERSHIP (MSP) PROGRAM Compared to students in many other countries, U.S. student performance in K 12 mathematics and science education continues to lag. 3 The lag poses a long-term challenge for building a vigorous, future workforce of scientists and engineers a workforce that includes teachers of mathematics and science at all grade levels. In light of this challenge, the National Science Foundation (NSF), a federal research agency also charged with nurturing and strengthening this workforce, supports a variety 1 See the Glossary for definitions of the italicized of programs in mathematics and science education. Among these programs, NSF started the Math and Science Partnership (MSP) Program as a major new initiative in Since its inception, the MSP Program has made competitive awards to many partnerships and related grantees. Each partnership is led by a core team at least one institution of higher education (IHE) and at least one K 12 public school system. 4 Other IHEs, school systems, businesses, and local community groups may serve as additional partners. The partnerships are not limited to any particular education activity. They are permitted to design activities that meet their local mathematics and science needs. As a result, the partnerships have been conducting a broad variety of activities (see Box): Partnerships Illustrative Activities Mathematics Provide professional development (PD) to 5th grade and middle school teachers, to give earlier attention to algebraic concepts Develop hands-on engineering tasks for teachers of middle school mathematics Help teachers to encourage underserved students to enroll in advanced placement (AP) courses Assist districts with inquiry-based curricula, by offering related PD Emphasize deep content knowledge, inquiry-based instruction, and hands-on problem solving and exploration in 11 PD courses, each requiring 9 days Science Assist teachers in delivering inquiry-based science, using kits at the elementary and middle school levels Assist teachers to implement a cutting-edge science curriculum Instill inquiry-based teaching and learning in different disciplines, developing new curriculum guides and classroom assessments Support teachers to help students use science notebooks, thus reinforcing students writing skills Increase inquiry-based teaching and real-world problem-solving skills of secondary school teachers terms. 2 The cited substudies represent only a portion of the evaluation s 43 substudies. All of the substudies are available at 3 See National Academy of Sciences, 2007 and 2010; and Fleischman et al., Although the systems and the MSP Program also cover pre-kindergarten, K 12 has been used throughout the report instead of the clumsier pre-k 12.

5 MSP PE Page 2 The MSP Program is distinctive because it calls for science, technology, engineering, and mathematics (STEM) discipline faculty to collaborate with K 12 systems. The faculty are to bring higher quality mathematics and science into the K 12 arena. The faculty may participate in different ways. The most common way has been to offer professional development (PD) opportunities for existing as well as prospective K 12 teachers. Other ways include assisting K 12 school systems with their curriculum standards, lesson plans, and instructional practices; developing improved classroom assessments; and working directly with K 12 students, such as at science fairs or in laboratory settings. The 48 partnerships have been located throughout the country (see Map). They serve urban, rural, and suburban areas. In a few cases, single partnerships serve school systems outside of their locale and even outside of their home state. Each partnership has been serving about 60 elementary, middle, or high schools. 5 Math and Science Partnership (MSP) Program: Location of 48 Partnerships Awarded in FY This is the median number of schools per partnership. The mean is much higher (100 schools per partnership) because a single large partnership serves a large number of schools.

6 MSP PE Page 3 BRIEF SUMMARY OF EVALUATION FINDINGS The findings in the remainder of this report may be briefly summarized as follows: 1. Positive correlations between schools participation in partnership activities and K 12 student achievement in mathematics and science; 2. Increased K 12 teachers content knowledge following participation in partnership activities; 3. Widespread involvement by science, technology, engineering, and mathematics (STEM) discipline faculty in partnership activities; 4. Extensive research publications by the partnerships, covering topics in mathematics and science education; and 5. Early signs that partnerships can sustain their collaborative work beyond their period of formal funding by the MSP Program. associated with improved student performance in mathematics and science, grades K 12. The needed analysis tested the correlation between schools participation in partnership activities and student achievement scores on state assessment tests (Dimitrov, 2010). A school was judged to be participating if, for any of a partnership s targeted activities, 30 percent or more of the school s targeted teachers participated for 30 or more hours in the activity, per year. The analysis covered six combinations: two subjects (mathematics or science), each at three grade levels (an elementary, middle, and high school grade). Across the partnerships, the results showed that: The greater the number of years that a school and its teachers participated in a partnership s activities during the latest four years being studied (from to ), the higher were its achievement scores at the end year ( ). The correlations were positive for all six test combinations and statistically significant for four of them (see Chart): Taking a mathematics test. Correlations Between Partnership Participation and Student Achievement Grade Level Mathematics Science Elem. +* +* Middle + + High +* +* * p<.05 EDUCATION OUTCOMES 1. Positive Correlations between Schools Participation in Partnership Activities and K 12 Student Achievement in Mathematics and Science. The first and most important evaluation question has been to determine whether the partnerships have been The patterns are virtually the same as those found in earlier years (Dimitrov, 2008; 2009a; and 2009b), suggesting a continued and strong relationship between participation in partnership activities and K 12 student achievement. A separate analysis of the participating schools showed that, within these schools, the African-American students had greater gains

7 MSP PE Page 4 compared to three other groups (Asian American, White, and Hispanic students). These results suggest that achievement gaps between the African-American and other students were diminishing during the four years also a desired change (the gaps between the Hispanic students and the Asian American and White students showed mixed changes). 2. Increased K 12 Teachers Content Knowledge Following Participation in Partnership Activities. To improve K 12 education, the partnerships have engaged in extensive professional development (PD) activities. The activities may be directed at existing K 12 teachers (inservice PD) or at prospective K 12 teachers (preservice PD) those still completing their formal education and who have yet to enter the teaching workforce. on-site assistance in individual K 12 classrooms or for a school s mathematics or science faculty as a group. All told, an estimated 82,537 K 12 teachers participated in some inservice PD activity during the four years from to (Moyer- Packenham & Westenskow, in press). An important question in assessing these activities is whether teachers have shown changes in their mathematics and science content knowledge after participating in the PD. The partnerships own assessments of these changes covered 12,952 of the participating teachers. A synthesis of these assessments showed that: Statistically significant gains were found for 63 percent of the teachers of mathematics and 78 percent of the teachers of science (see Chart): Engaging in a science professional development activity. Percent Gains in Content Knowledge by Participating Teachers Mathematics Science % Gains % No Gains The larger portion of the partnerships activities has supported inservice PD for existing K 12 teachers. 6 The inservice PD has taken many forms, such as: individual workshops; one- to three-week summer institutes; individual university courses; specially designed certification programs; 2- to 3-year master s programs; and 6 The partnerships have supported a broad variety of preservice PD, but these activities fall outside the scope of the present report. These findings are promising, but with the strong caveat that they only cover a small percentage of the participating teachers (no data were available about the others). 3. Widespread Involvement by STEM Discipline Faculty in Partnership Activities. The MSP Program s defining feature called for STEM discipline faculty to become heavily involved in partnership activities. Such involvement was aimed at deepening the mathematics and science content of the partnerships work. Thus, the involvement was expected to take the form of services provided to

8 MSP PE Page 5 K 12 schools, teachers, and students not just attendance at partnership functions. Traditionally, STEM discipline faculty, located in academic departments outside of an IHE s school of education, have not commonly collaborated with K 12 systems. The faculty s participation has usually been limited to the high school (but not elementary) grades. In contrast, the more frequent collaborators at all grade levels have come from the school of education faculty or from the staff of the K 12 districts themselves. Thus, recruiting and inspiring the STEM discipline faculty to engage in partnership activities has represented a key challenge. The evaluation found that 474 STEM discipline faculty and 391 education faculty reported participating in the 48 partnerships activities during (Yin, 2010b). Moreover, The STEM faculty participated in a higher proportion of the partnerships activities than did either the education faculty or the K 12 staff, whether the activities were categorized by: grade level (elementary, middle, or high school grades); type of partnership activity (PD as well as non-pd activities); or mathematics or science academic subject (Moyer-Packenham et al., 2009). Site visits to all of the 48 partnerships made by the evaluation team corroborated the partnerships reports. For instance, the STEM discipline faculty participated more than the others at all grade levels, beyond the conventional expectation of participation at the secondary school level only. The faculty s work also included offering PD workshops and institutes for existing K 12 teachers and not just courses within university programs. Some of the successful participation may be attributed to the MSP Program s emphasis on having a STEM discipline faculty member serve as one of the partnership s top leaders, such as the principal investigator or co-principal investigator, of these initial 48 partnerships. Such top leadership positioning may have made the recruitment of other STEM discipline faculty easier. At the same time, the number and proportion of STEM faculty involvement did decline in Possibly, as the 48 partnerships neared the end of their formal funding periods, the overall amount of partnership activity was declining, thereby affecting the STEM faculty s participation as well. RELATED PARTNERSHIP ENDEAVORS Beyond seeking to attain education outcomes, the partnerships also have tried to succeed in two other endeavors, discussed next. 4. Extensive Research Publications by the Partnerships, Covering Topics in Mathematics and Science Education. From the outset, NSF has expected the partnerships to study and evaluate their own activities, contributing to knowledge about K 12 student learning. 7 The funded partnerships have therefore been expected to foster new ideas in mathematics and science education, not just to support educational activities. Collaborating in a classroom exercise. 7 See the first two solicitations for proposals (NSF , p. 8; and NSF , p. 8). NSF has more recently called the MSP Program a major research and development (R&D) effort designed to improve K 12 student achievement in mathematics and science (NSF , p. 4).

9 MSP PE Page 6 To assess the partnerships contributions, the evaluation tracked the extent of the partnerships research publications appearing in peer-reviewed academic journals. 8 Each journal, by agreeing to publish a paper, in effect vouches for the quality and originality of the paper s ideas, as well as the soundness of the paper s methods (Yin, Hackett, & Chubin, 2008; and Davis & Yin, 2009). The published articles have covered many educational topics (see Box): Illustrative Topics of Partnerships Articles in Peer-Reviewed Journals Instruction and PD Technology impacts on teaching and learning Teachers decomposing of instructional tasks Peer coaching effects on teacher interactions and student achievement Use of observation guides to support science instruction Mentoring of science teachers in an alternative certification program A science lesson plan analysis for evaluating teacher education programs Delivering online mathematics PD to rural educators Design of tasks supporting teachers' development of coherent mathematics meanings Curriculum and Assessment Teachers use of practice tests for formative purposes Linking formative assessment principles to classroom practice Writing assessment items that reveal student learning Judging the alignment of curriculum standards and assessments Differences in middle school science achievement Visualizing division of fractions Tool for assessing students precalculus reasoning abilities and understandings Student discourse viewed in relationships between intended and enacted math curricula All told, The awardees had published 172 articles in peerreviewed journals as of 2009 (another 69 articles were published in journals that were not peerreviewed or whose peer-review status could not be determined). 9 These numbers are substantial, although no benchmark exists for comparing the partnerships contributions in relation to other research and development (R&D) programs. For the MSP Program, many more manuscripts still were in the publication pipeline as of The total volume of publishing activity seems to support the conclusion that the MSP Program has been operating as the desired R&D program (see Chart): 304 Published Research Works 172 Articles in Peer- Reviewed Journals 69 Articles in Other Journals 63 Other Publications (chapters, books, etc.) Nevertheless, important study questions still deserve more attention. Recalling that professional development (PD) has been the dominant type of partnership activity, a clear description of these activities is needed to assess properly their role. Such description also is needed by other K 12 systems wishing to replicate any successful PD. 8 The articles were produced by the 48 partnerships as well as a smaller group of Research, Evaluation, and Technical Assistance (RETA) awards made by the MSP Program from 2002 to RETA awardees do not implement partnerships but provide evaluation-related assistance to the partnerships and conduct research on relevant partnership topics. 9 The finding was not dominated by the RETAs, as the number of articles per partnership exceeded the number per RETA. Similarly, no single or small group of partnerships or RETAs dominated the results.

10 MSP PE Page 7 To date, the partnerships have only collected limited details about their PD activities. Needed are independent observations 10 regarding the nature of a PD activity s providers, settings, curricula, instructional methods, and assessment tools as the activity was actually enacted, not just as it was planned (Yin, 2010a). Such an articulation therefore serves as a gap to be filled by the MSP Program s future studies and evaluations. 5. Early Signs that Partnerships Can Sustain Their Collaborative Work Beyond Their Period of Formal Funding by the MSP Program. NSF also encouraged partnerships to sustain their collaborative work beyond the period of formal funding from the MSP Program. Such collaborative work could be funded by new external sources or by the partners themselves. numerous instances where STEM faculty have designed, modified, or enhanced their courses in STEM academic departments for enrollment by existing K 12 teachers (see Box): Courses for Existing K 12 Teachers One partnership s PD participants included 16 interns and 100 certified teachers. All were taking courses related to either of two advanced degrees, both having STEM concentrations. Another partnership has existing teachers enrolled either in a master s of mathematics for teaching program or in a second program leading to a certificate of advanced graduate study in math education. Existing teachers may take upper level math methods courses as part of a supplemental credential program where they can gain a higher level of mathematics knowledge and an increase in pay. A partnership s IHE partner offers degree programs for existing teachers that include a 12-credit sequence with a level of difficulty reported to be equivalent to the upper-level courses taken by mathematics majors. The two main partnering IHEs offer 9 new graduate courses designed for both preservice and existing K 12 teachers. During the summer of 2007, 205 existing teachers and 25 preservice students enrolled in the courses. Teaching mathematics. Sustaining by Offering IHE Courses and Programs for Existing K 12 Teachers. Many partnerships have continued to offer their original professional development to new cohorts of prospective and existing K 12 teachers every year. Especially promising, from a sustainability perspective, have been 10 The partnerships have commonly used surveys of PD participants as the basis for describing a PD activity. However, such information does not represent independent observations, and the survey questions usually do not cover critical details about the enacted PD. The enrollees in these courses may include preservice students and undergraduates, but the courses are primarily oriented toward the inservice education of the teachers in the K 12 partnering districts (Yin, 2009). The evaluation found that, from to and across the 48 partnerships: 63 IHEs reported offering 492 newly designed, modified, or enhanced courses, of which 381 (77 percent) were located in STEM discipline departments (Yin, 2010b). All told, the 492 courses had 86,314 enrollees. Unfortunately, the data do not clarify the precise nature of the enrollees in these courses. However, many of the courses were created as part of formal programs offering advanced degrees or other certificates to existing K 12

11 MSP PE Page 8 teachers (not prospective K 12 teachers). Most important, the courses were being offered in STEM discipline departments and engaging STEM discipline faculty, reflecting the priorities of the MSP Program. Engaging in a professional development activity. From a sustainability perspective, to the extent that the enrollees do include large numbers of existing K 12 teachers, the activity offers mutual, longer-term benefits. The partnering STEM departments and faculty may benefit from the increased enrollment in their programs. In turn, the partnering K 12 districts and their teachers may benefit from the more intense lessons in mathematics and science than might occur in traditional PD workshops or summer institutes. For instance, formal course offerings at an IHE, especially if part of a new degree program, can require an academic review. 11 Such review serves as a quality control measure not present in the offering of off-campus PD workshops and institutes. These IHE courses for existing K 12 teachers may therefore form the basis for lasting collaborative work. The lasting effects may nevertheless be sustained at only diminishing activity levels. Thus, the previously described declining participation by the IHE STEM faculty, potentially related to the ending of the original partnership awards, may mean that the number 11 See Shapiro et al., 2006, p. 7. of courses offered may not continue at the same levels. Sustaining by Changing IHE Tenure and Promotion Rules. Other efforts at sustainability have involved changes in the policies or practices of the partnering organizations. For instance, some partnering IHEs have changed their tenure and promotion rules to recognize and reward faculty for collaborating with K 12 education. 12 As suggested by the evaluation s site visits, such changes, though promising, are but the beginning of a longer and somewhat uncertain path. First, within the same IHE, the new rules may pertain to some STEM departments but not others. Second, even after favorable rules are in place, the changes then need to be translated into concrete actions in reviewing the work and advancement of individual faculty members. Third, given the small number of faculty reviewed each year, broader institutional effects are only likely to occur over long periods of time. CHALLENGES OF WORKING WITHIN U.S. EDUCATION SYSTEMS Although the MSP Program committed about $629 million to these initial 48 partnerships and a smaller number of related awards made from 2002 to 2004, the challenges of working within this country s numerous K 12 local education systems should not be underestimated. First, the MSP Program s investments represent only a fraction of the federal funds devoted to K 12 education. In turn, federal investments represent only about 8 percent of the funds spent by the K 12 systems themselves. 13 Second, the systems are only one part of a larger K 20 pathway, whereby students receive a mathematics or science education and 12 See Kutal et al., President s Council of Advisors on Science and Technology, 2010, p. 23.

12 MSP PE Page 9 ultimately enter the STEM workforce. The pathway consists of multiple institutions e.g., K 12 school systems, community colleges, 4-year colleges, and graduate degree programs (see Diagram). These institutions do not necessarily coordinate their policies or formally collaborate. For instance, high school or community college graduation requirements may not align with the entrance requirements of a four-year IHE in the same locale. A major reason for appreciating the fuller K 20 pathway is to set realistic expectations for the work of initiatives such as the MSP Program. Successful partnerships must strengthen local education systems but also make K 20 pathways more coherent. Although the MSP Program has been showing concrete progress as presented by this report, more meaningful changes may only follow a persistent and longterm investment strategy. To this end, the MSP Program has continued to fund new partnerships. It also has made additional funds available to a small number of the original 48 partnerships, to deepen their research on selected activities i.e., those showing promise as potential models for the broader STEM education field. Pathways to an Enhanced STEM Workforce

13 MSP PE Page 10 References Davis, D., & Yin, R. K. (2009). Articles Published in Peer- Reviewed Journals: Progress by MSPs and RETAs in Contributing to Education Research and Practice. COSMOS Corporation, Bethesda, MD (an evaluation substudy). Dimitrov, D. (2008). Initial Trends in MSP-Related Changes in Student Achievement with MIS Data. Peabody Journal of Education, 83(4): (an evaluation substudy). Dimitrov, D. (2009a). Intermediate Trends in Math and Science Partnership-Related Changes in Student Achievement with Management Information System Data. Journal of Educational Research & Policy Studies, 9(2): (an evaluation substudy). Dimitrov, D. (2009b). Longitudinal Trends in Math and Science Partnership-Related Changes in Student Achievement with Management Information System Data. George Mason University, Fairfax, VA (an evaluation substudy). Dimitrov, D. (2010). Longitudinal Trends in Math and Science Partnership-Related Changes in Student Achievement with Management Information System Data Across Five Years ( to ). George Mason University, Fairfax, VA (an evaluation substudy). Fleischman, H. L., et al. (2010). Highlights from PISA 2009: Performance of U.S. 15-Year-Old Students in Reading, Mathematics, and Science Literacy in an International Context (NCES ). U.S. Department of Education, National Center for Education Statistics, Washington, DC: U.S. Government Printing Office. Kutal, C., et al. (2009). Engaging Higher Education Faculty in K 16 STEM Education Reform, in J. S. Kettlewell, & R. J. Henry (Eds.), Increasing the Competitive Edge in Math and Science. Rowman & Littlefield Education: Lanham, MD. Moyer-Packenham, P. S., et al. (2009). Participation by STEM Faculty in Math and Science Partnership Activities for Teachers. Journal of STEM Education, 10(3-4): (an evaluation substudy). Moyer-Packenham, P. S., & Westenskow, A. (in press). Processes and Pathways: How do Mathematics and Science Partnerships Measure and Promote Growth in Teacher Content Knowledge. School Science and Mathematics (an evaluation substudy). National Science Foundation (2002). Math and Science Partnership: Program Solicitation. (NSF ). Arlington, VA: National Science Foundation. National Science Foundation (2003). Math and Science Partnership Program: Program Solicitation. (NSF ). Arlington, VA: National Science Foundation. National Science Foundation (2010). Math and Science Partnership: Program Solicitation. (NSF ). Arlington, VA: National Science Foundation. President s Council of Advisors on Science and Technology (2010). Report to the President, Prepare and Inspire: K 12 Education in Science, Technology, Engineering, and Math (STEM) for America s Future. Washington, DC: Executive Office of the President. Shapiro, N., et al. (2006). Report on Course and Curriculum Changes in Math and Science Partnership (MSP) Programs. Prepared for the National Science Foundation by the CASHÉ (Change and Sustainability in Higher Education) Project Team, University System of Maryland, Adelphi, MD. Yin, R. K., Hackett, E., & Chubin, D. (2008). Discovering What s Innovative: The Challenge of Evaluating Education R&D Efforts. Peabody Journal of Education, 83(4): (an evaluation substudy). Yin, R. K. (2009). Establishing Long-Term Partnerships between K 12 Districts and Science, Technology, Engineering, and Mathematics (STEM) Faculty. COSMOS Corporation, Bethesda, MD (an evaluation substudy). Yin, R. K. (2010a). Defining Professional Development in Relation to Student Achievement: A Synthesis of Selected Partnerships Evaluations. COSMOS Corporation, Bethesda, MD (an evaluation substudy). Yin, R. K. (2010b). The Math and Science Partnership Program Evaluation (MSP PE): Year-6 Update. COSMOS Corporation, Bethesda, MD (an evaluation substudy). All photographs are taken from individual partnership Web sites. National Academy of Sciences (2007). Rising above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future. Washington, DC: National Academies Press. National Academy of Sciences (2010). Rising Above the Gathering Storm, Revisited: Rapidly Approaching Category 5. Washington, DC: National Academies Press.

14 3 Bethesda Metro Center Suite 700 Bethesda, MD 20814