An Overview of Federal STEM Education Programs Grover (Russ) Whitehurst, Ph.D. Senior Fellow Director of the Brown Center on Education Policy The Brookings Institution
Why should we care about science education? Growing national concern about economy Globalization Out-sourcing & off-shoring Rise of other nations Friedman: The World is Flat 40+ weeks on the list of top selling books Communicated the message Rising Above the Gathering Storm National Academies of Science report on competitiveness
Competitiveness Indicators The United States is today a net importer of high-technology products. Its trade balance in high-technology manufactured goods shifted from plus $54 billion in 1990 to negative $50 billion in 2001. Chemical companies closed 70 facilities in the United States in 2004 and tagged 40 more for shutdown. Of 120 chemical plants being built around the world with price tags of $1 billion or more, one is in the United States and 50 are in China. In 2005, only four American companies ranked among the top 10 In 2005, only four American companies ranked among the top 10 corporate recipients of patents granted by the United States Patent and Trademark Office.
More Competitiveness Indicators In South Korea, 38% of all undergraduates receive their degrees in natural science or engineering. In France, the figure is 47%, in China, 50%, and in Singapore 67%. In the United States, the corresponding figure is 15%. Some 34% percent of doctoral degrees in natural sciences and 56% of engineering PhDs in the United States are awarded to foreign-born students. t In the U.S. science and technology workforce in 2000, 38% of PhDs were foreign-born Federal funding of research in the physical sciences, as a percentage of GDP, was 45% less in FY 2004 than in FY 1976.
Student Performance: NAEP
Student Performance: NAEP
PISA Science 2006
Top and bottom performers % 100 Level 5 Level 4 Level 3 Level 2 Below Level 1 Level 1 80 60 40 20 0 530 563 515 527 531 534 500 474 475 424 410 489 522 20 40 60 Large proportion of top performers Large prop. of poor perf. New Zealand Finland Unit ted Kingdom Australia Japan Canada OE ECD average Portugal Italy Turkey Mexico Un ited States Korea OECD (2007), Learning for tomorrow s world: First results from PISA 2006, Table 2.1a
Federal responses to the competitiveness challenge
The America Competes Act of 2007 Responds to concerns that the United States may not be able to compete economically with other nations in the future due to insufficient investment in STEM Authorized increased funding for basic research and education Results in U.S. Department of Education Program 2008 funding 2009 Congressional Action AP and IB programs $48M Senate = $0; House = Level Teachers for a Competitive Tomorrow $2M Senate = $0; House = $0 Math Now $0 $0 Foreign Language Partnerships $0 $0
Academic competitiveness and SMART grants (HEA IV, subpart 1, section 401A) A National SMART Grant will provide up to $4,000 for each of the third and fourth years of undergraduate study to full-time students who are eligible for a Federal Pell Grant and who are majoring in physical, life, or computer sciences, mathematics, technology, or engineering or in a foreign language determined critical to national security. 2008 funding level = $395M to cover 638,000 students, $652M returned to Treasury because of lack of demand
The Academic Competitiveness Council The Deficit Reduction Act of 2005 (P.L. 109-171) established the ACC. The statute charged the ACC to: -- Identify all federal programs with a mathematics or science education focus; -- Identify the effectiveness of those programs; -- Determine areas of overlap or duplication among those programs; -- Identify target populations served by such programs; and, -- Recommend processes to efficiently integrate and coordinate those programs.
Methodology Developed an inventory of Federal education programs primarily focused on improving math and science education Grouped programs into categories based on their purposes and goals; K-12 programs to improve student learning Postsecondary fellowships to train future scientists Informal education and outreach to enhance awareness of science Determined common, outcome-based performance goals and measures for each program category. For example, K-12 investments t should be measured by whether they boost student t learning. Identified programs that could demonstrate results
Results of the ACC inventory 105 STEM education programs, with approximately $3.12 billion in total funding for Fiscal Year (FY) 2006 -- 24 elementary and secondary school (K 12) programs, which received approximately $574 million, or 18.4 percent of total funding -- 70 undergraduate, graduate and postgraduate programs, which received more than $2.4 billion, or 77.2 percent of total funding -- 11 informal education and outreach programs, which received close to $137 million, or 4.4 percent of total funding
Results of the ACC inventory Categorization of programs -- Research -- Minority or Underrepresented Populations -- K-12 Instructional Resources and Practice -- K-12 Teacher Education -- Undergraduate Fellowships -- Undergraduate Institutional Capacity Building -- Graduate and Post-graduate Fellowships -- Informal Education -- Other
Inventory
Largest Programs The Ruth L. Kirschstein National Research Service Awards ($761 million, National Institutes of Health: graduate and postdoctoral support) The National Science and Mathematics Access to Retain Talent (SMART) grants ($390 million, Department of Education: need-based undergraduate grant aid) The Mathematics and Science Partnerships program ($182 million, Department of Education: elementary and secondary education, including teacher training and professional development) Discovery Research K 12 ($93 million, NSF: elementary and secondary education, including teacher preparation p and professional development); The Graduate Research Fellowships program ($93 million, NSF: graduate education)
Findings and results of the ACC
Substantial overlap and duplication K 12 Teacher Quality: 45 programs have a goal to recruit and retain teachers with majors or minors in STEM fields or to increase the content knowledge of current K 12 STEM teachers. Pre-service teachers are a target population in 22 programs and in-service teachers are a target population in 39 programs. Underserved Populations: 57 programs, approximately half of the programs in the ACC program inventory, have a goal to support activities that ensure underserved populations are better represented in STEM fields and study.
Findings on effectiveness Each agency was asked to provide best examples of evaluations of programs it administered. 115 evaluations were submitted. The evaluations were evaluated based on the quality of the research design for answering questions of cause and effect: Tier I: Experimental Designs Tier II: Quasi-experimental Designs Tier III: Other designs, e.g., Pre-Post
Yield from the evaluation of evaluations 10 evaluations were scientifically rigorous evaluations that produced preliminary findings about a program or project s impact on education outcomes; -- Only four of these found positive effects! 15 were scientifically rigorous impact evaluations that t are currently under way and have yet to report results; 65 fell into the third level of the hierarchy, that is, they were less rigorous evaluations of program or project impact, such as pre-post studies, comparison group studies without careful matching, or randomized controlled trials with important design flaws 25 were not impact evaluations, that is, they did not seek to measure a program or project s impact on education outcomes and so fell outside the hierarchy.
Establish coherent goals K-12 Education -- Student Learning Prepare all students with the science, technology, engineering, and math skills needed to succeed in the 21st-century t technological l economy -- Teacher Quality Recruit and retain teachers with majors or minors in STEM fields and increase the content knowledge of current K 12 STEM teachers. -- Engagement Increase students engagement in STEM and their perception of its value to their lives.
Goals Postsecondary education -- STEM Workforce Increase the number of undergraduates who enroll in and complete STEM degree programs and are prepared to enter STEM or STEM-related careers or advanced education. -- Collaborative Communities Encourage and support STEM professional collaborations, networks, communities, and alliances among educators, students, practitioners, government, professional organizations, and industry. -- Institutional Capacity Support advancement and development of STEM personnel, programs, and infrastructure in education institutions.
Goals Informal education -- Public Awareness In the context of informal education and outreach, increase the awareness, interest, engagement, and understanding of STEM concepts, processes, and careers of the general public and other target populations. -- Professional Audiences Improve practice and build professional and institutional capacity by funding efforts that generate, develop, and apply innovative ideas and models for the informal science education field.
Establish aligned metrics K-12, e.g., -- Percentage of college students who took remedial or developmental courses in mathematics during their freshman or sophomore years Teacher Quality, e.g., -- Percentage of middle and secondary school students whose mathematics and science classes are taught by teachers with (a) a major and (b) a minor in the subject being taught Engagement, e.g., -- Number of students who major in STEM fields in college
Recommendations of the ACC 1. Inventory, goals and metrics should be updated regularly and used to facilitate stronger interagency coordination 2. Foster knowledge of effective practices through h improved evaluation and-or implementation of proven-effective, research-based instructional materials and methods 3. Improve the coordination of their K 12 STEM education programs with states and local school systems 4. Adjust program designs and operations so that programs can be assessed and measurable results can be achieved 5. Funding for federal STEM education programs should not increase unless a plan for rigorous, independent evaluation is in place, appropriate p to the types of activities funded 6. Agencies with STEM education programs should collaborate on implementation of ACC recommendations under the auspices of the NSTC
The follow-up The Education Subcommittee of the National Science and Technology Council is responsible for coordinating the recommendations of the ACC The initial actions of the subcommittee have focused on enhancing program evaluation and creating metrics that can be shared across programs
Required evaluation template for highleverage programs across all of government General Information: program name, description and discussion of why the program is high-leverageh Outcome Measures: national and common metrics (agencies were encouraged to use metrics included in the ACC report and, if others were used, to describe and justify them.) Description of How Measures are Used in Program Operations: frequency of reporting, methods to ensure quality, etc. Evaluation: information on whether the program has been evaluated in the past, plans for future evaluation, key research questions, which tier of the ACC-specified hierarchy h of study designs the planned evaluation resides (experimental, quasiexperimental, or other designs), and methods for ensuring the quality of the design, including securing external evaluators with appropriate expertise Disseminating and Using Evaluation Results: information on how agencies use findings in the design and/or operation of the program and to enhance program assessment Overall Agency Progress on Implementing ACC Recommendations: information about how agencies are implementing evaluation for the portfolio as a whole (i.e., beyond the high-leverage h program)
The challenging g process of developing an evaluation plan Linking program goals to evaluation questions Identifying appropriate metrics Developing comparison groups Planning for effective dissemination and use of evaluative information
Let s work an example NASA s Undergraduate Student Research Project -- Goal: Help prepare and enable students to become members of NASA s future workforce -- Population Served: Undergraduate d sophomores through h seniors -- Intervention: 10-15 week internship at a NASA facility Evaluation Plan -- Outcome or Metric? -- Design (comparison group)? -- Main and supplemental questions? -- Implementation issues?
The results of the process 10 of the 16 programs created a plan to use either randomized control trials (ACC Tier I) and/or quasi-experimental designs (ACC Tier II) to evaluate their programs Agencies expressed needs for resources: (1) a method to track students across their educational trajectory and into the workforce to evaluate the effect of STEM training; (2) information on the implementation of interventions, so that agencies can not only identify effective approaches but also learn about the best ways to ensure their successful application; (3) common or shared metrics so that similar programs, or program components, can be benchmarked against each other; (4) the ability to align curriculum with actual test items (rather than standards or test frameworks) so that the effects of educational programs can be directly measured; and (5) technical and financial resources to plan and carry out evaluations.
Conclusions Federal investments in STEM education are fractionated and poorly coordinated There is little evidence that Federal STEM programs are producing the desired results Investments should be guided by clear goals Practical metrics are needed to measure progress against national goals Expensive and important Federal programs in STEM education should be rigorously evaluated
Policy issues for discussion STEM education initiatives are supply side. What about demand? In a global economy, resources are obtained from the lowest cost providers. Are our costs for educating scientists and engineers too high for us to retain our competitive advantage? What is the relative role of the education of our workforce vs. our business climate in fostering growth in knowledge-intensive industries?
median total compensation 2002 120000 100000 80000 60000 40000 20000 0 math life sciences human lawyer resources
Job Outlook Scientific R&D Bureau of Labor Statistics -- Wage and salary employment in scientific research and development services is projected to increase 9 percent between 2006 and 2016, compared with 11 percent employment growth for the economy as a whole. -- While demand for new R&D is expected to continue to grow across all major fields, this industry will continue to experience rapid productivity growth as a result of advances in computer and communications systems, reducing employment opportunities. Increasing international competition should also dampen employment growth.