The Education Program of Space Telescope Science Institute s Office of Public Outreach

Transcription

1 The Education Program of Space Telescope Science Institute s Office of Public Outreach INTRODUCTION Two decades ago, the Space Telescope Science Institute (STScI) posed for itself this question: Could a small group of scientists and educators transform the breathtaking discoveries from the Hubble Space Telescope into meaningful and measureable education programs for America s schools, reaching millions of kids at a time? To undertake such an audacious goal required developing a new model for education and outreach. First Hubble had to build a partnership between forefront scientists and education specialists. Second this partnership had to develop relevant, effective and fieldtested educational tools. And finally to bring these tools into school classrooms and effect change a millions kids at a time, using only a small percentage of the Hubble budget, STScI became an early adopter of web-based technologies and developed partnerships that leveraged off existing educational structures and programs. The STScI education program was one of the first to exploit the internet and build relationships with teacher training institutions and state level Departments of Education. This strategy has allowed STScI s Office of Public Outreach (OPO) to bring cutting-edge science directly to the education community on a national scale. Today STScI science education materials are used in all 50 states and have been adopted by more than half the state departments of education in the United States. Each one of STScI s educational programs and products, whether for use in formal or informal education, is held to the same high standard: Each is aligned with national education standards and pedagogy, tested in an appropriate setting, independently evaluated, and its impact measured. These standards-based classroom-ready materials are used by ½ million educators and 6 million students each year, a significant fraction of middle school teachers and students in the United States. We have achieved our goal of having a program with national reach at a cost of about 25 per student per year. 1

2 OVERVIEW OF STSCI EDUCATION ACTIVITIES The OPO Education Program is standards-based, targets both formal and informal education audiences, and is designed to support connectivity between the classroom and out-of-classroom learning. The span of STScI s educational activities is represented in Figure 1. Standardsbased Curriculum Support Tools (online, hardcopy & mobile platform) Professional Development Student Activities Exhibits & Standardsbased Exhibit Support Materials Interns/Externs Education Community Support & Community Events Unique Content, Educator-Scientist Teams, Research-based, Evaluation (internal/external), Partnerships, Diversity Figure 1 The education program at STScI spans a wide variety of activities. Each education product, whether for formal or informal education, is standards-based, field-tested and evaluated for impact. Formal education efforts target the K-12 community, and include a suite of curriculumsupport materials as well as pre- and in-service professional development. Curriculumsupport materials range from online interactive activities (e.g., Hubble Deep Field Academy) to information graphics (Graphic Organizers) and misconception pieces (Myths vs Realities), to Star Witness News literacy materials. Professional development workshops share cutting edge Hubble discoveries, fundamental STEM topics, and real-world connections with pre-service and in-service educators to prepare them to integrate current science into their classrooms using trends from the latest educational research. In addition to working with educators, the program has a student intern component designed to provide STEM content and work experience for high school students. We partner with local school students who are recommended to our program. They become active members of the education team, and participate in STEM work experience. 2

3 Informal education efforts focus on forging links with science centers, planetariums, natural history museums, libraries and similar forums patronized by public audiences seeking to broaden their understanding of science. These efforts include developing exhibits designed to support informal science education programs and bring Hubble science to public audiences. Informal education products and programs are standardsbased and evaluated, just as in formal education. Examples include the Visions of the Universe traveling exhibit for libraries, and ViewSpace multimedia exhibit for museums. We support community outreach events with standards-based, field-tested activities. This component of the program includes inspire-level efforts, as defined by NASA s Strategic Education Coordination Framework and the Science Mission Directorate s stated outreach activities, with the goal of reaching out to the education community with student activities and events. Our community efforts serve to connect HST to the efforts of other organizations and groups, as well as to specific audiences and the general public. The Science Educator Partnership STScI s Office of Public Outreach is an integrated team that includes professional educators, outreach scientists, news professionals, web designers, software developers and other technologists. Every element produced by OPO starts with the science. For example, a science discovery is announced by a press release from the News Office, enters the formal education chain through our education program, is disseminated to the science-interested public in museums and science centers through ViewSpace, and is shared with the public through HubbleSite. The success of this team to communicate the excitement of Hubble discoveries requires an intimate partnership of education professionals and scientists. The Office of Public Outreach has education professionals with more than 150 years of combined experience, and outreach scientists. In addition, experts in virtually any area of astronomy are close at hand, as this team is embedded in an active research environment of over 100 scientists. STScI s research scientists and other staff members are quite active in bringing science into schools and the community. Communicating with students and the public has become part of the culture for Hubble and JWST scientists, as with all of SMD s missions. This commitment to communication is so deeply embedded that it has been called NASA s social contract with the nation. In fact, one prominent young scientist came to STScI in part because of OPO: he felt he would become a better science communicator if given the opportunity to work with the OPO team. Focus on Middle School The OPO Education Program focuses on working with middle school students because this is the time when student interest and achievement in science begins to decrease. Research studies indicate that by high school students are largely turned off and see science as a burden. 1 It has been noted that there is a decline in positive attitudes towards science among students from ten years old which continues throughout high school. Middle school is a critical time for the STEM pipeline. The National Science Teachers Association (NSTA) claims that, for students, the middle school years are a pivotal time 3

4 in their understanding for science if educators don t capture students interest and enthusiasm for science by grade 7, students may never find their way back to science. 2 Students often see the curriculum as being too theoretical and distant to their personal experiences. 3 This is the time when students need to have hands-on, real-world experiences in science to help maintain interest and encourage consideration of STEM careers. HST and JWST content provide the opportunity for real-world experience, the opportunity to use science process skills, and aligns well with national education standards taught at the middle school level (e.g., electromagnetic spectrum, galaxies, and the relationship between new technologies and the advancement of science). STScI leverages the science and engineering of the missions to provide the education community with the resources which are necessary for educating, engaging, and inspiring the middle school student population. STScI s education programs build on the natural appeal astronomy holds for children. Early learners are taught process skills to prepare them for more structured science study later on in middle school. Amazing Space content is interactive, inquiry-driven and involves real data. Students observe, classify and analyze with real data, like scientists. As active learners, they remain engaged and have the opportunity to see themselves as real scientists. These are the elements that have been proven to encourage them to enter STEM fields. DEVELOPMENT AND TESTING OF PRODUCTS AND PROGRAMS Every formal and informal educational product or program produced by STScI is standards-based, field-tested and evaluated for impact. We have a well-defined and rigorous development model, as shown in Figure 1. Needs Assessment Make a Design Plan Make a Prototype Pilot Test Field-Test NASA Product Review Dissemination To Partners Impact Reach Figure 2 Development and evaluation model. Every formal and informal educational product follows a rigorous development model. Every product is standards-base, field-tested and evaluated for impact. 4

5 NEEDS ASSESSMENT AND FORMATIVE EVALUATIONS The first step in the development and evaluation model begins with a needs assessment that includes three important components: Determine the needs of the target audience. For example, for the library exhibit Visions of the Universe, for example, input is sought from the American Library Association. Perform an evidence-based literature search to gather an accurate picture of the range of existing materials, follow trends and developments, and identify areas of need. Identify alignment with national education standards. The next step is to make a Design Plan. It must have well-defined goals and objectives which must be measurable. If it cannot be measured, the design is not pursued. A logic model is an evaluation planning tool to clarify what a project intends to do and what it hopes to accomplish. It relates the goals/objectives to some or all of the following elements: inputs, activities, outputs, outcomes, impact. This step is followed by a prototype of the product. At this stage, the product is reviewed by a development team, which is usually internal. The team ensures pedagogic and scientific accuracy, grade level assignment, and alignment to national education standards. Its feedback is used as diagnostic input to improve the design. In the next step, a completed product version is tested on a small target audience as a pilot test. This formative assessment, or diagnostic testing, is used as feedback and the product is revised as necessary. When the product has been revised and is deemed ready for the next step, it is tested on a representative sample of external users in a field test. The product is revised as appropriate based on the results of the field test. A list of evaluations performed or contracted by the Office of Public Outreach (OPO) is given in Appendix A. Where evaluations are performed by OPO, they are done by individuals who were not involved in the design in order to maintain separation. Finally, when the product is deemed ready for dissemination, it is submitted for NASA Earth and Space Science Education Product Review, carried out by the Institute for Global Environmental Strategies (IGES). This is a requirement for all NASA SMD education programs. Review criteria include relevance, accuracy, alignment to National Education Standards, effective instructional practices and appropriate student assessment. (These reviews are not listed in Appendix A.) PARTNERSHIPS Through these steps, there is ample evidence that the materials produced are of high quality and will achieve their intended goal when placed in the right hands. The next step in the development and evaluation model is dissemination to partners. Strategic partnerships are key to fulfilling our goal of having a national reach. In order to have the greatest reach in a cost-effective manner, a key strategy has always been to form strategic 5

6 partnerships with other programs and organizations to reach new audiences and to integrate new techniques into our programs without reinventing or duplicating efforts. We use a top-down approach and work with colleges and universities, state education initiatives, school districts, and informal venues to ensure appropriateness, wide dissemination, and great impact. We also develop and maintain partnerships with industry, non-profit organizations, etc., that result in the creation of products and programs and their dissemination on a national scale. We enter into partnerships based on specific criteria: Does the institution complement the work we do? Does the partner serve underserved/underrepresented populations? Does the partner demonstrate sustainability and can it carry on independently? Does the partner collect follow-up data collection (summative evaluation) that it is willing to share with us? Although the STScI education team is quite small, through its network of over 500 partners, it has been able to demonstrate very broad reach and high impact. Examples of partners include: NASA programs, such as NASA Explorer schools and Solar System Ambassadors Organizations, such as National Federation for the Blind, Space Grant Consortia, American Library Association, World Wide Telescope, the Maryland Science Center, Learning.com and the Ohio Resource Center for Mathematics, Science and Reading (ORC). Learning.com is an online repository that enables educators to locate standards-based resources. It reaches 150,000 educators nationwide. The Ohio Resource Center for Mathematics, Science and Reading (ORC) recommends Amazing Space to prepare 130,000 5 th grade students for a statewide science test. Universities, such as the Johns Hopkins University Center for Technology Education which will integrate our materials into the Boundless Learning Program for STEM; and University of California/Berkeley Lawrence Hall of Science GEMS Space Science Sequence for grade 6-8, which reaches 25,000 students across the US. Textbook publishers, such as McGraw-Hill, Houghton Mifflin, and National Science Teachers Association SciGuides State Departments of Education: Our materials have been adopted by the Departments of Education in 27 states, but are used in all 50. This will be described in more detail in a later section. The current list of partners is given in Appendix B. DIVERSITY Diversity is an essential element of our program. We seek to ensure that our audiences are diverse, reflecting the various communities that make up the United States and the desired future demographic of our technical workforce. OPO embeds diversity into its programming efforts, based on a plan developed in 2006 and approved by NASA. 6

7 Our partnerships are key to bringing our materials to underserved or underrepresented groups. Partners and specific programs targeting underrepresented groups include: JWST SIP Project Tactile Astronomy Women s Science Forum STEMcx Baltimore City Public Schools (professional development & SABES curriculum project) The After-School Institute (professional development for Baltimore City afterschool programs) Visions of the Universe traveling exhibit Girl Power event Expanding Your Horizons event Support for Baltimore City s Project Astro program Maryland MESA Partnership/Pre-College Fair Morgan State University - Innovative STEM Conference Delaware AeroSpace Education Foundation (professional development for rural teachers w/limited access to NASA) OPO high school internship program New Partnership with the Family League in Baltimore City Baltimore City STEM Steering Committee IMPACT After dissemination of our products, STScI generally has no direct access to students to confirm that the intended outcomes are achieved in the classrooms. This type of summative evaluation requires the cooperation of our partners. Various evaluation types are appropriate for different purposes and may include a quasi-experimental design that compares test scores of program participants with students in matched comparison groups...or a descriptive study, incorporating such techniques as surveys. 4 There are three ways that STScI uses to assess the impact, or establish the quality or effectiveness, of our products: Learning outcome study Impact study/follow-up Adoption by state departments of education, school districts, and professional education associations. We discuss each in turn in the sections below. Learning Outcome Study The learning outcome study comes closest to a controlled experiment in the classical sense: One group of students is taught a lesson with materials developed by STScI, and a second control group is taught without those materials. Analysis of the results reveals the impact of the materials. These types of assessments have been done only rarely by STScI for a variety of reasons - They are quite expensive for the limited OPO budget 7

8 (~$60k), they require students and teachers that are not within the control of STScI, and such tests can be rather disruptive. Similar impediments are acknowledged by the US Department of Education, which recommends combined formative and summative evaluations to meet multiple demands 4. STScI commissioned an external evaluator, Mid-continent Research for Education and Learning (McREL), to perform a learning-outcome study of Planet Impact. This was a quasi-experimental design that used pre-and post-assessments to measure the impact of Planet Impact materials on student learning. McREL found a statistically significant improvement indicating that the average student s performance in the treatment group would be 13 percentile points higher than the average student s performance in the control group on a standardized test. 5 The difference between the average pre- and postscore was highly statistically significant. 5 STScI s informal education products are developed using the same model as the formal education products. Cornerstone Evaluation Associates is in the process of completing an evaluation of our Library exhibit, Visions of the Universe. Preliminary results demonstrate an increased interest in astronomy books with a ~30% (TBR) increase in book check-out rates following the exhibit. Impact Study/Follow-up This is by far the most common form of impact study. In order to follow up on where, how and why OPO education products are used, we do interviews and surveys on an ongoing basis with a statistically significant subset of venues that use our materials. A map of these venues is shown below. OPO education materials are used throughout the United States, more widely than our ongoing sampling of ~1400 venues (including more than 200 colleges, 300 school districts and 100 informal science education venues.) These are sufficient to assure that these materials remain useful and effective. 8

9 The image part with relationship ID rid12 was not found in the file. Figure 3 STScI carries out an ongoing program of interviews and surveys of over 1400 representative venues (including colleges, school districts and science centers) that use STScI s education products, giving evidence that these materials remain useful and effective. Adoption by a State Department of Education Adoption by a state department of education represents the gold standard in validating the quality of our products and establishing impacts. More than half the state departments of education in the United States use our materials as a required or recommended element. Below are examples of education programs adopted by state departments of education: STScI s Amazing Space website is integrated into Ohio s required preservice educator training program, reaching over 20,000 pre-service educators annually. STScI s Amazing Space s Telescopes from the Ground Up is integrated into South Carolina s grade 8 standards, reaching an estimated 55,000 students annually. Amazing Space is defined in the Astronomy and Space Exploration section of Louisiana s Comprehensive Curriculum for Grade Eight Science, reaching about 52,000 students annually. Amazing Space is identified as a Core Curriculum Internet Resource for all grade 6 students in the state of Utah, reaching an estimated 45,000 students annually. An illustration of what is involved in adoption by a state department of education is summarized in the box below. 9

10 Sample data for impact study follow- up: Use by a State Department of Education Who Utah State Office of Education Users 6 th grade students (~ 55,000 per year) Identified Resource Hubble Deep Field Academy online activity How it is used Identified as a Core Curriculum Internet Resource for all grade 6 students in the Utah Elementary Science Core Curriculum. About the Utah Elementary Science Core Curriculum: The Elementary Science Core Curriculum describes what students should know and be able to do at the end of each of the K 6 grade levels. It was developed, critiqued, piloted, and revised by a community of Utah science teachers, university science educators, State Office of Education specialists, scientists, expert national consultants, and an advisory committee representing a wide variety of people from the community. The Core reflects the current philosophy of science education that is expressed in national documents developed by the American Association for the Advancement of Science, the National Academies of Science. Criteria for selection and use: - Matches Utah state education science standards, e.g., o Standard : Students will begin to investigate the nature of the universe. o Standard : Explain the relationship of the vastness of space to objects found in space. o Standard : Explain how scientific development changes knowledge of space. - The philosophy of Utah s core curriculum is that students should be active learners. It is not enough for students to read about science; they must do science. They should have hands on, active experiences throughout the instruction of the science curriculum. While doing the Hubble Deep Field Academy activity, students are active learners. Like scientists, they explore the HDF image, make observations, inquire, formulate and test hypotheses. They record, analyze data, report, and evaluate findings throughout the activity, and compare their results to astronomers results. As testament to the quality of our education products, 27 state departments of education have adopted our materials: 9 as required, and 18 as recommended resources. They are listed below: Required use of STScI materials Michigan Alaska South Carolina Utah California Georgia Kentucky Louisiana Texas Recommended use of STScI materials Hawaii North Carolina Vermont Virginia Washington Arkansas Colorado Delaware Florida Maine Maryland New Jersey Idaho Kansas Massachusetts Mississippi New Mexico New York Table 1 State Departments of Education that require or recommend use of STScI education materials. 10

11 REACH AND LEVERAGING The final step in the full development process is to estimate reach. We have distinguished reach explicitly from impact, because estimation of a project s impact on educational outcomes is often evaluated by Randomized Controlled Trials, quasi-experimental designs, and other designs such as pre-post studies 6. The Report of the Academic Competitiveness Council assumes that new middle school science curriculum elements introduced at a limited number of middle schools would have little influence because the scale of intervention would be so small, and that collective federal investment would be required 6. STScI has taken a proactive approach to leveraging so that a small team serves a big country. As a consequence, there is good reason to believe that our products reach a significant fraction of middle school students and middle school teachers across the United States. We believe a complete picture cannot be obtained without considering the reach of our education programs and products as well as their impact. Our approach to leveraging is seen in three areas, described below. Detailed derivations of the reach metrics described below can be found in a companion document Education Metrics for Space Telescope Science Institute s Office of Public Outreach. 7 Master teachers. We target middle school master teachers for our professional development workshops, in which they become proficient in use of our education materials. These master teachers each agree to teach two classes of (predominantly) middle school science teachers. These middle school science teachers in turn take their experience with STScI materials into their classrooms where they work with their middle school science students. Last year we trained approximately 1,100 master teachers. The leveraging factor of master teachers thus allows STScI materials to potentially reach over 55,000 middle school science teachers and 1.7 million middle school students through this means alone. The power of this approach has been recognized by the White House in a new initiative to launch a pilot program for the STEM Master Teacher Corps. 8 Partnerships with high-leverage institutions. By strategic partnerships, STScI places its materials in the hands of those who can best use them, and best disseminate them, for maximum effect and broad reach. This is illustrated by the wide use of STScI materials by state departments of education. A specific example of a strategic partner can be found in the Lawrence Hall of Science s GEMS Space Science Sequence for grade 6-8. The GEMS series of teacher guides provide educators with flexible, stand-alone units that can be used as enhancements for existing curriculum, or they can be used in combination to provide comprehensive coverage of a topic. These guides are distributed to in-service classroom educators nationwide. STScI s Hubble Deep Academy activity is integrated into this program. Through this program alone, STScI s materials reach an estimated 25,000 students. The value of strategic partnerships can be seen by the President s STEM Innovation Networks initiative 8, which helps school districts build strategic partnerships with science agencies, educational entities and other organizations. 11

12 Online access / high distribution platforms. STScI was an early adopter to the use of the internet and web-based distribution for our education products. This has enabled us to achieve the greatest reach in a cost-effective manner, providing access by teachers and students (including homeschoolers) across the nation. In addition, online learning offers benefits to enhance traditional classroom teaching. STScI s computer-based materials, such as Planet Impact, provide students with interactive, inquiry-based learning and allow students to self-test what they learn. These strategies have been shown to enhance student learning. 9,10,11 STScI s online site for education materials is Amazing Space. This is an award-winning website that includes a comprehensive collection of standards-based science education materials through Online Explorations, as well as A Hubble Gallery, Tactile Astronomy, and Tonight s Sky. In a special area of the site designated For Educators, teachers are able to locate materials and activities organized by topic and type to serve the specific instructional goals they have in mind. In addition, this area of the site contains support materials such as science background information, and overview pages that provide tips and strategies for using Amazing Space activities in the classroom. We estimate that approximately 3 million students and 100,000 teachers use Amazing Space each year 4. It is important to note that OPO cannot push these materials to teachers and students to be effective they must be pulled by teachers and educational institutions, and therefore must be shown to be relevant, aligned with education standards, and can be used effectively in the classroom. Their use validates their value. The value of online resources such as STScI s Amazing Space is seen by the President s investment in the STEM Virtual Learning Network, a national, online community of STEM educators allowing exchange of STEM education materials and best practices. 8 (ref 21 st century) Many teachers use, and even prefer, hardcopy materials, and OPO has distributed more than 1.2 million education items. This broad distribution has had an enormous impact, and we estimate that over 400,000 teachers currently use them, engaging over 3 million students. It is worth mentioning that online products have been very useful in informal education venues. STScI s ViewSpace is an internet- fed, self- updating, multi- media exhibit developed for museums and other informal science venues. ViewSpace shows feature the latest images and digital animations from astronomy and earth science, and include interpretive text. Since ViewSpace brings accurate, new content that is updated daily, it remains a new exhibit for museums, which is a very important feature. ViewSpace is supported by a connected suite of standards- based educational resources. It is currently exhibited in 215 museums, science centers and planetariums whose total annual attendance is about 29 million visitors. We estimate nearly 9 million viewers of ViewSpace annually. 12

13 SUMMARY METRICS Below is a top-line summary of the metrics we measured last year relating to impact and reach for STScI s Office of Public Outreach. A detailed discussion of each element can be found in a companion white paper. 7 For completeness, we include metrics for Public Outreach and News. Although these do not fall formally into the category of education, they do address the national goals of student engagement and public awareness. 6 For context, there are approximately 10 million students in public middle schools. Our estimates suggest through our formal education programs we reach about half this number. We have achieved our goal of having a program with national reach at a cost of about 25 per student per year. FORMAL EDUCATION 6 million students, ½ million teachers per year STScI Formal Education Metric Leverage K-12 students: Direct 2,000 Interactions K-12 Teachers: Direct Interactions 1,100 Through Master Teachers, a further 55,000 teachers and 1.7M students reached K-12 engaged * 6.7 million Materials used in all 50 states, integrated into programs of more than half the state departments of education K-12 teachers engaged 520,000 STScI s Amazing Space website is integrated into Ohio s required pre-service educator training program, reaching over 20,000 pre-service educators annually * Learning outcome study by McREL indicated 13 percentile points improvement. INFORMAL EDUCATION (e.g., museums, libraries) 9 million people per year STScI Informal Education Metric Measurement Example Participants/Observers 9 million Preliminary result by Cornerstone Evaluation, Inc. of library program evaluation shows 30% increase in astronomy book check-out rate PUBLIC OUTREACH 24 million people per year STScI Public Outreach Metric Measurement Example Outreach: Direct Interactions 46,000 Outreach 24 million HubbleSite receives 2 million visits per month (10% of NASA s online traffic) NEWS over 100 million potential readers every two weeks STScI NEWS Metric Reach metric Circulation Press Releases 28 per year Average circulation per release 140 million Online Articles 3,300 per year Total circulation per year 3.8 billion 13

14 REFERENCES 1. Festerwald, John. Middle School Science Critiqued Silicon Valley Education Foubdation, 22 March 2012 ( 2. Science Education for Middle Level Students, NSTA Position Statement, Feb 2003 ( 3. Porter, Cliff and Parvin, Joy. Learning to Love Science: Harnessing Children s Scientific Imagination. A report from the Chemical Industry Education Centre, University of York, September ( 4. Evaluating Online Learning: Challenges and Strategies for Success, US Department of Education, A Study of the Effects of Space Telescope Science Institute s Planet Impact Curriculum: Final Evaluation Report, Mid-continent Research for Education and Learning, 2004, pp Report of the Academic Competitiveness Council, U.S. Department of Education, May 2007 (see Figure 2 of Report of the Academic Competitiveness Council, US Department of Education, See Figure 2 and page Education Metrics for Space Telescope Science Institute s Office of Public Outreach, ( 8. Preparing a 21 st Century Workforce: Science, Technology, Engineering, and Mathematics (STEM) Education in the 2014 Budget, OSTP, April 10, Wise, Kevin C Strategies for teaching science: What works? Clearing House 69: Christmann, Edwin P. and Badgett, John L A meta-analyttic comparison of the effects of computer-assisted instruction on elementary students academic achievement Information Technology in Childhood Education Annual 15: Azevedo, Roger and Bernard, Robert M A meta-analysis of the effects of feedback in computer-based instruction. Journal of Educational Computing Research 13:

15 APPENDIX A. EVALUATIONS CONSULTANT EVALUATIONS Comprehensive Evaluation Cornerstone Evaluation Associates conducted a comprehensive evaluation on the Hubble Space Telescope Education Program for the Space Telescope Science Institute s Office of Public Outreach Review, Development and Field testing The review, development and field testing of an inquiry-based classification activity (Currently called the Mixed Up Solar System) were conducted by an AAAS Consultant Field Testing Lawrence Hall of Science conducted field testing on the Amazing Space Hubble Deep Field Academy activity included in one of their GEM Guides. Needs Assessment An education consultant conducted a needs assessment of Student Career Choices through a literature review of current research Review, Field testing & Revision ( ) The review, field-testing, and revision of inquiry-based Hubble Deep Field Academy were conducted by an AAAS Consultant. Field-testing study ( ) Designed Instruction. Inc. conducted a research and evaluation study of the Amazing Space interactive activity Telescopes From the Ground Up Impact Study WPI at Virginia Tech conducted an impact study of the Space Telescope Science Institute s public outreach website, HubbleSite Readability Study and Field Testing ( ) The junior version of the hard-copy Solar System Trading Cards were used in a readability study and field tested with second and third grade students by Space Sciences Laboratory at Berkeley, Sun-Earth Education Forum. 15

16 Learning Outcome Study ( ) A Study of the Effects of Space Telescope Science Institute s Planet Impact Online Activity to determine the effectiveness of an on-line supplemental curriculum activity and its impact on student learning by Mid- content Research for Education and Learning (McREL). Impact Study The IDEAS Retrospective was an impact study of the IDEAS (Initiative to Develop Education through Astronomy and Space Science) grant program administered by the Space Telescope Science Institute s Office of Public Outreach. The retrospective was conducted by a review panel consisting of educators and scientists from the E/PO community Formative Evaluation McREL conducted a formative review of Behind the Hubble Pictures: Classroom Activities on Light and Color Educator Guide. Formative Evaluation McREL conducted a formative review of the Educator Guide that contains activities that use the Whirlpool Galaxy and the Warped Galaxy lithographs. Formative Evaluation McREL conducted a formative review of the Stellar Evolution Guide that contains activities associated with a star s life and death Planet Impact Field Testing The online interactive classroom activity: Planet Impact was field tested by an education consultant with students to determine whether the concepts and graphics were ageappropriate. The results were used to revise the activity Field Testing of Galaxies Galore, Games and More and No Escape: The Truth about Black Holes activities, Galaxies Galore and Black Holes in Anne Arundel County Schools. activities, Galaxies Galore and Black Holes in Baltimore County Schools. activities, Galaxies Galore and Black Holes in Baltimore City Schools. 16

17 activities, Galaxies Galore and Black Holes in Carroll County Schools. activities, Galaxies Galore and Black Holes in Garrett County Schools. activities, Galaxies Galore and Black Holes in Harford County Schools. activities, Galaxies Galore and Black Holes in Howard County Schools. activities, Galaxies Galore and Black Holes in Montgomery County Schools. activities, Galaxies Galore and Black Holes in Prince Georges County Schools. activities, Galaxies Galore and Black Holes in Talbot County Schools. activities, Galaxies Galore and Black Holes at the University of Maryland Baltimore County with pre-service educators Field Testing of Star Light Star Bright activity, Star Light Star Bright in Anne Arundel County Schools. activity, Star Light Star Bright in Baltimore County Schools. activity, Star Light Star Bright in Baltimore City Schools. activity, Star Light Star Bright in Carroll County Schools. activity, Star Light Star Bright in Garrett County Schools. 17

18 activity, Star Light Star Bright in Harford County Schools. activity, Star Light Star Bright in Howard County Schools. activity, Star Light Star Bright in Montgomery County Schools. activity, Star Light Star Bright in Prince Georges County Schools. activity, Star Light Star Bright in Talbot County Schools. activity, Star Light Star Bright at the University of Maryland Baltimore County with pre-service educators Field Testing of the Hubble Deep Field Academy, Solar System Trading Cards and a Stellar Evolution activity activities: Hubble Deep Field Academy, Solar System Trading Cards and Stellar Evolution in Anne Arundel County Schools. activities: Hubble Deep Field Academy, Solar System Trading Cards and Stellar Evolution in Baltimore County Schools. activities: Hubble Deep Field Academy, Solar System Trading Cards and Stellar Evolution in Baltimore City Schools. activities: Hubble Deep Field Academy, Solar System Trading Cards and Stellar Evolution in Carroll County Schools. activities: Hubble Deep Field Academy, Solar System Trading Cards and Stellar Evolution in Garrett County Schools. 18

19 activities: Hubble Deep Field Academy, Solar System Trading Cards and Stellar Evolution in Harford County Schools. activities: Hubble Deep Field Academy, Solar System Trading Cards and Stellar Evolution in Howard County Schools. activities: Hubble Deep Field Academy, Solar System Trading Cards and Stellar Evolution in Montgomery County Schools. activities: Hubble Deep Field Academy, Solar System Trading Cards and Stellar Evolution in Prince Georges County Schools. activities: Hubble Deep Field Academy, Solar System Trading Cards and Stellar Evolution in Talbot County Schools. activities: Hubble Deep Field Academy, Solar System Trading Cards and Stellar Evolution at the University of Maryland Baltimore County with pre-service educators Field Testing of Astronaut Challenge A field test of the online interactive classroom activity Astronaut Challenge was conducted at the Johns Hopkins Center for Talented Youth with sixth to ninth graders K-12 Educator Needs Assessment An education consultant conducted an educator needs assessment to determine the level of computer access and educator knowledge and usage of computer in classroom. The method of data collection was surveys and interviews EVALUATIONS BY STSCI OFFICE OF PUBLIC OUTREACH STAFF Amazing Space Impact Study The purpose of the impact study is three-fold. First, collected data is utilized to determine where and how Amazing Space curriculum support tools are used. Secondly, the data is used to determine what criteria are employed for curriculum support tool selection. Third, the collected data provides a baseline for monitoring growth in the use of curriculum support tools over time. In addition to this, the impact study affords the possibility of establishing partnerships with educational institutions using Amazing Space 19

20 materials in order to enhance product dissemination. Initially, information about the educational institutions using Amazing Space materials was collected by using online search engines and monitoring requests for hard copy educational materials. Additional methods of data collection include web statistic referrals and trends; databases such as ERIC, LexisNexus, and ICON; NASA s NEEIS system; publications; colleagues; professional organizations and institutions; partners who share field-testing data; and targets of opportunity Tactile Astronomy Website Image of the Month. Formative evaluation. The purpose of this study was to conduct formative evaluation on the materials created for distribution on the Tactile Astronomy website. 20th Anniversary Formative Evaluation and Impact The purpose of this study was to conduct a formative evaluation of the 20th anniversary products and also to determine the impact of the 20th anniversary celebration. Student ERO Pilot Project Formative Evaluation The purpose of this study is to conduct a formative study to determine whether this project can be done on a larger scale. HubbleSite Exoplanets Web site Formative Evaluation The purpose was to provide formative evaluation for the Exoplanets website Amazing Space s SM4 website and DVD Formative Evaluation The purpose was to provide formative evaluation of the materials contained in the Servicing Mission 4 website. OPO s IYA projects Impact Study The purpose of this study is to determine the impact of the products and programs created to celebrate the IYA project. This evaluation includes surveys and interviews of the library staff who hosted the Visions of the Universe: Four Centuries of Discoveries traveling exhibit. An evaluation of the IYA Image Unveiling is also underway. ViewSpace Audience Evaluation The purpose of the study is to determine how ViewSpace is being implemented by various venues and the impact it is having upon the informal education community Educator Needs Assessment The purpose of this needs assessment was to determine high school physics educators needs for instructional resource materials. 20