K-12 SCIENCE CLASSROOM ACTION RESEARCH AS EMBEDDED PROFESSIONAL DEVELOPMENT TO IMPROVE STUDENT ACHIEVEMENT IN SCIENCE DISSERTATION

Transcription

1 K-12 SCIENCE CLASSROOM ACTION RESEARCH AS EMBEDDED PROFESSIONAL DEVELOPMENT TO IMPROVE STUDENT ACHIEVEMENT IN SCIENCE DISSERTATION Presented in Partial Fulfillment of the Requirements for The Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Margilee Planton Hilson, M.S. **** The Ohio State University 2008 Dissertation Committee: Professor Kathy Cabe Trundle, Adviser Approved by Professor Donna L. Farland-Smith, Co-adviser Professor Douglas T. Owens Adviser College of Education and Human Ecology School of Teaching and Learning

2 Copyright by Margilee Planton Hilson 2008

3 ABSTRACT This research was an analysis of the influence of participation in a classroom action research program upon student achievement and teacher professional development. This research evaluated three years of data from a district wide teacher action research program in a large urban Midwestern city. Sixty-seven teachers involved in an action research project focused on science instruction were included in this study. The purpose of the action research program was threefold: 1) improve student achievement, 2) identify best instructional strategies for promoting student achievement, and 3) recognize, replicate, and disseminate excellence in teaching. Determination of student achievement gain was conducted through comparing the mean difference between pre- and post project standardized assessment data relative to the school district averages. Standardized assessments such as the Metropolitan Achievement Test Version 8, State Department of Education created achievement tests and District created end of course exams were administered to students annually. The results suggest that teachers who engage in classroom action research may improve student achievement in science as measured by standardized tests. In the 42 cases with complete data sets, the mean student achievement gain above the district average was 3.65 Normal Curve Equivalents and an effect size of.46 was found with a 7.96 standard deviation. ii

4 Dedicated to my father, Herbert Clark Planton and my mother, Georgean Grace Witkoski whose sacrifices and encouragement enabled me to go to school. iii

5 ACKNOWLEDGMENTS I wish to thank my husband; Jeff F. Hilson III, whose patience and encouragement during the past six years was invaluable. I also wish to thank my advisors Kathy Cabe Trundle, Donna L. Farland-Smith and Douglas T. Owens for their steadfast support and wisdom. I am grateful to Sally Hobson, Tiffany Wild, Lori Marshall, and Cindy Schroeder, who were fellow graduate students and loyal members of The Writing Group. iv

6 VITA September 24, 1952 Born - Warren, Ohio, U.S.A B.S. Elementary Education, The Ohio State University 1976 M.S. Family Relations and Human Development The Ohio State University Teacher, Overbrook Weekday Preschool Teacher, Clintonville Academy present.teacher, Columbus City Schools present Regional Value Added Specialist, Ohio Department of Education Major Field: Education FIELDS OF STUDY Minor Field: Research Methods in Human Resource Development v

7 TABLE OF CONTENTS vi Page Abstract... ii Dedication... iii Acknowledgments... iv Vita... v List of Tables... xi List of Figures... xii Chapters: 1. Nature and Scope of the Study... 1 Context History of the Performance Advancement System... 2 Rationale for Action Research as Professional Development in Urban Settings... 4 Problem Statement... 5 Significance of the Study... 5 Research Questions... 6 Definition of Terms... 7 Limitations of the Study Literature Review Teacher as Learner Theoretical Influences on Teacher as Learner Types of Teacher Knowledge Science Education Professional Development Models/Strategies Utilized in Urban Settings Purpose of Professional Development Standards for Effective Professional Development Classification of Professional Development Current Research in Science Education Professional Development Analyses of Large Scale Multi-site Science Education Programs Aligning and Implementing Curriculum Collaborative Structure... 39

8 Examining Teaching and Learning Immersion Experience Practicing Teaching Vehicles and Mechanisms Summary of Professional Development in Science Education Overview of Action Research Classifications of Action Research Unique Characteristics of Action Research Data Collection and Analysis Methods in Action Research Situating Action Research in General Research Rationale for Classroom Action Research in Science Education Classroom Action Research in Science Education Science Action Research Studies Focused on Content Knowledge Science Action Research Studies Focused on Pedagogical Knowledge Science Action Research Studies Focused on Pedagogical Content Knowledge Summary of Action Research in Science Education Chapter Summary Methodology Overview of the Study Participants Context Research Design Conditions of Data Collection Data Sources PAS Participation Records Student Achievement Records Teacher Research Summary Reports Professional Development Records PAS Program Documents National Science Education Standards Data Analysis Quantitative Qualitative Trustworthiness Researcher Role Multiple Data Sources Multiple Voices Limitations of the Study Chapter Summary vii

9 4. Results Research Question #1: How Did Implementation of Teacher Action Research Projects Vary Across Grade Level Bands? Background Participation Results Interpretive Findings Science-Oriented Projects Strategy-Oriented Projects Testing-Oriented Projects Literacy-Oriented Projects Research-Oriented Projects Summary Research Question Research Question 2: What Growth in Teaching Knowledge and Skills Do PAS Teachers Report? Background Pedagogical Knowledge Pedagogical Knowledge of Strategy Refinement Pedagogical Knowledge of Reflective Practice Pedagogical Knowledge of Assessment Pedagogical Knowledge of Parental Involvement Pedagogical Content Knowledge Pedagogical Content Knowledge of Student Inquiry Pedagogical Content Knowledge of Building a Conceptual Framework Pedagogical Content Knowledge of Writing in a Content Area Summary Research Question Research Question 3: Do the Instructional Practices Reported by Teachers Reflect the National Science Education Standards? Background Unifying Concepts and Processes Science as Inquiry Physical Science Life Science Earth and Space Science Science and Technology Science in Personal and Social Perspectives History and Nature of Science Summary Research Question Research Question 4: Do the Instructional Practices Reported by Teachers Reflect the Knowledge and Skills Presented in Other Professional Development Episodes Available to the Teachers? Background Interpretive Findings Summary Research Question viii

10 Research Question 5: What Practical Issues Did Teachers Identify as Having an Impact on Student Science Achievement? Background Interpretive Findings Theme 1: Increasing Student Subject Knowledge Theme 2: Raising Test Scores Theme 3: Constructed Response Replies Theme 4: Improving Process Skills Theme 5: Improving Social Skills Theme 6: Improving Literacy Skills Summary Research Question Research Question 6: What Instructional Practices Did Teachers Utilize with Students to Improve Achievement on Science Assessments? Background Instructional Practice and Student Achievement Results Interpretive Findings Similarities and Differences Summarizing and Note-Taking Reinforcing Effort and Providing Recognition Homework and Practice Nonlinguistic Representation Cooperative Learning Setting Objectives and Providing Feedback Generating and Testing Hypotheses Cues, Questions, and Advance Organizers Summary Research Question Research Question 7: How Do the Student Achievement Outcomes of PAS Teachers Vary? Background Quantitative Results Interpretive Findings Summary Research Question Research Question 8: How Do Program Requirements Influence Implementation? Background Interpretive Findings Interaction with Students Diverse Student Learning Needs Curriculum Constraints Scheduling Limitations Poor Attendance Testing Issues Student Motivation Eligibility for Award Stipend Summary Research Question ix

11 Chapter Summary Conclusions and Discussion Professional Development Student Learning Teacher Learning Teaching Practice Organizational Goals Classroom Action Research Identifying a Problem Making an Intervention Plan and Acting on It Evaluating the Effectiveness of the Plan Personal and Prolonged Engagement Implications Professional Development Classroom Practice PAS Program Recommendations for Further Research Limitations List of References Appendices: A: Summary of Marzano et al. (2001) Research-Based Instructional Strategies as Used by PAS Teachers B: PAS Research Report Writing Prompts C: Professional Development Coding Categories D: Summaries of PAS Science Action Research Projects x

12 LIST OF TABLES Table Page 2.1 Attributes of High Quality Professional Development Factors Impacting the Design of Professional Development Programs Situating Action Research in Inquiry Paradigms by Purpose of Research Overview of Research Questions, Data Sources and Analysis Procedures Initial Coding Fields for the Analysis of Research Summary Reports Enrollment and Completion Rates by School Level Projects by School Level and Focus Professional Development Initiatives Present in 42 PAS Summary Reports Grouped Professional Development Initiatives in 42 PAS Summary Reports Frequency and Success of Instructional Strategies Reported by Teachers on PAS Applications Student Achievement Outcomes by School Level Effect Size by School Level Non-completion Rates by School Level xi

13 LIST OF FIGURES Figure Page 2.1 A Model of Teacher Knowledge xii

14 CHAPTER 1 NATURE AND SCOPE OF THE STUDY This chapter explains the historical context and rationale for this study. It also posits the statement of the problem and the significance of the study. A listing of the research questions, definitions of key terms, and limitations of the study end the chapter. Context of the Study Compliance with Federal mandates as enumerated in the Elementary and Secondary Education Act, also known as No Child Left Behind (NCLB) necessitates that school districts provide highly qualified teachers. Research has shown that highly knowledgeable science teachers are better able to facilitate student learning than less qualified teachers (Hewson, Kahle, Scantlebury, & Davies, 2001; Knight & Wiseman, 2005). However, recent National Assessment of Educational Progress (NAEP) scores seem to indicate that many American teachers are doing a less than adequate job in addressing science education through the National Science Education Standards (NAEP, 2005). Urban districts have fared particularly poorly due to opportunity to learn gaps (Loucks-Horsley, Love, Stiles, Mundry & Hewson, 2003.) As a result, policy makers have embraced teacher professional development as a critical piece in the puzzle 1

15 of implementing national science education reform efforts (Beyer, Delgado, Davis & Krajcik, 2007). Broadly speaking, the goals of teacher professional development revolve around teacher learning. Increasing the skills and knowledge of teachers may occur in several areas; subject content knowledge, pedagogical knowledge, or pedagogical content knowledge (Shulman, 1987). However, identifying the specific knowledge needs of teachers has been the subject of much debate and research because contextual factors of teaching and learning have been treated inconsistently (Abell, 2007; Kennedy, 1991; Loucks-Horsley, et al., 2003). Current models of teacher professional development have had variable success in changing teacher practices to those most associated with improved student achievement (Guskey, 2003). This may be the result of viewing teacher development from a deficit standpoint instead of a transformative stance (Loucks-Horsley, et al.). Information was needed about programs that empower teachers to identify and rectify problems in both teaching practice and student achievement within their science classrooms. History of the Performance Advancement System Federal legislation requires states to generate tracking systems to monitor the progress school districts are making in meeting accountability standards. When one large Ohio urban district fell into its state s School Improvement category of Academic Emergency, it was clear to all concerned that implemented district-wide instructional policies were not meeting the needs of all students and that locally validated practices were needed. A plan was devised to empower teachers to think like a researcher in terms of identifying a question, hypothesizing a solution, and enacting a treatment. 2

16 The Performance Advancement System (PAS) was and still is the program created in this large Ohio urban school district, which allows teachers to engage in classroom action research. PAS has three goals: 1) improve student achievement, 2) identify through classroom research the best instructional strategies for promoting student achievement in the urban school district, and 3) recognize, replicate, and disseminate excellence in teaching. Douglas Reeves, nationally recognized accountability expert, served as facilitator for the PAS development process. Reeves suggested a program design modeled after classroom action research. Guidelines were written requiring participants to select a sample, a State Department of Education accountability area, such as science, mathematics, reading or social studies, and a research-based instructional strategy. Participants were referred to Classroom Instruction that Works: Research-Based Strategies for Increasing Student Achievement by Marzano, Pickering and Pollock (2001) to select a strategy on which to base their action research intervention. Appendix A lists a brief description of each strategy. Parameters were set to allow all members of the teachers union to participate, including teachers, tutors, nurses, psychologists, and speech, physical, and occupational therapists. Sources of pre- and post- project achievement test scores were identified for data analysis to determine gain. At the end of the school year, participants were required to write a short summary report explaining their research questions, actions taken and the results of their actions on student achievement. A local educational testing service was hired to review student assessment data and calculate the achievement gains made by students. The mean class gain between the pretest and posttest assessments for participants was compared to the district gain 3

17 between the same pretest and posttest assessments. Teachers whose students demonstrated achievement results higher than the district average received a cash bonus of $2,000.00; after 2003 the bonus was raised to $2, In school year , in order to earn the cash award, teachers initially had to produce statistically significant results determined by one standard error of measurement. This standard of achievement proved to be too high for novice action researchers so the standard was decreased to.5 standard error. Eventually, all reference to standard error was removed and in school years , and , teachers only had to show gain greater than the district average gain. A plan was made to gradually increase the achievement standard each year until it again reflected statistical significance. Rationale for Action Research as Professional Development Classroom action research can be considered a viable means for professional development for several reasons. Classroom action research allows for maximum accountability in terms of addressing variable contexts and student differences. Teachers in urban settings face the extra challenges of high student mobility, language barriers, generational poverty and the effects of violent crime on students. Moll (1990) suggested that teachers should seek out and integrate into classroom practice learning strategies uniquely situated within the cultures of their students. He termed these cultural resources funds of knowledge. The practical and cyclical nature of classroom action research may be one way that teachers may systematically connect to the specific learning needs and potential of the students. There is a high level of teacher engagement in professional development enacted as classroom action research because the teacher initiates the research question. 4

18 Teachers who choose to participate in classroom action research report feelings of empowerment and increased efficacy to help their students achieve (van Zee, Lay & Roberts, 2003). In light of the negative view that the public media portrays urban teachers, professional development that foregrounds teacher practical knowledge may be viewed as a refreshing morale booster. Problem Statement The Performance Advancement System was instituted as a form of teacher professional development embedded in day-to-day practice. The school district expected that improved teacher knowledge and skills would transfer into improved student achievement. This research is an analysis of the personal professional development enacted by science teachers while conducting classroom action research projects in PAS and the resultant impact on student achievement in science. Significance of the Study Teacher professional development typically has been evaluated in terms of changed teacher attitudes or beliefs about teaching practice (Guskey, 2003). In addition to appraising changes in teacher attitudes and beliefs, this study also explored relationships between professional development as enacted through teacher classroom action research projects and improved student science achievement. Identifying instructional strategies employed by urban science teachers focused on school improvement will contribute teacher practical knowledge to science education canon (van Driel, Beijaard, & Verloop, 2001). Little actual classroom teacher action research has been published (Cochran-Smith & Lytle, 1993); examining teacher research records 5

19 will add to the knowledge base of how teachers interpreted the role of research in actual classroom practice. Research Questions The following research questions guided the design of this study and the analysis of the data: 1. How did implementation of teacher action research projects vary across grade band levels? 2. What growth in teaching knowledge and skills do PAS teachers report? 3. Do the instructional practices reported by teachers reflect the National Science Education Standards? 4. Do the instructional practices reported by teachers reflect the subject, pedagogical or pedagogical content knowledge presented in other professional development episodes attended by the teachers preceding or during the data collection period? 5. What practical issues did teachers identify as having an impact on student science achievement? 6. What instructional practices did teachers utilize with students to improve achievement on state achievement tests, nationally normed assessments, or district created end-of-course exams? 7. How do the student achievement outcomes of science teachers who participated in PAS vary? 8. How do PAS program requirements influence teacher action research implementation? 6

20 Definition of Terms Classroom Action Research Classroom action research is defined here to mean practitioner initiated inquiry into a classroom practice thought to influence student achievement. The inquiry is sustained throughout the school year and follows the cyclical model of problem identification, solution selection, implementation, and evaluation of outcomes. Multiple iterations of the research cycle are necessary throughout the school year informed by student progress toward the achievement goal (Calhoun, 1994). Improved Student Achievement Student achievement was operationalized to mean student scores on Ohio Achievement Tests, Metropolitan Achievement Test Version 8, or school district endof-course exams for high school students. Improvement was the measured gain in summative test scores from prior to current school year for each student relative to the school district mean gain (Columbus City Schools, 2007). Professional Development Professional development was considered to be any intentional sustained activity in which teachers engaged for the express purpose of improving their knowledge and skills to teach students science (Banilower, Boyd, Pasley, & Weiss, 2006). Opportunity to Learn Gaps Curriculum that is truncated by rigid scope and sequence timelines and bound by transmissive teaching practices limits opportunities for developing conceptual 7

21 understanding. Students subject to this type of limited curriculum are said to have gaps in their opportunity to learn science (Loucks-Horsley et al., 2003). Limitations of the Study This study was limited by factors inherent in the ex post facto research design. The design is employed to study events that have already occurred and to seek linkages between known outcomes and pre-existing conditions (Ary, Jacobs & Razavieh, 2002). The research subjects self-selected into the program being evaluated, therefore, outcomes may be the result of peculiarities intrinsic to the research sample. For example, in the years immediately preceding the data collection, a major district-wide Urban Systemic Initiative (USI) sponsored by the National Science Foundation was enacted. Teachers who selected science in PAS very likely also voluntarily participated in the extensive professional development offered through the USI grant. Additionally, self-reported teacher data, in form of research summary reports, was utilized. If the teacher reports were not accurate reflections of the classroom action research, then conclusions drawn from them may be skewed. Gains in student achievement were calculated utilizing student scores on standardized achievement tests. However, summative student achievement tests varied from grade level to grade level; consequently z-scores were utilized to compute gain. Utilizing different achievement tests from one grade level to another highlights the issue of comparable difficulty levels of the assessments, which was not determined. Producing a gain between the Metropolitan Achievement Test and the State Achievement Test may not have been as difficult as showing a gain when the pretest and posttest assessments were both State Achievement Tests. A further limitation 8

22 related to achievement tests was that student performance on standardized assessments was assumed to be a valid appraisal of classroom instruction. Generalizability of the results of this study is limited due to the situated nature of classroom action research (Feldman, 1994). It would be very difficult, if not impossible to replicate the conditions present within a collection of classroom action research projects. Successful application of research outcomes would depend upon the match with students and teachers in other settings. 9

23 CHAPTER 2 LITERATURE REVIEW The purpose of this research project was to investigate the role of classroom action research as embedded professional development for improving student achievement in science. This literature review is organized into three sections. First, a theoretical framework for teacher learning and types of knowledge teachers need to know in relation to their classroom work will be examined. Next, a discussion of professional development models utilized in science education for advancing the skills and knowledge of teachers will be presented. Finally, the application of one form of professional development, classroom action research, as practiced in science education will be reviewed. Teacher as Learner Theoretical Influences on Teacher as Learner Teacher learning will be considered here from the theoretical stance of constructivism. Constructivism, as a learning theory, has been built upon developmental theories such as Piaget s ontogenetic theory of logical thought processes (Phillips, 1969), Hunt s (1978) conceptual level theory, and Loevinger s (Loevinger & Blasi, 1977) ego development theory. The practical applications of constructivism have been manifest through information-processing approaches such as teaching for conceptual 10

24 change (Posner, Strike, Hewson & Gertzog, 1982) and the learning cycle (Lawson, Abraham & Renner, 1989). However, social learning theories such as Vygotsky s sociohistorical learning theory (Vygotsky, 1978) and Bandura s observational learning theory (Miller, 2002) have added a great deal to teacher knowledge about the strong influences of social context on teaching and learning. Employing a cognitive/developmental view of teacher learning permits teaching behaviorally discreet skills, in addition to acknowledging contextual factors (Sprinthall & Thies-Sprinthall, 1980.) Constructivism, when viewed as a personal enterprise, focuses on a few common components. Learners bring a set of preconceived notions and personal theories to every learning opportunity. Information is extracted from the environment, compared to what is known, and is either rejected, accepted as is, or accepted with modification into the learner s knowledge base (Bransford, Brown & Cocking, 2000; Loucks-Horsley, et al., 2003; Woolfolk, 2004). Two levels of conceptual change, assimilation and accommodation, are commonly discussed (Duit & Treagust, 2003). Assimilation occurs when a learner merely applies his existing beliefs and knowledge to a new situation. The information gained is not fundamentally different from currently held beliefs, but rather an enhancement or extension of what is already believed to be true. Accommodation occurs when a learner is not able to apply his existing beliefs and knowledge to achieve a satisfactory answer. When a whole class of problems defies solution within a conceptual system then the student will enact a fundamental change in his/her central conception in order to make sense of the phenomena. Accommodation is a transformation to a new conceptual understanding. Meaningful learning is the term 11

25 applied when students successfully exchange incorrect for correct conceptual knowledge. Through this evaluation process, learners construct meaning from both personal and social experiences. Constructivism, when viewed as a social enterprise, is grounded in sociohistorical learning theory and based on the belief that historical antecedents temper all knowledge (Wertsch, 1991). The key element of social constructivism is that humans construct their knowledge from social interactions with other people, objects, cultural mores, and social institutions. All information assimilated is processed through the lens of prior experience situated in particular social encounters. Sociohistorical theory adds the dimension of historical influence on the social construction of knowledge. Learning and subsequently development occurs on multiple levels, phylogenetic which refers to species level advancement, historical which refers to cultural level changes, ontogenetic which concerns personal growth over a life time, and microgenetic, which is also personal, but focuses mostly on growth in schooled knowledge (Cole, 1990). Constructivism based on observational learning is based upon two concepts: (a) social context influences learning through selective reinforcement and, (b) modeling complex behaviors facilitates acquisition of knowledge as a system of interactive components (Miller, 2002). Learning ballet, swimming, or other performance based learning tasks requires observation of a more experienced other to understand what counts as a successful completion. Observational learning may contribute heavily to firmly held beliefs, because it occurs over time and in socially meaningful contexts. Teachers have prior knowledge pertinent to subject matter, but they also have a great deal of experience with schooling. Teachers have spent over 3000 days as children and 12

26 young adults observing teachers (Kennedy, 1990a). Their experiences are tantamount to an apprenticeship of observation, and it is one which is invested with emotion, given the students dependence on the teacher (Kennedy, 1991, p. 8). In the study of teacher learning, the type of constructivist framework chosen is a function of what type of knowledge is being investigated or promoted. Backwards design, e.g., deciding what knowledge and/or skills are desired prior to planning instruction, may be as useful in planning instruction for teachers as it is for students (Wiggins & McTigue, 1998). Once a plan is in place for what knowledge is desired, then a suitable framework can be selected to guide instructional technique. For example, if the desired outcome is an increase in the depth of teacher content knowledge, professional development planners need to emphasize Piagetian-style constructivism that is focused on conceptual knowledge production. In contrast, if the goal is to generate effective classroom teaching practice, influences from the social learning theories such as Vygotsky-inspired constructivism are desirable. Researchers of teacher learning and professional development planners are advised to select a theoretical stance that most closely aligns to the type of knowledge being studied or desired as an outcome (Orgill, Bodner, Ferguson, Hunter & Mayo, 2007.) Different types of knowledge may warrant different theoretical leanings and instructional frameworks. Types of Teacher Knowledge Cognitive views of learning identify three types of knowledge: declarative, procedural and conditional (Woolfolk, 2004). Declarative knowledge is information that can be stated either verbally, written, or through some other symbol system. Declarative knowledge is factual and frequently the type of information tested through multiple- 13