Understanding and Using Standardized Test Results

Transcription

1 Page 1 Assessment is an integral part of a good educational process; it provides feedback on the effectiveness of our curriculum and instruction on student learning. Clearly, assessment is not an end in itself; it is a vehicle to help educators understand and support students learning. The data gathered across time enables us to make comparisons of progress and provides evidence of students understanding. The usefulness of an assessment depends on how the results are understood and applied across time to influence curriculum, instruction and learning. Ideally, this focus on results will occur on at least two connected levels: A quality large scale assessment program, such as a nationally standardized test that is used across a school or region to assess students achievement; and A quality classroom assessment program that is tied to the same curricular goals as the large scale assessment. Let s look at each of these assessment components at the school and classroom levels and how they can help when reviewing curriculum and instruction. Understanding and Using Standardized Test Results Standardized achievement tests have been used in American schools for over 80 years to measure student learning. Such tests require extensive research and psychometric expertise to develop; as a result, there are only a handful of publishers offering excellent achievement tests today. These are known for their technical quality, ensuring the tests are valid (measuring what they are intended to measure) and reliable (ensuring consistency in the results across time and populations.) A well constructed national achievement test will provide data that not only compares students performance to that of a national norm group, but also gives information related to students performance on specific skills expected at a grade level. These standardized tools allow schools to measure performance in a dependable way that is, the test items, the scoring and the report formats are the same, no matter which school or teacher administers the test. Too often in the past, achievement tests were administered, reports received, and results filed away in a file cabinet. Little or no use was made of those results to examine the effectiveness of the curriculum or to reflect on classroom instruction. That is not the case today in effective, data driven schools. Educators strive to understand their achievement test results and to use the data to make needed adjustments in the curriculum and instruction at their schools. To use data in this way, the critical element is not the norm referenced results comparing students to a national norm group; it is the objectives based reports detailing student performance on the specific skills and competencies assessed by the standardized test. To support this process at the classroom or school levels, a test must have a transparent objective

2 Page 2 structure which identifies the general skill areas tested at each level and gives a delineation of subskills related to those broad objectives. The clearer the test publisher is about the specific infrastructure assessed, the easier it is for educators to use their results to review their curriculum and instruction. A test s internal structure should be apparent on the objectives reports for that achievement test. For example, a Math subtest report will delineate that specific skills assessed included Number Relationships, Computation and Estimation, Measurement, Geometry, Data and Statistics, Algebra and Problem Solving. Not only do educators need reports or information to understand the test s objective structure and the underlying skills and subskills, they also need a gauge or criteria for interpreting whether students performance is adequate. The publisher should provide scores or criteria to differentiate grade level performance from below grade level performance. Generally there will be three performance levels: high, indicating on or above grade level performance; moderate, indicating near grade level performance; and low, indicating significantly low performance at a grade level. For example, on the TerraNova 3 rd Edition achievement test, objectives reports include a score called the Objective Performance Index (OPI) that graphically and numerically designates whether a student performed at or above expectations for the grade level, slightly below grade expectations, or in a low performance range. This is reported both as an individual student score and as average OPIs for groups of students. Once educators have an understanding of the achievement test s objectives reports and the scores explaining whether grade level content was mastered by the students, it is easy to examine the current curriculum and instructional program for its strengths and challenges using students assessment results. These objective based results should be analyzed across gradelevels (for curriculum review) and at the classroom level (for instructional review). Let s look at the following examples. Analyzing Grade level Results to Review Curriculum: For this review look for the objectives report at the school or grade level; this report will include all students tested at a given grade level in the school. 1. Review each content area subtest, emphasizing the results in core areas such as Reading, Language and Mathematics. 2. Identify the objectives within each content area where this group of students perform at or above the publisher s criteria for high mastery. As a rule of thumb, I look for objectives where 75% or more of the students are performing at or above grade level for a specific objective.

3 Page 3 3. These objectives would be considered strength areas: the current curriculum and instructional program is supporting the great majority of students to meeting gradelevel expectations. Little or no change is indicated for the curriculum in these areas. 4. Similarly, identify the objectives where students perform below the publisher s criteria for grade level expectations. A typical guideline is fewer than 75% of students meet the grade level criteria. These indicate areas where the current curriculum and instructional program may need to be enhanced Analyze more closely the performance of students not meeting the grade level criteria. On which objectives is performance close to the criteria for grade level performance? This generally indicates areas where the curriculum may need to be tweaked. Consider whether students need more instruction, more time practicing the targeted objective, or more rigor in their instruction. You will also want to know on which objectives performance is significantly low, e.g., fewer than 50% of students meet the grade level criteria. This generally indicates areas where substantial adjustments may be needed to the curriculum and instruction. While it may be an issue of more instruction, it is likely an issue of different instruction. You will want to examine the textbook and instructional materials, the sequence and scaffolding used in teaching, the instructional strategies or approaches, the level of active student engagement, and the rigor of classroom expectations. For objectives with low performance, it is important to drill down to more specific information. Examine the sub skills within an objective to determine details that might be affecting student performance. For example, with the TerraNova 3 rd Edition Achievement Test, educators can utilize the Teacher s Guide to TerraNova 3 rd Edition. This book lists the subskills assessed for each objective at each grade level as well as the Depth of Knowledge or rigor of each item assessed. Such information will be useful in examining the depth and breadth of current curriculum and instruction related to the targeted objectives. Once the strengths and challenges are established among the objectives assessed at one grade level, repeat these procedures at other grade levels assessed on this achievement test. Then examine the pattern of objectives strengths and challenges across the grade levels. A pattern of high achievement for a particular objective across the years indicates a strong curriculum in that area within the school. A pattern of weak or uneven achievement for an objective across grade levels indicates the need for a thorough curriculum review and possibly instructional adjustments in that targeted area.

4 Page 4 Analyzing Class level Results: Each teacher will want to examine their students objectives results using the publisher s individual student roster report. These results not only reflect students learning but also the effectiveness of the instruction provided to those students. 1. Identify students performing at similar levels on the objectives within each content area. Which students performed at or above the publisher s criteria for grade level expectations on each objective? Which students performed below grade level expectations for each objective? 2. Based on students performance, the teacher can consider grouping students with similar performance for differentiated instruction. 3. Differentiated instruction can be either acceleration for students with high performance on specific objectives or intervention for students performing below grade level expectations. Students with differing competency levels on certain objectives and skills will need differing instruction. 4. In general, when student performance is low for an objective, the teacher will want to examine the subskills assessed under that objective at the given grade level. In addition, a review of instructional strategies previously used is warranted. Focus on identifying or adopting instruction that will allow for strong scaffolding of instruction (first constructing a firm foundation for the targeted skills and then moving step by step to build student proficiency) and active engagement of students at each step along the way. Once specific objectives are targeted as problem areas, certain steps are indicated. Review the grade level curriculum for the identified objectives what specific skills and supporting skills are taught? How long are they taught? Is the skill sequence well developed for a solid foundation and learning path? Review the textbook, lesson plans and instructional resources related to the identified objectives does the instruction delivered match the written curriculum? What does instruction look like? Are students actively engaged both in learning activities and demonstrating their competency? Developing and Implementing a Supportive Classroom Assessment Plan Assessment for a strong curriculum and instruction program cannot depend on the once a year large scale achievement test; it must be complemented by a cumulative process that emphasizes ongoing, formative assessment of students learning throughout the school year. This assessment is focused on the learning goals at each grade level and particularly emphasizes the targeted areas of previous low achievement at this grade. Such classroom assessment is not focused on grades for the teacher s gradebook; it acknowledges that learning is a complex process that is best revealed in performance over time and gathers information for a more complete, accurate picture of learning in this classroom.

5 Page 5 As a first step, review the current classroom assessment program is it dynamic and centered on student learning, or is it primarily focused on grades for the teacher s gradebook and report cards? Is it tied to the same goals as the large scale assessment, e.g., shared expectations for learning at a specific grade level? Does it occur on a frequent basis to monitor progress toward those goals and to adjust instruction along the way? The following are indicators of a quality classroom assessment program that is focused on continuous review of instruction and learning, ensuring all students are making steady progress toward the learning goals expected at each grade level. 1. There is regular use of informal procedures linked to key lessons and daily instruction. These quick checks provide real time information to inform a teacher s instruction, allowing the teacher to immediately make changes in instruction or student groupings to improve students understanding and performance. 2. There are a variety of brief, well focused activities that directly assess what students know and can do; these are used on a daily to weekly basis. They may be planned ahead or on the fly, and usually are not graded they are primarily to provide the teacher up to the minute feedback on instruction. Examples include: a. Exit Slips assign a single question or problem the student answers at the end of a class lesson to show whether or not today s instruction connected; b. Daily Journal use an ongoing notebook where students answer questions or solve problems and then explain their reasoning; c. Partner checks have two students work together to answer a question or solve a problem, and then explain their work; d. Peer or self assessment using a simple rubric or checklist, a student rates their own work or that of another student; revisions or additions are made based on the rating; e. Open ended questions ask questions that require students to think and reason or explain; allow adequate wait time for students to process their responses f. Embedded assessment activities use a classroom assignment that reviews and integrates the past 3 or so lessons on a targeted skill or concept as an assessment of students learning across those lessons; g. Graphic organizers provide a concept map with some major vocabulary or concepts listed; students complete this with additional details and make connections to show their understanding of the topic. The internet can provide a wealth of these assessment activities. For example, a list of 60 formative assessment activities compiled by the Orange County, FL Public Schools can be found at and an A to Z list of similar activities is found at loticonnection.cachefly.net/iste_2010/informal_assessment_strategies.pdf.

6 Page 6 3. Unit or classroom tests, typically aligned with the textbook, are revised to include items assessing the targeted objectives from the standardized achievement test results. This can be done by adding targeted items (as review items) or adapting the publisher s items, e.g., delete answer options to change a multiple choice item into an open ended item requiring more reasoning or explanation from the student. By focusing assessment to refine ongoing instruction as well as address areas targeted through analysis of standardized tests objectives, educators are making connections across classroom instruction and the goals measured on standardized achievement tests. This model also strengthens the reliability and validity of classroom assessment by gathering more evidence in a variety of ways and allowing students to demonstrate their understanding in different ways. Such connections across daily classroom instruction and annual standardized test results provide a proactive approach to using assessment data to review curriculum and instruction and support students learning.