Considerations for Using STAR Data with Educator Evaluation: Ohio

Transcription

1 Considerations for Using STAR Data with Educator Evaluation: Ohio Purpose Renaissance Learning has developed this document in response to customer requests for information on how to use the data generated by STAR assessments (STAR Reading, STAR Math, and STAR Early Literacy) to support the teacher evaluation process in Ohio. Note: This document provides examples of ways that educators use STAR data. This information is not intended as a recommendation or to invalidate district policy. In addition, educators are always cautioned against placing too much emphasis on any one evidence source. Improve teaching effectiveness and student outcomes We believe that the purpose of educator evaluation is to support effective teaching and improve student learning. Educator effectiveness refers to the degree of quality in specific dimensions of teaching, such as classroom management and a deep understanding of how learning progresses in a discipline. Educator evaluation, generally a summative measure determined from an examination of quantitative as well as qualitative data, refers to a degree of quality in overall performance and achievement for a particular school year or span of years. Effectiveness for all educators is the goal. Evaluation, as required by local, state, and federal legislation, is one pathway to achieve that goal, and is the focus of this document. As an interim assessment, STAR provides educators with reliable data during the year so they can see the path ahead in time to impact it. This document will show how STAR can provide teachers with critical data for documenting instructional practice and building a body of evidence of student growth and achievement as part of their district s educator evaluation process. STAR assessments can be administered multiple times throughout the school year, creating a trustworthy trend line that tells a story. Teachers and principals can demonstrate student progress toward incremental benchmarks, mid-course corrections in instruction and the resulting effects, efforts to screen and identify students in need of (and in response to) intervention, trends toward state proficiency, and patterns in learning. Instruction is key to growth Accelerating growth for all students requires ongoing focus on instruction to accomplish learning content goals and meet growth targets. Assessing students, setting targets, and monitoring growth aren t enough; there must also be an instructional plan to advance learning. Quality instruction is key to achieving growth. Student Growth Percentiles as reported in STAR In states that use Student Growth Percentile (SGP) for state reporting of student growth, SGP is an acceptable source of evidence for student growth for student learning objectives (SLOs) and other elective sources of data. With the updated SGP model, STAR is now even better at measuring within-year growth for educator evaluation purposes. STAR assessments continue to fully meet the requirements for measuring within-year growth for educator evaluation, including in states where it is approved for this purpose. A Student Growth Percentile, or SGP, compares a student s growth to that of his or her academic peers nationwide. Academic peers are students in the same grade with similar achievement history on STAR assessments. SGP is reported on a 1 99 scale, with lower numbers indicating lower relative growth and higher numbers indicating higher relative growth. Page 1 of 12

2 For more information on SGP and value-added growth scores, including information on why SGP is suggested as a statistical growth metric to use with STAR (as compared to other change measures, such as change in scaled score), see the Educator Effectiveness/Educator Evaluation and the SGP FAQ. For SGPs to be reported in STAR, students must be tested within at least two of the following date ranges: Fall: August 1 - November 30 Winter: December 1 - March 31 Spring: April 1 July 31 Special considerations for K-3 teachers Over the course of a given school year, many K-3 students transition from non-reader to reader status. To get SGPs, students must pretest and posttest with the same assessment. In other words, students who test with STAR Early Literacy in the fall must test with STAR Early Literacy in the spring in order to get an SGP. Of course this does not preclude also testing with STAR Reading Enterprise during the year as non-readers become readers. Please note that STAR Reading does not produce an SGP for kindergartners; STAR Reading reports SGP beginning in first grade. (STAR Early Literacy does report SGP for kindergarten.) Page 2 of 12

3 Original Teacher Evaluation Original Framework (Source) Ratings and Points (Source) Page 3 of 12

4 Alternative Framework for Alternative Framework (Source) Ratings and Points (Source) Page 4 of 12

5 Process and Components Approved Vendor Source The original and alternative frameworks allow the use of approved vendor assessments, such as: STAR Early Literacy STAR Reading STAR Math Safe Harbor Source Safe harbor only applies to educators that use value-added ratings from state tests: Districts may decide to use student growth measures other than value-added results for evaluations, including approved vendor assessments and student learning objectives (SLOs), to replace value-added results from state tests This Safe Harbor provision allows for the use of STAR assessments in grades 4-8. Because of the Safe Harbor provision, STAR data could be used to measure growth for students in K through 12. Student Learning Objectives Source A Student Learning Objective (SLO) is a measurable, long-term academic growth target that a teacher sets at the beginning of the year for all students or for subgroups of students. SLOs demonstrate a teacher s impact on student learning within a given interval of instruction based upon baseline data gathered at the beginning of the course. SLO Scoring Source Page 5 of 12

6 Student Learning Objective Teacher Name: Verna Morris Content Area/ Grade Level: Math/ Grade 5 Baseline and Trend Data What information is being used to inform the creation of the SLO and establish the amount of growth that should take place? My objective is to help the students in my class grasp both the foundational and grade-level principles of fractions and decimals in order to meet the fifth grade state standards in these two important areas. I am choosing this objective because solidifying these skills creates the foundation for ongoing conceptual understanding and problem solving in math, which is critical for college and career readiness, as well as the increased focus in this area due to state testing. When examining my students state test results from last year, I noted these were areas of struggle. This was confirmed by examining previous years assessments on multiplying and dividing with decimals, rounding decimals, comparing decimals, as well as adding and subtracting fractions with unlike denominators and multiplying and dividing unit fractions by a whole number. To measure my students improvement in math achievement for this SLO, I will use student growth percentile (SGP) from STAR Math. Because SGP is a growth score, it appears on the STAR Growth Report after two tests. I will monitor my students progress mid-year after winter testing. While I will use the SGP from winter testing to monitor my students growth, I will use only the SGP from spring testing to document growth for my SLO. I will also use the constructed-response items and performance tasks available in STAR Math to provide additional insight into my students conceptual development and proficiency on the standards in Numbers and Operations in Base Ten and Fractions for grade 5. In addition, I will continue to closely monitor classroom tasks to confirm that students are transferring their learning to daily problem-solving. Student Population Which students will be included in this SLO? Include course, grade level, and number of students. All nineteen students in my seventh hour grade 5 math class are included in this SLO. Interval of Instruction What is the duration of the course that the SLO will cover? Include beginning and end dates. This SLO will cover the school year. I will use the fall-to-spring SGP score in my SLO. However, throughout the school year, I will monitor student growth using STAR s Growth Proficiency Chart, which provides a real-time snapshot of my students performance and growth in relation to the benchmark proficiency level. This year, I will begin to follow the assessment activity timeline shown on p. 12 of this document. Page 6 of 12

7 Standards and Content What content will the SLO target? To what related standards is the SLO aligned? In addition to regular math instruction, I will place special emphasis on helping students understand and demonstrate proficiency on the Numbers and Operations in Fractions for grade 5. This includes the following from Ohio s New Learning Standards. NUMBER AND OPERATIONS IN BASE TEN 5.NBT Understand the place value system. 1. Recognize that in a multi-digit number, a digit in one place represents 10 times as much as it represents in the place to its right and 1/10 of what it represents in the place to its left. 2. Explain patterns in the number of zeros of the product when multiplying a number by powers of 10, explain patterns in the placement of the decimal point when a decimal is multiplied or divided by a power of Read, write, and compare decimals to thousandths. 4. Use place value understanding to round decimals to any place. Perform operations with multi-digit whole numbers and with decimals to hundredths. 5. Fluently multiply multi-digit whole numbers using the standard algorithm. 6. Find whole-number quotients with up to four-digit dividends and two-digit divisors, using strategies based on place value, the properties of operations, and/or the relationship between multiplication and division. 7. Add, subtract, multiply, and divide decimals to hundredths, using models or drawings and strategies based on place value, properties of operations, and/or the relationship between addition and subtraction; relate the strategy to a written method and explain the reasoning used. NUMBER AND OPERATIONS FRACTIONS 5.NF Use equivalent fractions as a strategy to add and subtract fractions. 1. Add and subtract fractions with unlike denominators (including mixed numbers) by replacing given fractions with equivalent fractions in such a way as to produce an equivalent sum or difference of fractions with like denominators. 2. Solve word problems involving addition and subtraction of fractions referring to the same whole, including cases of unlike denominators, e.g., by using visual fraction models or equations to represent the problem. Use benchmark fractions and number sense of fractions to estimate mentally and assess the reasonableness of answers. Apply and extend previous understandings of multiplication and division to multiply and divide fractions. 3. Interpret a fraction as division of the numerator by the denominator (a/b =a b). Solve word problems involving division of whole numbers leading to answers in the form of fractions or mixed numbers, e.g., by using visual fraction models or equations to represent the problem. 4. Apply and extend previous understandings of multiplication to multiply a fraction or whole number by a fraction. 5. Interpret multiplication as scaling (resizing), by: a. Comparing the size of a product to the size of one factor on the basis of the size of the other factor, without performing the indicated multiplication. Page 7 of 12

8 Assessment(s) What assessment(s) will be used to measure student growth for this SLO? I will use the STAR Math computer-adaptive assessment to measure growth in math computation and math problem solving this year. SGP reported via STAR will be a key metric in my growth target. I will also use constructed-response items and performance tasks to provide additional insight into my students proficiency on the Number System domain for grade 5. Constructed-response items and performance tasks are available as instructional resources in STAR Math. Additionally, evidence of improved achievement will be determined from classroom problem-solving assignments. Growth Target(s) Considering all available data and content requirements, what growth target(s) can students be expected to reach? I have selected a growth target of 35 SGP. As noted earlier, I will use the spring SGP reported in STAR Math to document growth for my SLO. Rationale for Growth Target(s) What is your rationale for setting the above target(s) for student growth within the interval of instruction? I reviewed my previous years data to determine how grade 5 students have historically grown in my classroom. Over the last four years, the median SGP of my students has been: 35 SGP in 2012, 36 SGP in 2013, 36 SGP in 2014, and 35 SGP in Based on my students growth over the years as it is reflected in the SGP metric, I will set a growth target this year of 35 SGP. Page 8 of 12

9 How to incorporate STAR data into an SLO framework For schools in their first year using STAR data A SLO is calculated by determining the percentage of students in a class who meet a growth target. The first table below illustrates the percent of classrooms in which a specified percent of students hit the 35, 40, and 50 SGP growth targets in Ohio, based on data collected by Renaissance Learning. Ohio Impact Data STAR Math (n=9,570 classrooms) Percentage of students within a classroom who met the growth target Growth Targets 35 SGP 40 SGP 50 SGP 60% of students hitting target per classroom 70% 61% 40% 70% of students hitting target per classroom 52% 42% 25% 80% of students hitting target per classroom 32% 24% 12% Percentage of individuals students meeting target (n= 214,374) INFORMATION TO SUPPORT YOUR DECISION MAKING: 69% 64% 54% need Ohio to meet Impact the growth Data target using STAR Reading as the growth-measuring assessment in order STAR for the Reading teacher s (n=15,096 SLO rating classrooms) to be Average. 42% of teachers in Ohio would receive that rating on their SLO. If the minimum percentage to be Above Average is set at 80%, 24% of Growth Targets teachers Percentage would receive of students this rating. within a classroom who Note met that the in growth the final target row of the table, when not rolled up into classrooms, 64% of individual students would meet the 40 SGP growth target. However, SLOs 35 are SGP generally 40 SGP scored by 50 a SGP percent of students in a class meeting the growth target. Educators should always consult their own 60% school of students and district hitting baseline target data per classroom when considering the setting 74% of growth 65% targets. 37% 70% of students hitting target per classroom 55% 33% 24% 80% of students hitting target per classroom 32% 23% 11% Percentage of individuals students meeting target (n= 247,472) INFORMATION TO SUPPORT YOUR DECISION MAKING: 70% 65% 56% need to meet the growth target using STAR Reading as the growth-measuring assessment in order for the teacher s SLO rating to be Average. 42% of teachers in Ohio would receive that rating on their SLO. If the minimum percentage to be Above Average is set at 80%, 24% of teachers would receive this rating. Note that in the final row of the table, when not rolled up into classrooms, 64% of individual students would meet the 40 SGP growth target. However, SLOs are generally scored by a percent of students in a class meeting the growth target. Educators should always consult their own school and district baseline data when considering the setting of growth targets. Page 9 of 12

10 How to use the suggested median SGP in a non-slo framework Ohio Student Growth Category Points Suggested Median SGP Source of Suggested Median SGP (Based on 3 years of STAR data in OH)* Most Effective % of teachers Above Average % of teachers Average % of teachers Approaching Average % of teachers Least Effective % of teachers * Note: These Suggested Median SGPs were developed by analyzing three years of STAR Reading, STAR Early Literacy, and STAR Math pre-test and post-test scores from Ohio schools. Samples sizes were approximately 350,000 students for STAR Reading, 85,000 students for STAR Early Literacy, and 250,000 students for STAR Math. Student growth data was grouped at the classroom level, and the median SGP for each class was found. We looked at all of the classrooms that had 10 or more students to estimate the distribution of teacher ratings. In total, data from approximately 18,000 STAR Reading classrooms, 4,000 STAR Early Literacy classrooms, and 13,000 STAR Math classrooms was examined. Between 1-5% of teachers were Least Effective; between 15-20% of teachers were Approaching Average; between 35-40% of teachers were Average; between 30-35% of teachers were Above Average; and approximately 5-10% of teachers were Most Effective. Page 10 of 12

11 Determining student growth The STAR assessments include an easy-to-read Growth Report that displays each student s SGP score. The following sample report shows the outcome of the sample SLO. In Ms. Morris s class, sixteen out of nineteen students (84%) met or exceeded the growth target of 35 SGP. Page 11 of 12

12 Assessment Activity Timeline Time of Year Fall Fall Winter Winter Winter Spring Spring Assessment Activities Teachers assess students with the STAR Math computer-adaptive test for fall screening. The STAR software generates Screening and Instructional Planning Reports in real time. Teachers administer and score the constructed-response items and performance-based tasks that are available in STAR s learning progression. These items are similar to State assessment items. Teachers assess students with the STAR Math computer-adaptive test for winter screening. The STAR software generates Screening, Instructional Planning, and Growth Reports in real time. The STAR Math Growth Report provides teachers with student growth percentile (SGP) scores for individual students, and for the class as a whole, for mid-year analysis. Teachers administer and score the constructed-response items and performance-based tasks that are available in STAR s learning progression. These items are similar to State assessment items. Teachers assess students with the STAR Math computer-adaptive test for spring screening. The STAR software generates Screening, Instructional Planning, and Growth Reports in real time. The STAR Math Growth Report provides teachers with SGP scores for individual students, and for the class as a whole, for growth reporting, student learning objectives (SLOs), and program analysis. Disclaimer of Liability The information contained in this document is solely for educational purposes and marketing. Renaissance Learning, Inc., its affiliated companies and their respective owners, officers, employees, agents {collectively, "Renaissance") make no, and disclaim all, express and implied representations or warranties, including, without limitation, THE IMPLIED WARRANTI ES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, concerning STAR, the information or materials described herein or any uses made thereof (including evaluation or assessment) or the results or consequences derived or resulting from the foregoing. Although this document has been produced from sources believed to be reliable at the time, no representation or warranty, express or implied, is made regarding the accuracy, adequacy, timeliness, completeness, legality, reliability or usefulness of any information contained herein. Each user of STAR must determine for itself the appropriateness and advisability of using STAR, the information or materials described herein, or any output, work product, analysis, results or conclusions derived or resulting, in whole or in part, there from, and assumes sole liability therefor. Each such user waives to the fullest extent permitted by law any and all claims, demands, suits, causes of action and other rights against Renaissance with respect thereto (regardless of whether any such claim, etc. is based upon contract, tort, negligence, strict liability, common law, statutory or other theory of recovery). Each such user agrees that it (and not Renaissance) shall be responsible for any claims, demands, suits, causes of action, losses, damages and other costs and expenses asserted by any third party affected by user's use of the foregoing. Copyright 2016 Renaissance Learning, Inc. All logos, designs, and brand names for Renaissance Learning s products and services, including but not limited to STAR Math, STAR Reading, STAR Early Literacy, and STAR are trademarks of Renaissance Learning, Inc., and its subsidiaries, registered, common law, or pending registration in the United States and other countries. All other product and company names should be considered the property of their respective companies and organizations. PO Box 8036 Wisconsin Rapids, WI (800) Page 12 of 12