Decline in Predictive Percent Proficient/Advanced on Discovery Education Assessment: Tennessee Test B for Selected Grades/Subjects Dear Tennessee Educators: A number of our Tennessee partner districts have provided valuable feedback regarding their observation of a downward trend in student proficiency from Test A to Test B for the 2012-2013 school year. Discovery Education Assessment has conducted further analysis of its predicted percent of students who are proficient/advanced on the following 2012 Test A and Test B benchmarks: Grades 3-8: Reading, Mathematics, and Science End-of-Course: English 2, Algebra 1, and Biology Based on the results of our analysis, we are providing the following information for you to share with your educators. In addition, we will offer two webinar events to explain our analyses and address any concerns you may have. The webinar events will be recorded and posted at a link we will provide to you so you may access them on demand. Discovery Education Assessment uses a Continue-to-Learn Model in setting cut scores for its proficiency prediction. These cut scores are derived from an analysis of Discovery benchmark data and TCAP 2012 results. Based on customer demand, a certain percentage of test items were revised or replaced on these benchmarks for the 2012-2013 school year. Cut score adjustments were made to account for these test revisions. With the Continue-to-Learn Model, Discovery expects the predicted percent proficient/advanced to be approximately the same on Test A and Test B. Slight increases or decreases in proficiency on Test B over Test A could reflect improvement or decline over the previous year. Both Test A and Test B predictions tend to be conservative and, in most cases, will under predict actual performance on TCAP. The results of Discovery s analysis are shown in the following tables, which present the predicted percent proficient/advanced for 2012-2013 Test A and Test B interim benchmarks in Reading, Mathematics, and Science:
Reading Percent Prof/Adv Grade 3 43.4 34.2-9.2 Grade 4 43.7 48.2 4.5 Grade 5 52.3 49.2-3.1 Grade 6 44.8 46.1 1.3 Grade 7 39.9 32.8-7.1 Grade 8 41.5 46.1 4.6 English 2 47.9 40.9-7.0 Math Percent Prof/Adv Grade 3 43.6 37.6-6.0 Grade 4 37.9 45.4 7.5 Grade 5 50.0 45.2-4.8 Grade 6 30.0 43.5 13.5 Grade 7 41.1 29.8-11.3 Grade 8 34.9 27.2-7.7 Algebra 1 45.3 47.7 2.4 Science Percent Prof/Adv Grade 3 54.7 59.6 4.9 Grade 4 45.6 38.8-6.8 Grade 5 50.2 54.7 4.5 Grade 6 42.5 52.2 9.7 Grade 7 57.2 44.0-13.2 Grade 8 61.8 55.0-6.8 Biology 41.4 41.8 0.4
From this data, Discovery has noticed significant declines over expectations in predicted percent proficient/advanced on Test B in the following grades/subjects: Grade 3 Reading Grade 7 Reading English 2 EOC Grade 3 Math Grade 7 Math Grade 8 Math Grade 4 Science Grade 7 Science Grade 8 Science The declines on the above tests appear to be the result of a variety of factors: shifting standards within the state and a shift in when those standards are taught during the school year; an earlier test B window this year than last year; a higher item refresh rate than in previous school years based on customer demand; and conservative cut scores on Test A and B. Discovery has also analyzed the vertical scale score growth between Test A and Test B for 2012-13 and 2011-12 for Reading and Mathematics and the growth for 2012-13 for Science. Reading Scale Scores showed growth in all grades from Test A to Test B in 12-13 except for Grade 5. The amount of growth was slightly smaller than for 11-12 except for Grades 6 and 8. This pattern of slightly smaller growth could also contribute to proficiency predictions being lower on Test B for Grades 3, 7 and English 2. Reading Scale Scores 1213 Reading Scale Scores 1112 Test A Test B Growth B - A Test A Test Growth B - A B Grade 3 1430 1461 31 Grade 3 1428 1479 51 Grade 4 1503 1529 26 Grade 4 1498 1534 36 Grade 5 1556 1548-8 Grade 5 1555 1550-5 Grade 6 1551 1580 29 Grade 6 1578 1594 16 Grade 7 1589 1595 6 Grade 7 1595 1611 16 Grade 8 1623 1631 8 Grade 8 1633 1623-10 English 2 1640 1654 14 English 2 1656 1678 22
On the other hand, growth in Mathematics benchmarks in 12-13 from Test A to Test B was positive (except for Algebra 1) and higher than for 11-12, except for Grades 7 and 8 (with Grade 5 growth being very similar for the two time periods). The decrease in growth in these two grades could also affect the lower predicted percent proficient on Test B. Math Scale Scores 1213 Math Scale Scores 1112 Test A Test B Growth B - A Test Test Growth B - A Grade 3 1370 1394 24 Grade 3 1388 1407 19 Grade 4 1452 1493 41 Grade 4 1464 1485 21 Grade 5 1534 1573 39 Grade 5 1529 1571 42 Grade 6 1568 1598 30 Grade 6 1585 1610 25 Grade 7 1609 1610 1 Grade 7 1615 1625 10 Grade 8 1620 1632 12 Grade 8 1635 1655 20 Algebra 1 1636 1626-10 Algebra 1 1645 1626-19 This is the first year for vertical scaling of Discovery Science benchmarks. Growth between Test A and Test B in 12-13 was strong in all grades except Grade 7 and 8. Science Scale Scores 1213 Test A Test B Growth B - A Grade 3 1420 1451 31 Grade 4 1469 1497 28 Grade 5 1500 1520 20 Grade 6 1534 1565 31 Grade 7 1561 1549-12 Grade 8 1578 1577-1 Biology 1580 1599 19 Based on this analysis, we recommend focusing on the scale score growth that your students are achieving between tests while exercising caution when interpreting the decrease in percent proficient from Test A to Test B. In order to ensure the most accurate predictions on 2012-2013 Discovery Test C, additional time for analysis of Test C data will be needed prior to releasing full reports for grades 3 through high school. We expect that sufficient data will be available for analysis approximately two weeks after the Test C window opens on Monday, January 28. We encourage you to test as early in the window as possible to ensure sufficient data is received in a timely manner.
We realize that educators rely on Test C results to best focus their instruction prior to TCAP and want to ensure that these results are as accurate as possible for our Tennessee educators. We sincerely appreciate the feedback you have provided and want to assure you that we take your concerns very seriously. We remain committed to providing you with high quality assessments and hope our explanations will be beneficial as you continue your post Test B instruction and decision-making. Please feel free to contact our customer support team at assessments@discovery.com or toll-free at 1-866-814-6685, option 2, with any questions. Sincerely, Hardin Daniel Vice President, Discovery Education Assessment