The Colorado Growth Model Accountability 2.0 & Educator Effectiveness

Transcription

1 The Colorado Growth Model Accountability 2.0 & Educator Effectiveness Richard J. Wenning Associate Commissioner Colorado Department of Education

2 Acknowledgement I wish to thank Dr. Damian W. Betebenner, the developer of student growth percen>les, for his partnership and contribu>ons to this presenta>on.

3 Overview Theory of Ac>on: Accountability 2.0 Choosing a Growth Model: Strategic and Tac>cal Considera>ons Evalua>ng Educator Effec>veness: Essen>al Policy Ques>ons Role of Student Growth Percen>les Statewide Results

4 Theory of Ac>on for the Work? Accountability for educa>onal effec>veness exists within systems of student, school, district, state and federal accountability BePer when these systems are aligned to support the business we are in

5 What Business Are We In? Maximize student progress toward college and career readiness Implies new bright line: all kids ready by exit Requires a defini>on of readiness and the standards leading there Requires measurement system that determines how well students are progressing toward and reaching the des>na>on

6 Policy Perspec>ve on Growth Why is measuring student growth so important? NCLB (Accountability 1.0) had right intent but incomplete measures and incen>ves Incen>ves focused on short- term increases in percent proficient, on bubble kids, invited moral hazard Instead of long- term effec>veness and progress for all kids ESEA reauthoriza>on provides opportunity to get the measures & incen>ves right

7 Desired Level of Performance? Drama>c, not incremental improvements required for students that need to catch up to become college and career ready

8 Desired System: Accountability 2.0 Coherent system focused on learning and building student, educator, school, district, state and federal performance management capacity. Maximize local ownership of high quality informa>on to drive insight and ac>on We should ensure educator effec>veness conversa>on not stuck in Accountability 1.0

9 Accountability for Learning Designing a Coherent System Developed by NAS 9

10 What Models? What sta>s>cal models of longitudinal student growth will promote the most coherence and alignment in our accountability system?

11 Some Framing Ideas We understand best those things we see emerge from their very beginnings. - - Aristotle All Models are wrong but some are useful. - - George E. P. Box It is be:er to have an approximate answer to the right ques>on than a precise answer to the wrong ques>on. - - John Tukey

12 Consequen>al Validity Henry Braun (2008) Assessment prac>ces and systems of accountability are consequen>ally valid if they generate useful informa>on and construc>ve responses that support one or more policy goals (Access, Quality, Equity, Efficiency) within an educa>on system, without causing undue deteriora>on with respect to other goals.

13 Ques>ons Set the Table Growth models address specific ques>ons Different techniques are good at answering different ques>ons Different ques>ons lead to different conversa>ons which lead to different uses and outcomes Star>ng with the right ques>ons simplifies development and mo>vates the proper use of the growth model results

14 Policy Q s: Educator Effec>veness What ques>ons do we want to answer about growth rates of students associated with educators? What mapers and when? Student growth rates as evidence of effec>veness or ineffec>veness? How many categories of effec>veness and ineffec>veness are important? Which categories are consequen>al and for what? What body of evidence will be used to evaluate and infer teacher value- added? Norma>ve and criterion- referenced growth?

15 Who/What is Responsible for Student Growth? Some analyses of student growth apempt to determine the amount of student progress that can be apributed to the school or teacher Called value- added analyses, these techniques apempt to es>mate the teacher/school contribu>on to student academic growth Value added is an inference a causal conclusion drawn from the data All growth models can be used for value- added purposes

16 How much growth did a student make and is it enough? Describing growth versus ascribing responsibility The Colorado Growth Model began by separa>ng the descrip>on of growth from discussions of responsibility/ accountability Incorpora>ng growth into accountability followed from the accepted descrip>on of growth The descrip>on of growth facilitated stakeholder engagement and inves>ga>ons of responsibility for good/bad growth That in turn led to greater stakeholder support for par>cular forms of accountability

17 Describing Student Growth Discussing student growth, even with a ver>cal scale, is not a simple task Growth and change require context. Consider, for example, height: A child might grow 4 inches between ages 3 and 4 4 inches is a well understood quan>ty The 4 inch increase becomes meaningful only when understood alongside the growth of other 3 to 4 year olds Student growth percen>les were developed to provide a norm- referenced basis for describing student growth

18 Colorado Growth Model Asks What is? What should be? What could be? How much growth did a child make in one year? How much growth is enough to reach proficient (or advanced)? How much growth have other students made with the same score history?

19 Student Growth Percen>les Should we be surprised with a child s current achievement given their prior achievement? Student growth percen>les answer this ques>on Consider a low achieving student with 90th percen>le growth and a high achieving student with 10th percen>le growth The low achieving student grew at a rate exceeding 90 percent of similar students The high achieving student grew at a rate exceeding just 10 percent of similar students The low achiever s growth is more exemplary (probabilis>cally) than the high achiever s Judgments about the adequacy of student growth require external criteria together with standard seing

20 Establishing Growth Standards Based Upon Growth Norms The most common adequacy criterion is judging growth toward an achievement goal (i.e., growth- to- standard) Results from student growth percen>le analyses can be used to calculate percen>le growth trajectories for each student These trajectories indicate what future rates of growth will lead to and are used to make adequacy judgments This growth- to- standard approach was approved as part of Colorado s successful applica>on to the Growth Model Pilot Program

21

22 One Student s Growth Percen>les

23 Students in a Grade in a School

24 Views of Schools within a District

25

26 View of Districts

27

28

29 Colorado Growth Model Shows Growth not correlated with how close a student is to proficient Norma>ve growth gaps narrowing among student groups Most low achieving students not growing fast enough to catch up Many proficient students not keeping up Many high achieving schools show low growth and many low achieving schools show high growth

30 Gap in MGP Growth Gaps in Reading Closing over Time Black- White Gap Hispanic- White Gap FRL- Non FRL Gap

31 Gap in MGP Growth Gaps in Math Closing over Time Black- White Gap Hispanic- White Gap FRL- Non FRL Gap

32

33

34

35 Percentage of Students Growing Enough to Catch Up and Keep Up in Reading % of All Students Keeping Up Catching Up Non- FRL FRL

36 Percentage of Students Growing Enough to Catch Up and Keep Up in Math % of All Students Keeping Up Catching Up Non- FRL FRL

37 14 States with MOU 14 states have signed MOU to use the Colorado Growth Model growth percen>le methodology and SchoolView display tools: Arizona, Colorado, Georgia, Indiana, Kansas, MassachusePs, Missouri, Nevada, New Hampshire, New York, Virginia, Washington, West Virginia, Wisconsin Pursue cloud- based analy>cs and visualiza>ons Crea>ve Commons Share Alike Noncommercial Use