Illinois State Board of Education

Transcription

1 Illinois State Board of Education February 2013 Guidance Document Model Teacher Evaluation System Measuring Student Growth Using Type III Assessments This document is intended to provide non-regulatory guidance on the subject matter listed above. For specific questions, please contact the person(s) identified in the document. Dr. Christopher Koch, State Superintendent Printed by AFL-CIO (AFSCME Local #288 and IFSOE Local #3236) Employees

2 Model Teacher Evaluation System: Measuring Student Growth Using Type III Assessments Subject Recommendation to adopt a student learning objective (SLO) process as a measurement model for student growth for Type III assessments Type of Guidance This guidance document provides recommendations to districts, unions, and Joint Committees for using SLOs as a measurement model for Type III assessments in teacher evaluation. This guidance can be used by any Illinois district, including those districts not implementing the Model Teacher Evaluation System. When implemented with rigor and fidelity, the SLO process meets the criteria and requirements of both the Performance Evaluation Reform Act (PERA) and Part 50 of the Illinois Administrative Code for including student growth in teacher evaluation. This guidance document provides very basic information about SLOs, recommended processes for districts to establish SLOs, and information on the required elements of SLOs in the Model Teacher Evaluation System. For detailed information about what SLOs are, how they should be developed, and critical examples and tools, please see the Guidebook on Student Learning Objectives for Type III Assessments. Explanation Illinois Teacher Evaluation Context To comply with PERA, Sec. 24A-5(c), Illinois districts must develop a teacher evaluation plan that includes measures of student growth as a significant factor in teachers summative performance evaluation. To support districts in this task, Sec. 24A-7 of PERA requires the Illinois State Board of Education (ISBE), informed by recommendations from, to adopt rules defining student growth and methods for measuring student growth. 1 Student growth is a demonstrable change in a student s or group of students knowledge or skills, as evidenced by gain and/or attainment on two or more assessments, between two or more points in time. Measurement model is the process in which two or more assessment scores are analyzed to identify a change in a student s knowledge or skills over time. Source: Illinois Administrative Code, Part 50, Sub. A, Sec The statutory definitions appearing in the box on the right include the term attainment; however, the word attainment in the statutory language here should not be misconstrued as allowing SLOs to be set that do not reference a student s starting point and ending point. For example, setting an SLO target that calls for 80 percent of students to pass the final course exam is not appropriate. Such a target does not account for a student s skill levels at the beginning of the course and it does not reference two assessments (a pretest and posttest) at two points in time. Measuring Student Growth Using Type III Assessments _02/13

3 In addition, district Joint Committees must identify two assessment types to measure student growth for each category of teacher as well as one or more measurement models that use multiple data points to determine student growth using the selected assessments (Illinois Administrative Code, Part 50, Sub. A., Sec [b]). In selecting assessments to measure student growth, districts must select from three types of assessments: Table 1. Illinois Assessment Types Assessment Type Type I Type II Type III Description A reliable assessment that measures a certain group or subset of students in the same manner with the same potential assessment items, is scored by a non-district entity, and is administered either statewide or beyond Illinois Any assessment developed or adopted and approved for use by the school district and used on a districtwide basis by all teachers in a given grade or subject area Any assessment that is rigorous, that is aligned to the course curriculum, and that the qualified evaluator and teacher determine measures student learning in that course Note: Type I and II assessments also may be considered Type III if they align to the curriculum and are capable of measuring student learning in the subject (Illinois Administrative Code, Part 50, Sub. A, Sec ). District Plan Requirements Under PERA and Illinois Administrative Code, Part 50, Joint Committees composed of equal representation from the district and the teachers union are required to develop comprehensive teacher evaluation plans that include data and measures of student growth. The following bullets summarize the PERA and Administrative Code requirements that all district plans must include for measuring student growth: Select a measurement model for each assessment that includes multiple data points. Identify at least one Type III assessment that will be used to measure student growth for each category of teacher. State the general nature of any Type III assessment chosen (e.g., teacher-created assessments, assessments designed by textbook publishers, student work samples or portfolios, assessments of student performance, and assessments that are designed by staff who are subject or grade-level experts that are administered commonly across a given grade or subject area in a school); describe the process and criteria that the qualified evaluator and teacher will use to identify or develop the specific Type III assessment to be used. Determine the categories of teachers who have neither a Type I nor a Type II assessment available. For teachers without a Type I or II assessment, the evaluation plan must include a minimum of two Type III assessments. Measuring Student Growth Using Type III Assessments 2

4 Set student growth expectations that are consistent with the assessment and model selected. Develop a uniform process for collecting formative student learning data at the midpoint of the evaluation cycle that will assess progress and inform instructional adjustments but will not be included in student growth scores. Discuss how student characteristics (e.g., special education placement, English language learners) are used in the measurement model. recommends SLOs as one approach for measuring student growth using Type III assessments. It is important to note, however, that SLOs are not required for all districts; districts have the autonomy to implement Type III measures, including SLOs, in a way that best fits their specific contexts. For districts that default to the Model Teacher Evaluation System, this document includes descriptions of the minimum requirements for the SLO process. Detailed information about SLOs and how to establish an SLO process are provided in the Guidebook on Student Learning Objectives for Type III Assessments. Using SLOs 2 to Measure Student Growth SLOs are increasingly used in states and districts across the country as a measure for student growth. Early experiences suggest that SLOs, when implemented with fidelity, offer a measurement model for student growth that aligns more directly with actual classroom instruction and teacher practice than other growth models. By providing teachers and principals with a structured process for selecting assessments and setting goals for student learning, an SLO process builds collaboration and communication while giving teachers greater control over how their students growth is assessed and measured. When coupled with strong professional development for educators on developing rigorous, valid, and high-quality assessments, an SLO process can support improved alignment between state standards, curriculum, and classroom assessment while promoting the professional growth of teachers. Benefits of Using SLOs In addition, recommends SLOs as a measurement model for Type III assessments for the following reasons: SLOs promote the intent of Type III assessments to ensure the following: (1) that teachers are assessed using student growth measures aligned with the content, curriculum, and student needs specific to teacher and school contexts; and (2) that teachers impact on student learning is assessed through multiple measures, including assessments other than standardized test scores. 2 The guidance provided in this document is taken from the following publications: Lachlan-Haché, L., Cushing, E., & Bivona, L. (2012). Student learning objectives as a measure of educator effectiveness: The basics. Washington, DC: American Institutes for Research. Lachlan-Haché, L., Cushing, E., & Bivona, L. (2012). Student learning objectives: Benefits, challenges, and solutions. Washington, DC: American Institutes for Research. Lachlan-Haché, L., Cushing, E., & Bivona, L. (2012). Student learning objectives: Core elements for sustainability. Washington, DC: American Institutes for Research. Measuring Student Growth Using Type III Assessments 3

5 SLOs meet the requirements of PERA and Administrative Code, Part 50 for including student growth in teacher evaluation. SLOs support reflective teaching practice. The SLO process asks teachers to identify standards and curriculum, analyze student needs, set goals, use data to assess student progress, and adjust instruction based on formative assessment data. SLOs can promote collaboration and a shared vision. SLOs can be developed by a team of teachers, if appropriate conditions are established, or by an individual teacher. By identifying district and school goals and collaboratively developing SLOs, a culture of shared reflection on practice and mutual support is reinforced. SLOs are adaptable. In addition to being available for all teachers to use, SLOs can also adapt to changes in curriculum and assessments. Challenges of Using SLOs Although SLOs have many benefits, they also present serious challenges and tradeoffs. To implement the process fairly and with fidelity, SLOs require a significant time commitment from teachers and administrators. As a result, the decision to implement SLOs requires a district s Joint Committee to commit to providing the necessary time, resources, and support. Carefully designing an SLO process is critical to reducing the time commitment as much as possible. SLOs can quickly become unduly burdensome if districts overdesign the process and create overly complicated or inefficient systems for completing the SLO. As a result, an SLO process can unnecessarily end up as an additional layer of expectations and tasks for teachers rather than as a supportive process integrated into effective instructional practices. Compared to standardized measures, SLOs present a challenge for assessing the comparability of student growth results on Type III assessments across schools and districts. Although true comparability is an important goal, it is not always attainable. Joint Committees will need to focus carefully on establishing processes and procedures that allow for fairness in the teacher evaluation plan within their district. Finally, SLOs can have unintended consequences when used for high-stakes decisions. One consequence may be inappropriate decisions about rigor in growth targets over time as pressure is placed on educators to meet expected targets. While fully acknowledging these challenges, is committed to working with ISBE to ensure that SLOs are implemented to maximize their potential benefits and ameliorate their challenges. In addition, recognizes that SLO processes and supports will need to be continuously revised and improved as ISBE learns from districts and educators implementing SLOs during the next three years. Necessary Conditions for a Fair, Meaningful SLO Process in Illinois For SLOs to be successful and reliable as a measure of student growth, a number of conditions must be in place. Following are a series of activities that need to occur in order for districts to Measuring Student Growth Using Type III Assessments 4

6 support their teachers in developing SLOs that are rigorous and comparable across all teachers in the state. All districts and educators must receive ongoing support, training, and guidance on how to integrate SLOs in their evaluation system. A well-integrated SLO process cannot be created through general information sharing. Joint Committees, administrators, and teachers need to deeply engage with the SLO process and each other to design an SLO process that fits their district context. Stakeholders need a clear understanding of the basic components of an SLO and the staff capacities required, such as analyzing baseline data and assessment literacy. Principals and teachers must develop a shared understanding of how to set rigorous and realistic growth targets, what a high-quality SLO looks like, and how to score SLOs in a fair and reliable way. Table 2 highlights some specific training and resource topic ideas and the intended audience for each. Table 2. Resource Topics and Intended Audience Topic Aligning SLOs to the overall evaluation system Developing teacher-designed assessments Analyzing baseline student data Developing rigorous and realistic SLOs Providing guidance on selecting assessments Assessing the rigor of SLOs Scoring SLOs Scoring calibration sessions Offering training refreshers in the SLO process over time, informed by ongoing monitoring and evaluation Audience Joint Committees Teachers Teachers Evaluators and teachers Evaluators and teachers Evaluators and teachers Evaluators and teachers Evaluators Evaluators and teachers ISBE will develop a process for periodically gathering SLO information. This process will include collecting relevant SLO documentation, assessments used, and scoring rubrics from districts to monitor and evaluate the rigor and comparability of the SLOs. This information also will be used to continuously improve the SLO process and supports provided to educators. ISBE, in consultation with, will provide districts with the Guidebook on Student Learning Objectives for Type III Assessments. This guidebook provides detailed information, recommendations, examples, and SLO tools such as the following: SLO definition and explanation SLO templates SLO checklist SLO cycle timeline SLO examples Measuring Student Growth Using Type III Assessments 5

7 Guidance on selecting appropriate assessments Guidance on approving and scoring SLOs In addition, Illinois school districts have a responsibility to plan, design, and implement an SLO process with fidelity for each school. Specifically, recommends that Illinois districts consider the steps described in the Guidebook on Student Learning Objectives for Type III Assessments. These steps will ensure that district-level development of SLOs produces a credible, sustainable measurement model that is supported and understood by all teachers, principals, and community stakeholders. More detailed guidance, tools, and information is provided in this guidebook. Model Teacher Evaluation System: SLO Requirements Definitions Student growth: A demonstrable change in a student s or group of students knowledge or skills, as evidenced by gain and/or attainment on two or more assessments, between two or more points in time. Measurement model: The process in which two or more assessment scores are analyzed to identify a change in a student s knowledge or skills over time. Student learning objectives (SLOs): An SLO process creates a measurement model that enables an evaluator to analyze scores from a Type III assessment (e.g., a pretest and posttest) and identify whether a pre-established goal(s) has been met through a demonstrated change in a student s knowledge and skills over time. Following are specific requirements for using SLOs. 1. Number of SLOs A minimum of one SLO is required for each Type III assessment. Table 3 provides a summary of the number of SLOs required for different categories of teachers. Table 3. Number of SLOs Required (in Relation to Teacher Category and Type III Assessments) Category of Teacher Has a Type I or Type II assessment available No Type I or Type II assessment available Minimum # of Type III Assessments Minimum # of SLOs 1 Type III assessment 1 SLO 2 Type III assessments 2 SLOs For teachers who are required to have two Type III assessments, and therefore a minimum of two SLOs (see the last row in Table 3), the following rules apply: One SLO must be set by the evaluator and must be aligned with the school improvement plan and district initiatives. When possible, this SLO should include shared SLOs developed with groups of teachers. Measuring Student Growth Using Type III Assessments 6

8 One SLO must be set by the teacher, but this SLO can include shared SLOs developed with groups of teachers and should be aligned with classroom or student population needs. District and unions should work together to establish the necessary conditions (e.g., trust, mutual support, common curriculum and assessments, collaborative time for setting group and individual growth targets) for teams of teachers to set group or shared SLOs. 2. Inclusion of Students The Model Teacher Evaluation System does not require a minimum threshold percentage of students be included in each SLO. Rather, encourages teachers and evaluators to collaboratively make this decision based on the subject area or grade, analysis of student data, and school goals; the decision must be supported with a clear, defensible rationale. If the teacher and evaluator are unable to reach agreement, the following rules must be used: For a teacher with two Type III assessments, the district must determine the percentage of students that should be covered under the first SLO; the teacher must determine the percentage of students covered under the second SLO. Each SLO will require a written rationale with supporting baseline student data. For a teacher with one Type III assessment, the teacher must determine the percentage of students that should be covered under any SLO and provide a written rationale with supporting baseline student data. 3. Rationale for an SLO The Model Teacher Evaluation System requires teachers to provide a documented rationale for each SLO growth target set for each Type III assessment. 4. SLO Approval Process The Model Teacher Evaluation System requires districts to develop a review process for SLOs to ensure that the SLOs are rigorous and comparable across teachers. 5. SLO Scoring The Model Teacher Evaluation System does not require a specific scoring methodology or rubric; however, districts should consider the guidance in training and supporting materials provided in the Guidebook on Student Learning Objectives for Type III Assessments to develop these materials. 6. Weighting SLOs The Model Teacher Evaluation System does not require a specific method for combining multiple SLOs; however, because student growth is required to be 50 percent of a teacher s performance evaluation score within the Model Teacher Evaluation System, the following rules should be adopted: Measuring Student Growth Using Type III Assessments 7

9 The Joint Committee should decide upon the distribution of the weight applied to each student growth measure, including SLOs. However, in cases where the district Joint Committee cannot reach agreement, the measures must be weighted equally. For example: For a teacher with one Type I or II assessment and one Type III assessment (one SLO), each assessment score must constitute 25 percent, for a total of 50 percent. For a teacher with two Type III assessments (two SLOs), each SLO must constitute 25 percent, for a total of 50 percent. 7. SLO Training and Support All educators and evaluators using SLOs must be provided with ongoing support and training, enabling them to understand the SLO process and ensuring that they can implement a fair process with fidelity. Districts using the Model Teacher Evaluation System should clearly and explicitly link the SLO process with other district goals and initiatives especially school improvement planning to ensure that a single, coherent vision is communicated to all educators in the district. In addition, creating an SLO process that is rooted in teachers and principals work will require engaging educators deeply in the planning, development, implementation, and ongoing refinement of the process. Frequently Asked Questions 1. Should our district use SLOs? Your district s Joint Committee must identify a Type III assessment for each category of teacher and a measurement model to assess student growth on Type III assessments. An SLO process is one option that a Joint Committee can select as a measurement model. Although recommends an SLO process for measuring student growth on Type III assessments, it is not required unless your district is defaulting to the Model Teacher Evaluation System. For more information on the steps and requirements necessary for designing and implementing a fair and efficient SLO process, review the Guidebook on Student Learning Objectives for Type III Assessments. This information can help your district s Joint Committee decide whether an SLO process will meet teacher and student needs in your district. 2. Should SLOs be used for all teachers in my district? If your district selects an SLO process for measuring student growth on Type III assessments, this process should be used with all teachers in the district. All teachers must have a Type III assessment as one measure of student growth, and SLOs are the recommended measurement model for Type III assessments. If your district defaults to the Model Teacher Evaluation System, SLOs are the required measurement model for Type III assessments. Measuring Student Growth Using Type III Assessments 8

10 3. Should our district provide opportunities for teachers and administrators to develop team or group SLOs and Type III assessments? Yes. Providing resources and opportunities for groups of teachers and/or administrators to develop shared SLOs and assessments across the district lessens the burden on individuals and creates better consistency and comparability in the measurement of student growth. For additional guidance and examples on using team or group SLOs, see the Guidebook on Student Learning Objectives for Type III Assessments. 4. Should our district focus more on identifying and developing Type III assessments or on developing high-quality SLOs? Both. An SLO growth target is only as credible as the assessments used to measure it. Similarly, using a valid and reliable assessment alone does not ensure that teachers across the district are being evaluated in a fair and credible manner without a standardized SLO process for measuring and judging student growth using the assessment. An SLO ensures that a teacher has the ability to ensure that his or her assessments are aligned with the classroom curriculum and standards. 5. Can an assessment that meets the criteria for a Type I or II assessment also count as a Type III assessment? Yes. This situation is possible if the assessment meets the criteria for Type I and Type II, aligns to the curriculum, and is capable of measuring student learning in the subject. (See Illinois Administrative Code, Part 50, Sub. A, Sec ) 6. Do teachers have to include a minimum percentage of students in each SLO? No. The percentage of students included should be determined collaboratively by the teacher and evaluator based on their knowledge and understanding of the students and school context (including baseline and trend data); however, as part of the design process, the Joint Committee should develop procedures to encourage consistency across teachers within the district. 7. Do teachers have to set an SLO for every class they teach? No. Districts have the flexibility to decide how many courses and which courses should be included in SLOs. Districts also may leave the decision up to the teacher and evaluator; but if this approach is adopted, it is strongly recommended that districts carefully evaluate and monitor the SLOs set in the district to improve consistency across teachers in similar subject areas and grades. 8. How do I establish SLOs if I am a new teacher? New teachers are likely to have less knowledge and understanding with specific student learning needs and should be encouraged to collaborate with more experienced teachers or a mentor in setting his or her SLOs. Measuring Student Growth Using Type III Assessments 9

11 9. Must SLO goals be based on growth, or could mastery/attainment SLOs ever be allowable? To meet the statutory and Administrative Code requirements for measuring student growth, Illinois teachers must base all SLOs on growth targets. Mastery or attainment SLOs do not meet the requirement to show demonstrable change in a student s or group of students knowledge or skills, as evidenced by gain and/or attainment on two or more assessments, between two or more points in time (Illinois Administrative Code, Part 50, Sub. A, Sec ). The word attainment in the statutory language should not be misconstrued as allowing SLOs to be set that do not reference a student s starting point and ending point. For example, setting an SLO target that calls for 80 percent of students to pass the final course exam is not appropriate; it does not account for the students skill levels at the beginning of the course, and it does not reference two assessments (a pretest and posttest) at two points in time. Measuring Student Growth Using Type III Assessments 10