Student Assessment That Boosts Learning Judy Arter Independent Consultant Loren Ford Clackamas Community College January 27-28, 2011 Anderson Conference Shifting From a Grading Culture to a Learning Culture: ASSESSMENT THEORY AND PRACTICE Judy Arter PO Box 470, Beavercreek, OR 97004 judy@bctonline.com Adapted from: Stiggins, R., Arter, J., Chappuis, J., and Chappuis, S. (2006). Student Assessment For Student Learning (CASL), Portland: Pearson Assessment Training Institute. Arter, J. and Chappuis, J. (2006). Creating and Recognizing Quality Rubric (CAR), Portland: Pearson Assessment Training Institute.
Seminar Goals 1. Understand the relationship betwen assessment and student motivation 2. Understand the difference between assessment of and for learning 3. Deepen your understanding of keys to quality student assessment 4. Deepen your understanding of assessment for learning strategies Goals For This Session: Overview of Big Ideas 1. Understand the relationship betwen assessment and student motivation 2. Understand the difference between assessment of and for learning 3. Deepen your understanding of keys to quality student assessment, especially the necessity for clear learning goals for students and choosing the best assessment method for different types of learning goals List of Sessions 1. Student Assessment That Boosts Learning: Overview of big ideas 2. Assessment For Learning and You Be George: Assessment for learning strategies and a concrete example using selected response/short answer assessments 3. Assessment For Learning Using Rubrics: Concrete examples of how to use rubrics to help students learn 4. Rubrics the Good, the Bad, and the Ugly: Characteristics of high quality rubrics for instructional uses 5. Providing Descriptive Feedback to Students: Research on the most effective feedback to give students if the goal is further learning, and a concrete example 6. The Final Frontier: Sound Grading Practices: How can we make grading more productive? FIVE KEYS TO QUALITY CLASSROOM ASSESSMENT Think of a time when you were assessed and it was a negative experience. What made it negative? Next, think of a time when you were assessed and it was a positive experience. What made it positive? Judy Arter and Loren Ford, Anderson Conference, PCC, January 2011 1
KEYS TO QUALITY CLASSROOM ASSESSMENT Accurate Assessment Key 1: Clear Purpose What s the purpose? Who will use the results? What will they use the results to do? Key 2: Clear Targets What are the learning targets? Are they clear? Are they appropriate? Key 3: Sound Design What method? Quality questions? Sampled how? Avoid bias how? Effectively Used Key 4: Effective Communication How manage information? How report? To whom? Key 5: Student Involvement Students are users, too Students can track progress and communicate, too Students need to understand targets, too Students can assess, too Judy Arter and Loren Ford, Anderson Conference, PCC, January 2011 2
QUESTIONS TO ASK OF CLASSROOM ASSESSMENTS Key 1: Clear Purpose: How and by whom will the results of the assessment be used? Big Ideas Different users have different information needs. Students are crucial decision-makers whose information needs must be met during the learning. Questions to Ask Who will use the information instructors, students, others? How will the information be used assessment of or for learning? What information, in what detail, will be required? Do classroom assessment practices meet students' information needs if they are to be among the intended users? Is there a plan for how assessment for and of learning fit together over time? Key 2: Clear Targets: What is to be assessed? Big Ideas Learning targets can be classified into five categories. Learning targets must be clear to both instructors and students. Learning targets common across instructors must be understood in the same way. Classroom assessments must reflect the learning targets that are the focus of instruction alignment. Questions to Ask Are learning targets stated, easy to find, and in user-friendly language? What kinds of achievement are to be taught and assessed knowledge, reasoning, performance skills, products? Are the learning targets the focus of instruction; is there alignment between targets, instruction, and assessment? Key 3: Sound Assessment Design : Are the learning targets translated into high quality assessments that will yield accurate and usable results? Will the results have the level and type of detail to serve the intended purpose? Big Ideas Assessment methods must match learning targets. The sample of questions/tasks is representative of the learning targets. Items, tasks, and scoring guides are of high quality. Bias is minimized. Questions to Ask Do the assessment method(s) (selected response, short answer, extended written response, performance assessment, personal communication) match the kind of learning target(s) to be assessed? Do the learning targets represent what was taught? (Or what will be taught?) Does the relative importance of each learning target match its relative importance during instruction? Is the sample size large enough to inform the decisions intended to be made, or is it part of a larger plan to gather evidence over time? Do the assessment items, exercises, scoring procedure, and scoring guides/rubrics adhere to standards of quality? Is there anything in the assessment itself or in the conditions under which it is administered that could lead to inaccurate estimates of student learning? Judy Arter and Loren Ford, Anderson Conference, PCC, January 2011 3
Key 4: Effective Communication: Is assessment information managed and reported in ways that will meet the needs of users? Big Ideas Communication differs based on assessment purpose. Students receive descriptive feedback during learning. Students track and communicate about their learning. Grading practices accurately communicate about student achievement. Questions to Ask Formative assessment Does feedback focus on what was taught and describe what was done well as well as what needs work? Are formative results communicated so that there is time to act on them? Is feedback descriptive rather than evaluative? Are students engaged in tracking, reflecting on, and sharing their progress? Summative assessment Do grades communicate achievement accurately? Key 5: Student Involvement: Are students involved in the assessment process when appropriate? Big Ideas Students are identified as important users of assessment information. Students understand the learning targets and standards of quality. Assessments are designed so that students can use the results to self-assess and set goals for further action. Students keep track of and share their achievement. Questions to Ask Do students have a clear vision of the intended learning? Is the assessment designed so that students can use the results to self-assess and set goals? Do students receive and offer effective descriptive feedback during the learning? Do students have opportunities to engage in further learning before the graded events? Judy Arter and Loren Ford, Anderson Conference, PCC, January 2011 4
Key 1: Clear Purpose TWO PURPOSES FOR ASSESSMENT Summative Assessment = Assessment OF Learning Assessments used to determine how much students have learned at a particular point in time in order to report status to others. Formative Assessment = Assessment FOR Learning All those activities undertaken by teachers and by their students (that) provide information to be used as feedback to modify the teaching and learning activities in which they are engaged (Black & Wiliam, 1998): assessments used to help students learn more. Black & Wiliam Synthesis of Research on Effects of Formative Assessment From Inside the Black Box, Phi Delta Kappan, October, 1998 What they asked: Does better FORMATIVE assessment = higher achievement? What they found:.4 to.7 SD gain; largest gain for low achievers A.7 standard deviation score gain equals: 25 percentile points on the ITBS (in the middle of the score range) 70 SAT score points (each portion) 4 ACT score points (composite score) Needed improvements to realize the gains: Increased descriptive feedback, reduced evaluative feedback Increased student self-assessment Increased opportunities for students to communicate their evolving learning during the teaching Increased commitment to accurate assessment Judy Arter and Loren Ford, Anderson Conference, PCC, January 2011 5
Key 2: Clear Targets KINDS OF ACHIEVEMENT TARGETS Master Factual and Procedural Knowledge Facts & concepts we want students to know Some to be learned outright Some via reference Use Knowledge to Reason and Solve Problems The mental processes we want students to engage in the ways in which they use knowledge Analyzes Classifies Compares/contrasts Evaluates Infers Synthesizes Demonstrate Mastery of Specific Performance Skills Performances that must be demonstrated and observed heard or seen to be assessed Demonstrates physical skills Speaks/converses/presents Performs Create Quality Products Using knowledge, reasoning, and skills to create a product, where creating the object itself is the focus of the learning Creates writing samples Creates artistic products Creates research reports Creates science exhibits Acquire Positive Affect * Dispositions we want to nurture in students Sees self as capable Desires to learn Possesses positive attitude toward school or subject Demonstrates respect toward self and others *Affect targets are not strictly speaking achievement targets. Rather, they are targets that describe feelings, attitudes, and dispositions. From CASL, Chapter 3, pages 61-74. Examples of each kind of target can be found on page 63 Judy Arter and Loren Ford, Anderson Conference, PCC, January 2011 6
Key 3: Sound Design Possible Assessment Methods and Match to Targets Selected Response/Short Answer Students select the correct or best response Multiple-choice True/False Matching Fill in the blank (one to a few words) Label a diagram Evaluated with an answer key. Extended Written Response Students construct a written answer at least several sentences in length in response to a question or task How are these two similar? Explain the effects of the Stamp Act on the colonists. Evaluated with a checklist or scoring guide. Performance Assessment Assessment based on observation and judgment. Demonstrating skills Developing products Consists of a task (what students will do) and scoring criteria (how you will judge quality) Personal Communication Conversations with students Questions & answers Conferences Interviews Oral examinations Kind of Target Selected Response Extended Written Response Assessment Method Performance Assessment Personal Communication Knowledge Reasoning Performance Skill Product Judy Arter and Loren Ford, Anderson Conference, PCC, January 2011 7
TARGET METHOD MATCH Selecting the appropriate assessment method depends in large part on the kind of learning target to be assessed. A strong match between method and kind of learning target results in accurate information gathered as efficiently as possible. A mismatch occurs when the assessment method is not capable of yielding accurate information about the learning target. Knowledge Targets For each of the knowledge targets below, place a plus sign (+) in each assessment method column that you believe is a strong match. Knowledge Targets SR EWR PA PC Knows that the pitch of a sound depends on the frequency of the vibration producing it Understands the concept of diversity Understands connotations and multiple meanings of words Knows ways to avoid, recognize, and respond to negative social influences and pressure to use alcohol, tobacco, or other drugs (e.g., refusal skills, self-control) Reasoning Targets For each of the reasoning targets below, place a plus sign (+) in each assessment method column that you believe is a strong match. Reasoning Targets SR EWR PA PC Establishes relationships based on evidence and logical argument (e.g., provides causes for effects) Makes reasonable inferences based on what is read Interprets a bar graph Solves problems using rate as a measure (e.g., velocity, acceleration) Performance Skill Targets For each of the performance skill targets below, place a plus sign (+) in each assessment method column that you believe is a strong match. Performance Skill Targets SR EWR PA PC Reads aloud with fluency and expression Participates in civic discussion with the aim of solving current problems Uses equipment and tools to gather data Pronounces words correctly in Spanish: vowel/consonant sounds; dipthongs Product Targets For each of the product targets below, place a plus sign (+) in each assessment method column that you believe is a strong match. Product Targets SR EWR PA PC Creates a personal fitness plan Produces maps, charts, and graphs to scale Builds a table Writes a lab report Judy Arter and Loren Ford, Anderson Conference, PCC, January 2011 8
Think about your responses to the plus/minus activity. Which keys to quality did they address? Key A Negative Experience That Violated This Key What Happened? A Positive Experience That Reinforced This Key What Happened? Key 1: Clear Purpose Key 2: Clear Targets Key 3: Sound Design Key 4: Effective Communication Key 5: Student involvement Judy Arter and Loren Ford, Anderson Conference, PCC, January 2011 9
Judy Arter and Loren Ford, Anderson Conference, PCC, January 2011 10