1 A New Lens for Examining Cognitive Rigor in Standards, Curriculum, & Assessments What are some implications for the transition to Common Core State Standards? Karin K. Hess, Ed.D., Senior Associate National Center for Assessment, Dover, NH Presentations made in
2 Before we begin Take a couple of minutes to write your personal definition of cognitive rigor as it relates to instruction/learning/assessment.
3 Now, let s apply your rigor definition iti Your class has just read a version of Little Red Riding Hood. What is a basic comprehension question you might ask? What is a more rigorous What is a more rigorous question you might ask?
4 Introducing the Hess Cognitive Rigor Matrix Handout: CRM for English Language Arts Source (article): What exactly do fewer, clearer, and higher standards really look like in the classroom? Using a cognitive rigor matrix to analyze curriculum, plan lessons, and implement assessments (Hess, Carlock, Jones, & Walkup, 2009)
5 Developing the Cognitive Rigor Matrix Different states/schools/teachers use different models to describe cognitive rigor. Each may address something different. Bloom What type of thinking (verbs) is needed to complete a task? Webb How deeply do you have to understand the content to successfully interact t with it? How complex or abstract is the content?
6 Bloom s Taxonomy [1956 ] & Bloom s Cognitive Process Dimensions i  Knowledge -- Define, duplicate, label, list, name, order, recognize, relate, recall Comprehension -- Classify, describe, discuss, explain, express, identify, indicate, locate, recognize, report, review, select, translate Application -- Apply, choose, demonstrate, dramatize, employ, illustrate, interpret, practice, write Analysis -- Analyze, appraise, explain calculate, categorize, compare, criticize, discriminate, examine Synthesis -- Rearrange, assemble, collect, compose, create, design, develop, formulate, manage, write Evaluation -- Appraise, argue, assess, choose, compare, defend, estimate, explain, judge, predict, rate, core, select, support, value Remember Retrieve knowledge from long-term memory, recognize, recall, locate, identify Understand -- Construct meaning, clarify, paraphrase, represent, translate, illustrate, give examples, classify, categorize, summarize, generalize, predict Apply -- Carry out or use a procedure in a given situation; carry out or use /apply to an unfamiliar task Analyze -- Break into constituent parts, determine how parts relate Evaluate -- Make judgments based on criteria, check, detect inconsistencies/fallacies, critique Create -- Put elements together to form a coherent whole, reorganize elements into new patterns/ structures
7 Webb s Depth-of-Knowledge Levels DOK-1 Recall & Reproduction - Recall of a fact, term, principle, concept; perform a routine procedure; locate details DOK-2 - Basic Application of Skills/Concepts -Use of information; conceptual knowledge; select appropriate procedures for a given task; two or more steps with decision points along the way; routine problems; organize/display data; interpret/use simple graphs; summarize; identify main idea; explain relationships; make predictions DOK-3 - Strategic Thinking - Requires reasoning, or developing a plan or sequence of steps to approach problem; requires decision making or justification; abstract, complex, or non-routine; often more than one possible answer; support solutions or judgments with text evidence DOK-4 - Extended Thinking - An investigation or application to real world; requires time to research, problem solve, and process multiple conditions of the problem or task; non-routine manipulations; synthesize information across disciplines/content areas/multiple sources
8 DOK is about depth & complexity not t difficulty! The intended student learning outcome determines the DOK level. What mental processing must occur? While verbs may appear to point to a DOK level, it is what comes after the verb that is the best indicator of the rigor/dok level. Describe the process of photosynthesis. Describe how the two political parties are alike and different. Describe the most significant effect of WWII on the nations of Europe.
9 Hess Cognitive Rigor Matrix: Applies Webb s DOK to Bloom s Cognitive Process Dimensions Depth + Level 1 Recall & Level 2 Basic Skills & Level 3 Strategic thinking Reproduction Concepts Thinking & Reasoning Remember - Recall, locate basic facts, details, events Level 4 Extended Thinking Understand - Select appropriate words to use when intended meaning is clearly evident - Specify, explain relationships - summarize identify main ideas - Explain, generalize, or connect ideas using supporting evidence (quote, example ) -Explain how concepts or ideas specifically relate to other content domains or concepts Apply -Use language structure (pre/suffix) or word relationships (synonym/antonym) to determine meaning Use context to identify meaning of word - Obtain and interpret t information using text features - Use concepts to solve non-routine problems - Devise an approach among many alternatives to research a novel problem Analyze -Identify whether information is contained in a graph, table, etc. Compare literary elements, terms, facts, events analyze format, organization, & text structures - Analyze or interpret author s craft (literary devices, viewpoint, or potential bias) to critique a text Analyze multiple sources or texts - Analyze complex/abstract themes Evaluate Create Cite evidence and develop a logical argument for conjectures - Evaluate relevancy, accuracy, & completeness of information - Brainstorm ideas - Generate - Synthesize - Synthesize about a topic conjectures based on information within information across observations or prior one source or text multiple sources or knowledge texts
11 Let s practice using the CRM Handout: Little Red Riding Hood CRM template Where would you place your sample questions in the CRMbasic and more rigorous questions?
12 The CR Matrix: A Reading Example Back to Little Red Riding Hood, Depth + Level 1 Recall & Level 2 Level 3 Skills & Strategic thinking Reproduction Concepts Thinking/ Reasoning Remember Level 4 Extended Thinking Understand Apply Analyze Evaluate Create
13 What are some implications for current-to-future t t Curriculum & Instruction? School/Classroom Assessment? Formative Assessments & Progress Monitoring?
14 Figure out where you are now & where your transition i plan begins 1. Revisit your definition of rigor has it changed? In what ways? 2. Establish a shared understanding of what cognitive rigor looks like across content t areas for your district/school/classrooms. 3. What existing curriculum or assessment materials could you re-examine to increase the range of cognitive rigor? 4. What gaps will need to be filled?
15 Cognitive Rigor & Some Implications for Assessment Assessing only at the highest DOK level will miss opportunities to know what students do & don t know go for a range; end high h in selected/prioritized content Performance assessments can offer varying levels of DOK embedded in a larger, more complex task Planned formative assessment strategies and tools can focus on differing i DOK levelsl
16 Examples of formative assessment tools that uncover thinking & require cognitive i rigor Handouts: TBEAR, Bookmark, Reading Strategy Use, Literary Essay graphic organizer
17 Some Related Resources Papers available at Hess, Carlock, Jones, & Walkup (2009). What exactly do fewer, clearer, and higher standards really look like in the classroom? Using a cognitive rigor matrix to analyze curriculum, plan lessons, and implement assessments Hess, K. (2004). Applying Webb s Depth-of-Knowledge (DOK) Levels in reading, writing, math, science, social studies, science [online]: KH08 pdf Also contact Karin Hess about formative tools, CRM Also contact Karin Hess about formative tools, CRM templates, and use of an accompanying 20-minute video produced by NYC public schools.
Exploring Cognitive Demand in Instruction and Assessment Karin K. Hess Over the past decades, educators and psychologists have attempted to develop models for understanding cognitive complexity as it relates
TOOL 1 HESS COGNITIVE RIGOR MATRIX (READING CRM): Applying Webb s Depth-of-Knowledge Levels to Bloom s Cognitive Process Dimensions Revised Bloom s Taxonomy Webb s DOK Level 1 Recall & Reproduction Remember
Understanding Depth of Knowledge and Cognitive Complexity Keystone Exam Review of Items One of the steps in the item-review process involves Pennsylvania educators review of items for cognitive complexity
IACBE Advancing Academic Quality in Business Education Worldwide Bloom s Taxonomy of Educational Objectives and Writing Intended Learning Outcomes Statements International Assembly for Collegiate Business
A Guide for Using Webb s Depth of Knowledge with Common Core State Standards by Karin Hess, Ed.D. 1 This guide consolidates numerous tools educators use to implement Webb s Depth of Knowledge for curriculum
Writing learning objectives This material was excerpted and adapted from the following web site: http://www.utexas.edu/academic/diia/assessment/iar/students/plan/objectives/ What is a learning objective?
A Guide for Writing Student Learning Outcomes Office of the Provost University of Florida Continuous Quality Enhancement Series Institutional Assessment Timothy S. Brophy, Director Website: http://assessment.aa.ufl.edu/
Writing Learning Outcomes What is a learning outcome? Learning outcomes are statements that indicate what learners will know or be able to do at the completion of a learning experience (program, conference,
Revised Bloom s Taxonomy Revised Bloom s Taxonomy (RBT) employs the use of 25 verbs that create collegial understanding of student behavior and learning outcome. Bloom s Revised Taxonomy Taxonomy of Cognitive
Alignment of Taxonomies Bloom s Taxonomy of Cognitive Domain Bloom s Taxonomy Cognitive Domain Revised Cognitive Demand Mathematics Cognitive Demand English Language Arts Webb s Depth of Knowledge Knowledge
Black Board Vista project Checklist How to use this Check List... 2 Analysis... 5 Targeted Students / Identify... 5 Content Analysis... 5 Design / Identify... 5 Objectives... 5 Evaluation / Validation...
HONORS INDEPENDENT STUDY GUIDELINES: According to the Lake Land College catalog course description, independent study is designed to permit the student to pursue a course of study not typically available
APPENDIX Revised Bloom s Taxonomy In 1956, Benjamin Bloom and his colleagues published the Taxonomy of Educational Objectives: The Classification of Educational Goals, a groundbreaking book that classified
March 2010 BLOOM'S TAXONOMY AND CRITICAL THINKING Critical Thinking across the Curriculum Page 2 of 7 Contents Bloom's Taxonomy and Critical Thinking... 3 Bloom s in a group ICT project -... 4 Learning
SLO Action Verbs Action verbs are abundant in the English language, but how to do we know which ones are right to include in our SLO statements? Benjamin Bloom, an American educational psychologist, created
Lesson Planning 101: Essential Parts of a Lesson Plan by Sarah Sahr firstname.lastname@example.org Part 1: Objectives No one would argue that writing a good lesson plan starts with an excellent objective or two Let s
Getting Started A working definition of a performance-based objectives: A Learning objective is a statement that describes the specific skills or knowledge a student will be able to demonstrate as a result
CREATING LEARNING OUTCOMES What Are Student Learning Outcomes? Learning outcomes are statements of the knowledge, skills and abilities individual students should possess and can demonstrate upon completion
What is a learning outcome? A learning outcome states what a student should be able to do or know as a result of having completed a particular course or program. Learning outcomes can include knowledge,
Webb Webb s Depth of Knowledge Guide Career and Technical Education Definitions 2009 1 H T T P : / / WWW. MDE. K 12.MS. US H T T P : / / R E D E S I G N. R C U. M S S T A T E. EDU 2 TABLE OF CONTENTS Overview...
NAME: Lesson Plan Template DATE: Updated 11/14/2011 Title of Lesson: Subject Grade Level DOK Level(s) Unit Length Resources/Supplies needed for this lesson- GLE/CLE Code & Specific Expectation / Process
Resources for Writing Program Learning Outcomes Supplementary Materials for Writing and Revising Learning Outcomes Workshop Presented Jointly by TLA and Student Affairs LEVEL OF SPECIFICITY AND REACH Learning
TAXONOMY OF EDUCATIONAL OBJECTIVES (Excerpts from Linn and Miller Measurement and Assessment in Teaching, 9 th ed) Table 1 Major categories in the cognitive domain of the taxonomy of educational objectives
Degree- Level Expectations and Course Learning Outcomes Introduction to DLEs Degree-Level Expectations (DLEs) are a threshold framework for the expression of the intellectual and creative development of
Bloom s Taxonomy Each of the three categories requires learners to use different sets of mental processing to achieve stated outcomes within a learning situation. Benjamin Bloom (1913-1999) was an educational
Name Subject Area: Unit: Topic: Grade Level: Example Reading, Math, Social Studies, Science Example Biographies, Addition/Subtraction, Wars, Environment Example Famous Women, 4 digit, World War II, Recycling
UNDERSTAND Construct meaning from instructional messages. What is the main idea of? Can you find an example of...? How would you summarize? What might happen next? How do you explain? What ideas or facts
Learning Outcomes Workbook Student Services How to write Learning Outcomes Academic Affairs July 2012 Overview of the Learning Outcomes Workbook This workbook will serve as your introduction to the concept
Writing Student Learning Outcomes for an Academic Program Student learning outcomes (SLOs) are statements of the knowledge, skills, competencies, or attitudes that students are expected to gain through
Bloom's Taxonomy Revised Key Words, Model Questions, & Instructional Strategies Bloom s Taxonomy (1956) has stood the test of time. Recently Anderson & Krathwohl (2001) have proposed some minor changes
Outcome-Based Education Outcome based education A student-centred learning system focusing on measuring student performance. Does not specify any specific style of teaching just that the student be able
11/20/2009 21st century learning GERLINDE OLVERA, M.ED., JENNIFER ELKINS and JoHN.R.WALkUP, PH.D. Depth of Knowledge in the 21st Century As a principal of (fictitious) Memorial Junior High, and an observer
Developing Clear Learning Outcomes and Objectives The Design phase of a typical curriculum development process (Analysis, Design, Development, Implementation, Evaluation) is largely concerned with developing
Selecting Verbs for Task Statements Choosing the most appropriate verb to begin a task statement takes practice. The lists below contain verbs arranged according to learning domain and according to concept.
Critical Thinking Skills Student Edition 2010 Document created by DCPS Critical Thinking Classrooms PLC Sources DeBono, E. (1999). Six thinking hats. New York: Back Bay Books. Foundation for Critical Thinking.
Training & Instructional Design Creating a Lesson Plan Creating a Lesson Plan Learning Objectives Write measurable objectives for a training program Write learning objectives that use Blooms Taxonomy to
Three domains of learning Leslie Owen Wilson all rights reserved Three domains of learning What are the differences between the cognitive, affective, and psychomotor taxonomies? There are three main domains
Looking at the First Year Objectively Stephen C. Zerwas University of North Carolina Greensboro 1 Measuring Student Learning What do we want students learn? Do students have the opportunity to learn it?
2.5: Inquiry Tutor/Student Handout 2.5.1 (1 of 2) Three-Story House (Costa s Levels of Questioning) To better understand the content being presented in their core subject areas, it is essential for students
ACT Aspire Information Meeting September 29, 2015 Shifts in English Language Arts Standards Alabama Standards are called the College and Career Readiness Standards. In 2010 the state adopted these standards.
SLOs, Bloom s Taxonomy, Cognitive, Psychomotor, and Affective Domains. Benjamin Bloom (1948) developed classifications of intellectual behavior and learning in order to identify and measure progressively
Educational Goals and Objectives A GUIDE TO DEVELOPING LEARNER BASED INSTRUCTION Educational Objectives for this Presentation At the end of this presentation you will be able to: Compare and contrast educational
nd Grade Science Unit Project Unit A: Physical Sciences Magnets Teacher Planning Guide The Scientific Method-Flow Map Find a problem and ask a question. Research the problem and find out all you can. Make
Bloom s Taxonomy Bloom s Taxonomy provides an important framework for teachers to use to focus on higher order thinking. By providing a hierarchy of levels, this taxonomy can assist teachers in designing
The Six Levels of Questioning Level 1 Knowledge Exhibit memory of previously-learned materials by recalling facts, terms, basic concepts, and answers. who what why when where which omit choose find how
Chapter 2 Determining Learning Objectives In this chapter we help you identify the goals and objectives you want students to reach by the end of the course, and provide guidelines for using your objectives
Degree- Level Expectations and Course Learning Outcomes Introduction to DLEs Degree-Level Expectations (DLEs) are a threshold framework for the expression of the intellectual and creative development of
Support Materials for Core Content for Assessment Version 4.1 Mathematics August 2007 Kentucky Department of Education Introduction to Depth of Knowledge (DOK) - Based on Norman Webb s Model (Karin Hess,
Bloom s Taxonomy How will it impact in your classroom? Eileen Herteis The Gwenna Moss Teaching & Learning Centre What is Bloom s Taxonomy? A theory to identify cognitive levels (Levels of thinking) Represents
Multiple Choice Items that Test Higher Learning Objectives To assess higher learning objectives with multiple choice items you must use questions that students cannot answer by relying solely on memory.
OBJECTIVES A. Compliance and Consumer Protection A1 Recognize the impact of regulation and legislation on the mortgage industry A1.1 Recognize requirements related to financial reporting and other reporting
Core Competencies for Public Health Practice Lessons Learned (USA) Joan P. Cioffi, Ph.D. Member, Council on Linkages Between Academia and Public Health Practice Core Competency Workgroups 2001 and 2009
6 Week Strategic Onboarding Program: Creating a Culture of Learning to Increase Success for Financial Services First Timers Matt Ragan- GMAC Mortgage The New Hire Process A successful New Hire On-boarding
Kindergarten Science Unit Project Unit : Life Sciences Effects of Tap, Purified, and Sea Water on Plants. Teacher Planning Guide The Scientific Method-Flow Map Find a problem and ask a question. Research
Writing Educational Goals and Objectives What are Goals? Goals are broad, generalized statements about what is to be learned. Think of them as a target to be reached, or "hit." What are Instructional Objectives?
Unit 6: Handout 1 Writing and Using Aims and Learning Outcomes: A Short Guide The following is a copy (with minor adaptations) of a handout written for use within Dentistry and therefore includes references
Tool 4.1 Relationship between training outcomes and training components Purpose Use this tool to demonstrate the impact of coaching on teachers practices. 4.2 Coaching interactions Gain an understanding
Arthur Costa s Levels Arthur Costa s Model of Intellectual Functioning in Three Levels from Developing Minds: A Resource Book for Teaching Thinking Level 1: defining Ex. What is the definition of "lunar
WHITE PAPER Differentiating Mathematics Instruction So EVERYONE Learns by Jennifer Taylor-Cox, Ph.D. Educational Consultant, Author and Speaker Differentiated mathematics instruction is powerful way to
A GUIDE TO DEVELOPING COGNITIVE LEARNING OBJECTIVES Jolly T. Holden, Ed.D Table of Contents Introduction... 3 Classification of Learning Objectives... 3 General or overall objective... 3 Specific objective...
Planning your questions using Bloom s Taxonomy Taken from The Ultimate Teaching Manual Gererd Dixie published by Continuum - available at all good book stores from 14 th April Research (Wragg and Brown,
The italicized script provides direction for the presenter while the plain text script is the narrative for the presenter. All handout titles are included in the slide descriptions below and are numbered
Step by Step Guide to Developing Targeted Professional Development Sessions Aligned with the Core Body of Knowledge Step1. Gather Information 1. If possible, conduct an observation of the early childhood
How to Write Learning Outcomes This document is a work in progress. As faculty utilize these guidelines, they will make suggestions for improvement. Changes will follow. What are learning outcomes? Learning
Connections to the Common Core State Standards for Literacy in Middle and High School Science This document is based on the Connections to the Common Core State Standards for Literacy in Science and Technical
First Grade Science Unit C: Earth Science Unit Project Weather Station Teacher Planning Guide The Scientific Method-Flow Map Find a problem and ask a question. Research the problem and find out all you
Integrating Critical Thinking Skills into the Standard Curriculum Bob Iseminger, Pieces of Learning, IAGC, 02/09/16 email@example.com WHAT IS CRITICAL THINKING? Accepting ideas and acting on them
Bloom s Taxonomy: A New Look at an Old Standby Traditional Hierarchy of Thinking Processes In 1956, Benjamin Bloom wrote Taxonomy of Educational Objectives: Cognitive Domain, and his sixlevel description
The Marzano Framework Design Question 1 Element 1 Providing Clear Learning Goals and Scales (Rubrics) The The School District of Palm Beach County, Fl DQ1: Providing Clear Learning Goals and Scales (Rubrics)
What Connections Do I Make As I Read? What Connections Do I Make As I Read? Strategy: Make Connections Good readers note pieces of text that relate to or remind them of: their own lives, such as past experiences
The Learning Skills Pyramid Brett A. Brosseit, 2013 To develop strong critical thinking and legal analysis skills, students need to: Develop new patterns of thinking Understand the mental processes they
Education in our times must try to find whatever there is in students that might yearn for completion, and to reconstruct the learning that would enable them autonomously to seek that completion. Allan
RUBRIC IMPLEMENTING STANDARDS FOR MATHEMATICAL Page 1 of 5 Using the Rubric: Review each row corresponding to a mathematical practice. Use the boxes to mark the appropriate description for your task or
Assessment and Learning Partnerships Short Course for School Leaders Using Data for Decision-Making Unit 2 Data is essential in monitoring the learning progress a student is making. It is integral in identifying
CUBING STRATEGY Edition: Ballantine Books: New York 1958; Corresponds to pp. 55-56 Purpose of the Strategy Developed by the Cowan and Cowan (1980), the cubing strategy allows discussion of a topic within
Understanding the Intellectual Levels It is recommended best practice in accounting education to give students as much guidance as possible on how much study is required in order to achieve syllabus aims
Developing Your Course Outline Facilitator: Dr. Mervin E. Chisholm h Manager/Coordinator, IDU What Will We Be Doing Today? Use instructional design models and to decide on appropriate it approaches to:
ALIGNMENT STUDY FINAL REPORT DISTRICT OF COLUMBIA PUBLIC SCHOOLS COMPREHENSIVE ASSESSMENT SYSTEM ALTERNATE ASSESSMENT (DC CAS- ALT) Presented to District of Columbia Public Schools June 2007 Karin Hess,
GRADE: Sixth Grade NAME OF ASSESSMENT: Reading Informational Texts and Argument Writing Performance Assessment STANDARDS ASSESSED: Students will cite textual evidence to support analysis of what the text