Assessing Growth in Critical Reading Skills Middle Schools Leadership Conference Orlando, April 30, 2004 Pat Howard Florida Center for Reading Research
What we heard here In order to change things, we need to know where we are (Phyllis Hunter, 2004) We need to use more effective assessments to show the change brought about by our efforts (Phyllis Hunter, 2004) Principals must provide leadership in using assessments for screening, progress monitoring, diagnosis and outcomes (Laura Hassler,, 2004; Miranda Free, 2004) Research shows that teachers did a lot of assessment but did almost NO comprehension instruction (Paige Pullen, 2004)
What we know Who to target for more intensive reading instruction FCAT performance: Level 1 or 2 on CRT, less than 50th percentile on NRT Critical elements needed for successful reading performance The fab 5 Phonemic awareness Phonics Fluency Vocabulary comprehension
What we want to know How to distinguish different groups of students who failed to meet the standards Current research on student profiles. Not all poor readers are alike. How do we differentiate for instruction? (Holly Lane, 2004) A testing model that is efficient yet effective and flexible Tests that facilitate the process How to manage the data and use the data for student, school and system decisions
The FCAT Places special demands on fluency Passage length at different levels 3rd grade - 325 words 7th grade - 816 words 10th grade - 1008 words The FCAT become more and more challenging with increasing grades because of increase in percentage of items that demand relatively complex inferential and other kinds of reasoning skills.
What research tells us Study by Torgesen,, et al (2004) Identify the most important predictors of performance on reading comprehension as measured by the FCAT reading test
Variables and Tests Administered Reading Fluency and Efficiency Oral Reading Fluency passages (3 types) Test of Word Reading Efficiency (TOWRE) Gray Oral Reading Test -4th Ed. (GORT-4) Listening and Reading Comprehension FCAT passages read aloud by examiner GORT-4 4 Comprehension
Variables and Tests Administered Verbal Reasoning Wechsler Abbreviated Scale of Intelligence (WASI) - Vocabulary and Similarities subtests Non-Verbal Reasoning WASI Matrix Reasoning and Block Design subtests Working Memory Listening Span and Reading Span
Results 3rd Grade 4 Predictors: Fluency, language, nonverbal reasoning, memory Fluency Fluency completely dominated the remaining three predictors Language (verbal reasoning and comprehension) dominated nonverbal reasoning and memory Nonverbal reasoning and memory were equally non dominant
Results 7th Grade 3 Predictors: Fluency, verbal knowledge and reasoning, memory Fluency and verbal reasoning equally strong and completely dominated memory 10th Grade 3 Predictors: Fluency, verbal knowledge and reasoning, memory Verbal reasoning dominated all, fluency dominated memory
What research tells us Study Valencia & Riddle-Buly (2004) Understand the garden variety test failure - those students typically found in the regular classroom who are experiencing reading difficulty Targeted word identification (decoding), meaning (comprehension and vocabulary), and fluency (rate and expression)
Tests Administered Woodcock Johnson -Revised (WJ-R) Letter-word Identification and Word Attack subtests Qualitative Reading Inventory - II (QRI-II) II) Reading Accuracy, Acceptability, Rate, Expression, Comprehension Peabody Picture Vocabulary Test (PPVT-R) Vocabulary Meaning State Fourth-Grade Passages Reading Accuracy, Acceptability, Rate, Expression
Tests Administered Woodcock Johnson -Revised (WJ-R) Letter-word Identification and Word Attack subtests Qualitative Reading Inventory - II (QRI-II) II) Reading Accuracy, Acceptability, Rate, Expression, Comprehension Peabody Picture Vocabulary Test (PPVT-R) Vocabulary Meaning State Fourth-Grade Passages Reading Accuracy, Acceptability, Rate, Expression
Tests Administered Woodcock Johnson -Revised (WJ-R) Letter-word Identification and Word Attack subtests Qualitative Reading Inventory - II (QRI-II) II) Reading Accuracy, Acceptability, Rate, Expression, Comprehension Peabody Picture Vocabulary Test (PPVT-R) Vocabulary Meaning State Fourth-Grade Passages Reading Accuracy, Acceptability, Rate, Expression
Cluster Analysis Student Profile #1 #2 #3 #4 #5 #6 Word Ident. -- ++ - - + + Meaning -- -- - + ++ - Fluency -- ++ ++ - - - Description Very weak in all areas - 9% Reads fast, accurate but no meaning - 18% Reads indiv.. words; rate high with errors - 15% Labored reading for meaning - 17% Slow but relatively accurate - 24% Good decoding, slow, poor meaning - 17%
Cluster Analysis Student Profile #1 #2 #3 #4 Word Ident. -- ++ - - Meaning -- -- - + Fluency -- ++ ++ - Reading Strategy Emphasize phonics and phonemic awareness Slow down, attend to meaning Phonics and comprehension Decoding and Fluency #5 #6 + + ++ - - - Rereading to build fluency Increase reading rate and comprehension
Research indicates. Performance on tests of fluency and verbal knowledge/reasoning predict scores on the FCAT (Torgesen( Torgesen,, et al) Younger students (5th grade) who fail state reading tests are more likely to show weaknesses in fluency and vocabulary/ comprehension (Valencia and Riddle-Buly Buly) It is less likely that low-performing students will be weak in all critical areas but will show profiles of strengths and weaknesses (Valencia and Riddle-Buly Buly)
Reading Assessment Flowchart FCAT Performance 1-2 Word Reading Efficiency Skills Yes No Verbal Reasoning Skills Decoding Skills Yes No Comp. Strategies Vocabulary Background Knowledge Fluency Yes No Phonics &/or PA Higher Order Thinking Test Taking Adapted from Torgesen & Hayes (2003)
Sight Word Efficiency Some Test Options TOWRE CBA Sight Words Oral Reading Fluency Verbal Reasoning SDRT* GORT-4 OWLS TOLD-3 PPVT-3 GRADE* CELF-4 WJ-III QRI-3 WASI *Group Administered
Some Test Options Phonemic Decoding TOWRE WRMT QRI-3 WJ-III Word Attack and Letter-Word Identification Fluency CBM Fluency Practice FCAT Passages Grade Level Text Passages GORT-IV
Oral Reading Fluency Options FCAT Practice Passages Grade-level Text Curriculum-Based Measurement (CBM) Oral Reading Fluency Passages
Correct words per minute 200 180 160 140 120 100 80 60 40 20 0 7th Grade Average Reading Fluency Scores Across 3 Measures 1 2 3 4 5 FCAT Level FCAT Text ORF Torgesen et al. (2004)
Data Analysis Set up a system that Selects students at Level 1 or 2 of FCAT (Screening) Sets goals (short and long term if possible) and calculates trend lines (where student is headed) Profiles students on word recognition efficiency Profiles student data on measures of verbal reasoning and fluency Monitors fluency with one or more measures (progress monitoring) Accepts group administered measures of comprehension (Gates-MacGinitie MacGinitie,, GRADE, QRI, etc. (outcome measures)
Data Analysis Set up a system that Indicates where interventions are implemented and changed Permits aggregating and disaggregating of data Provides immediate feedback to teachers and administrators
Data Analysis Resources www.interventioncentral.org (charting student progress over time in multiple subject areas: FREE) www.edformation.com (6-8th grade reading fluency, math, spelling, writing: $4-7.00 per student) www.dibels.uoregon.edu (6th grade reading fluency: $1.00 per student)
www.interventioncentral.org
www.interventioncentral.org
www.edformation.com
http://dibels.uoregon.edu
C A B http://dibels.uoregon.edu
Other Resources Project CENTRAL - projcentral@mail.ucf.edu Curriculum-based measurement training in reading fluency, math, spelling, writing; survey level assessment Online course - Applied Data Analysis for School Leaders (contact Laura Hassler, lhassler@mailer.fsu.edu) Florida Center for Reading Research - Valid and reliable reading diagnostic measures www.fcrr.org/assessment
Using Data to Inform Instruction Establish a process for routinely analyzing data at the student, classroom, grade level, school, and district levels Show how data drives decisions Share innovative practices with others And if you think this is tough,..
phoward@fcrr.org