Promoting Math Achievement through a Tiered System of Assessment and Intervention Renee O. Hawkins
The Big Picture (National Mathematics Advisory Panel, 2008) National economic competitiveness dependent on educated workforce Accelerating retiring in science and engineering concurrent with job growth Stress to sustain workforce For individuals, math performance correlates with access to and graduation from college, and income.
Current Status of Math Achievement in U.S. American students achieve at a mediocre level compared to peers worldwide. National Report Card shows positive trends at 4 th and 8 th grade but 32% of students are at or above proficient level in 8 th grade; only 23% are proficient in 12 th grade. (NMP, 2009)
Current Status of Math Achievement in U.S. Big demand for remedial math education at college level Large, persistent achievement gaps in math related to race and income Sharp decline in math achievement happens as students begin coursework in algebra (NMP, 2009)
Why is Algebra Important? Algebra is a gateway to later achievement. Completion of Algebra II correlates significantly with success in college and income. Students who complete Algebra II are more than twice as likely to graduate college. Differences in college graduation rates for AA and Hispanic students vs. general student population are half as large as the differences for students who do not complete Algebra II. (NMP, 2009)
Using a Tiered Approach To prevent math difficulties To ensure adequate preparation of students for algebra coursework To help struggling students Includes increasing and decreasing intensities of intervention AND assessment
Tier 1: Core Instruction High quality math instruction for all students Curricula should be focused, coherent progression of learning, with emphasis on proficiency with key topics. Provide sufficient practice to develop automatic recall of facts All prepared students have access to algebra Instruction focusing on critical foundations of algebra (NMP, 2009)
Table 2: Benchmarks for the Critical Foundations (NMP, 2009) Fluency With Whole Numbers 1) By the end of Grade 3, students should be proficient with the addition and subtraction of whole numbers. 2) By the end of Grade 5, students should be proficient with multiplication and division of whole numbers. Fluency With Fractions 1) By the end of Grade 4, students should be able to identify and represent fractions and decimals, and compare them on a number line or with other common representations of fractions and decimals. 2) By the end of Grade 5, students should be proficient with comparing fractions and decimals and common percent, and with the addition and subtraction of fractions and decimals Geometry and Measurement 1) By the end of Grade 5, students should be able to solve problems involving perimeter and area of triangles and all quadrilaterals having at least one pair of parallel sides (i.e., trapezoids). 2) By the end of Grade 6, students should be able to analyze the properties of twodimensional shapes and solve problems involving perimeter and area, and analyze the properties of three dimensional shapes and solve problems involving surface area and volume
Tier 1: Core Instruction Includes use of formative assessment Coooperative learning and Direct Instruction identified as effective instructional approaches Cautious use of calculators Increased emphasis on effort Text books are too long; districts should focus on specific topics and work with publishers High quality computer programs may be an effective instructional tool (NMP, 2009)
Tier 1: Screening Universal screening of students Using valid measures Typically 3-4 times a year Screening data used to identify at-risk students to provide early intervention
Tier 2 ADDITIONAL assistance provided to at-risk students or students with slow progress SUPPLEMENTAL small group math instruction targeting specific math proficiencies 20-40 minutes, 4-5 times each week Ongoing progress monitoring
Tier 3 ADDITIONAL intervention for students not improving (or not improving fast enough) in response to Tier 2 More intensive, and individualized intervention One-on-one tutoring with a variety of instructional strategies In some cases, special education services and specialized personnel may be included. Ongoing, frequent progress monitoring
Institute of Education Sciences Practice Guide (Gersten et al., 2009) Guide for providing tiered math instruction and intervention Includes 8 recommendations with supporting evidence
Recommendation 1. Screen all students to identify those at risk for potential mathematics difficulties and provide interventions to students identified as at risk.
Universal Screening Systematic screening of all students at least twice a year Close monitoring of students who score poorly (especially those scoring near the cut point) Measures selected by a team with expertise in measurement and math instruction
Universal Screening Results from previous year s state testing may be a first stage of screening. Same screening measure(s) should be used across all schools in a district. Data may help with district level instructional decisions Research and evaluation staff may analyze data biannually
Evaluating Screening Measures Predictive validity How well score predicts later math achievement Look for measures with predictive validity coefficients of at least.60 within a school year
Evaluating Screening Measures Content validity Measures should target the instructional objectives of curriculum for students grade level. Focus on most critical content Lower elementary= whole numbers and computational efficiency Upper elementary= rational numbers and computational efficiency
Evaluating Screening Measures Reliability Consistency of measure Look for measures with reliability coefficients of.80 or higher
Evaluating Screening Measures Efficiency Should require no more than 20 minutes Select most efficient without sacrificing validity and reliability Appropriate for progress monitoring
Progress Monitoring Measures in Math (Foegen, Jiban, & Deno, 2007) Literature review on CBM math measures Two broad approaches for developing CBM tasks: Curriculum sampling Robust indicators
Curriculum sampling Measures made by taking representative samples of year s math curriculum Applied to both computation and applications problems Advantages: Direct link to curriculum Useful for intervention planning Disadvantages: Different measures for each year Doesn t assess growth over several years Results don t generalize (Foegen et al., 2007)
Robust indicators Measures that represent broadly defined proficiency in math Not necessarily reflective of a specific curriculum (like oral reading fluency) Advantages: 1 measure can be used across years Can assess growth Disadvantage: Less useful for intervention planning (Foegen et al., 2007)
Technical Adequacy of Elementary Math CBM Research examining robust indicators and curriculum sampling approaches Generally strong reliability, with coefficients above.80 Lower reliability for more complex problems Validity coefficients ranging from.50-.70, modest but similar to commercially available math achievement tests (Foegen et al., 2007)
Elementary Math CBM Measures Monitoring Basic Skills Progress (MBSP) (Fuchs, Hamlett, & Fuchs, 1998, 1999) Computation and concepts and applications measures available Curriculum sampling measure using TN curriculum for grades 1-6 (computation) and 2-6 (concepts/apps) Problem-solving measures- using word problems from instructional curriculum (Jitendra, Sczesniak, & Deatline-Buchman, 2005) Basic fact probes (Foegen et al., 2007)
Technical Adequacy of Early Math CBM Early math research only on robust indicators Quantity discrimination, identifying the missing number, number identification Strong reliability Validity coefficients ranging from.40-.60, higher (.70) for grade 1 (Foegen et al., 2007)
Recommendations from the Review Consider the advantages and limitations of approaches. If considering MBSP Computation, evaluate match with curriculum. Performance does not improve as rapidly as seen with reading CBM. Increased sensitivity may be needed for students with significant difficulties. May need several weeks of data for decision making (Foegen et al., 2007)
Roadblocks and Solutions Resistance in allocating time Establish data collection teams. Select efficient measures. Questions about assessing students who are doing fine Creates a complete picture of performance
Roadblocks and Solutions False positives and false negatives Follow-up assessments and ongoing evaluation of cut-scores May identify large numbers of at risk students Evaluation of resources and greatest needs Consider addressing problem at school, class, or grade level
Recommendation 2. Instructional materials for students receiving interventions should focus intensely on in-depth treatment of whole numbers in K through grade 5 and on rational numbers in grades 4 through 8. These materials should be selected by committee.
K-5 Intervention (and for older, struggling students) Tiers 2 and 3 should focus almost exclusively on whole numbers and operations. Counting Number composition and decomposition Meaning of addition and subtraction and reasoning that underlies algorithms Solving problems using whole numbers Place value Visual representations and number lines Fluency for basic math facts
Grades 4-8 Intervention Tiers 2 and 3 should focus on in-depth coverage of rational numbers and advanced topics in whole number arithmetic. Fractions Decimals Ratios Percents Visual representations Solving problems with fractions, decimals, ratios, and percents Fluency building and review of whole numbers
Selecting Intervention Curricula District math experts should review materials for 4 criteria: 1. Integration of computation with solving problems and pictorial representations 2. Stresses reasoning behind calculation methods 3. Builds algorithmic proficiency 4. Includes frequent review Assessment to help place students
Roadblocks and Solutions Intervention may not work if not aligned with core instruction. Intervention is supplemental Intervention materials may cover topics not essential to building some basic competencies. Doesn t have to cover everything
Recommendation 3. Instruction during the intervention should be explicit and systematic. This includes providing models of proficient problem solving, verbalization of thought processes, guided practice, corrective feedback, and frequent cumulative review.
Effective Instruction Introduces concepts in a logical order Provides ample opportunity for students to apply concepts Presents step by step guides for how to solve problems Uses think alouds Scaffolds practice Provides specific and corrective feedback Incorporates cumulative review
Checklist for Effective Instruction Teacher demonstration Student verbalizations Guided practice Corrective feedback Cumulative review
Roadblocks and Solutions Intervention agents may need additional support Professional development; math coaches Intervention materials may not have enough models, think alouds, practice, and review Team of experts can add components to program
Recommendation 4. Interventions should include instruction on solving word problems that is based on common underlying structures.
Instructional Objectives for Word Problems Students should learn: The structure of various problem types Problem types are groups of problems with similar math structures Examples include change, compare How to categorize problems by type How to solve different problem types To transfer known solution methods to unfamiliar problems
Instructional Strategies Initially teach solution rules with examples, followed by practice. Use visual representations of problem types. Use explicit instruction to help students discern critical underlying structure of problems. Have students identify and explain relevant and irrelevant information in problems.
Roadblocks and Solutions Curricular material may not classify problem types. May need help from school/district experts to identify problem types and instructional sequence Discriminating among problem types gets complicated. May also need professional development
Recommendation 5. Intervention materials should include opportunities for students to work with visual representations of mathematical ideas and interventionists should be proficient in the use of visual representations of mathematical ideas.
Visual Representations Number lines Number paths Strip diagrams Drawings In early grades, manipulatives help with teaching basic concepts with whole numbers
Roadblocks and Solutions Limitations of interventional materials May need help from school/district experts to develop visuals Perception that using manipulatives takes too much time Should be used strategically at the initial stages Interventionists may not understand math ideas underlying representations. Professional development and support from local experts
Recommendation 6. Interventions at all grade levels should devote about 10 minutes in each session to building fluent retrieval of basic arithmetic facts.
Carrying out the recommendation Beginning in grade 2 Goal is quick retrieval of facts using the digits 0-9 For K-2, include explicit instruction in methods for efficient problem solving For grades 2-8, teach use of commutative, associative, and distributive properties
Strategies Technology (computer games) Flash cards Incremental Rehearsal Folding-In Cover, Copy, & Compare Presenting facts in number families Cumulative review
Roadblocks and Solutions Students may be bored with fluency practice Games- make it fun Curricula may not include enough practice or incorporate strategies Supplemental program
Recommendation 7. Monitor the progress of students receiving supplemental instruction and other students who are at risk.
Assessment Plan Assess Tier 2, Tier 3, and at-risk Tier 1 students at least once a month. Use valid, reliable, and efficient measures Math CBM- curriculum sampling or robust indicators Provide broad perspective on performance Increase intensity of assessment with intensity of intervention
Assessment Plan Use assessments embedded in intervention materials. May be daily for K-1 and biweekly for 2-6 Use data to regroup students when appropriate.
Roadblocks and Solutions Difficult to group students because of various instructional levels Consider grouping across classes Insufficient time for progress monitoring Consider training other school staff
Recommendation 8. Include motivational strategies in tier 2 and tier 3 interventions.
Strategies Reinforce effort and engagement Consider using rewards Completion-contingent rewards Performance-contingent rewards Set goals and have students graph progress
Roadblocks and Solutions Rewards may undermine intrinsic motivation Research consistently shows that rewards and praise improve academic performance without decreasing interest in learning. Fade rewards over time Difficult to select appropriate rewards Consider student s interests and include parents Not enough time to deliver rewards Praise requires little time Distribute tangibles at the end of the day
Summary There are specific actions schools can take to maximize the effectiveness of math instruction and intervention delivered through a tiered model IES recommendations combined with report from National Math Advisory Panel provide a framework for structuring math instruction and intervention
References Foegen, A., Jiban, C., & Deno, S. (2007). Progress monitoring measures in mathematics: A review of the literature. The Journal of Special Education, 41, 121-139. Fuchs, L. S., Hamlett, C. L., & Fuchs, D. (1998). Monitoring basic skills progress: Basic math computation (2 nd ed.). Austin, TX: PRO-ED. Fuchs, L. S., Hamlett, C. L., & Fuchs, D. (1999). Monitoring basic skills progress: Basic math concepts and applications. Austin, TX: PRO-ED. Gersten, R., Beckmann, S., Clarke, B., Foegen, A., Marsh, L., Star, J. R., & Witzel, B. (2009). Assisting students struggling with mathematics: Response to intervention (RtI) for elementary and middle schools (NCEE 2009-4060). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Jitendra, A. K., Sczesniak, E., & Deatline-Buchman, A. (2005). An exploratory validation of curriculumbased mathematical word problem-solving tasks as indicators of mathematics proficiency for third graders. School Psychology Review, 34, 358-371. National Mathematics Advisory Panel. (2008). Foundations for success: The final report of the national mathematics advisory panel. Washington, DC: National Academies Press.