Intervention Matters! Best Practices and Strategies Keith Henry, Sr. Assessment Manager
Agenda Research Basis Reports from CCRC Types of Intervention Measures of Success Results Developing and Planning Interventions Resources
A Sampling of Research Community College Research Center Columbia University http://ccrc.tc.columbia.edu/college-readiness.html Bridging the Gap: An Impact Study of Eight Developmental Summer Bridge Programs in Texas Improving Students College Math Readiness: A Review of the Evidence on Postsecondary Interventions and Reforms Reshaping the College Transition: Early College Readiness Assessments and Transition Curricula in Four States Reshaping the College Transition: States That Offer Early College Readiness Assessments and Transition Curricula
Overall Conclusions Immediate short-term positive effect on college math coursework Overall positive impact on performance in Math and Writing courses last for 1 ½ years after intervention No evidence of impact on credits attempted/earned or persistence More research needed
Types of Interventions Teacher-directed o College readiness course designed around content measured by ACCUPLACER o Embed content in current Math and English courses Software aligned to ACCUPLACER blueprint o Independent Study o Independent Study with Guidance o Computer Lab o Blended Instruction
Interventions Go by Many Names Enhanced 11 th -12 th grade courses in Math/English Independent Study College Readiness course After School Program Saturday Academy Summer Boot Camp Developmental Ed courses
How Can Results Be Measured? Increases in Placement scores from pre- to post-test o Overall growth Gains in course placement as measured by placement testing o Bypassing courses saves time and money Increases in Diagnostic domain scores from pre- to post-test o Growth in the specific areas measured on Diagnostics Performance in subsequent courses o Sustainable achievement gains
The Takeaway? Intervention Works! Pilots used pre- and post-test models Variety of implementations produced results Improvement measured using Placement scores, Diagnostic scores, and course placements Student achievement improved even in a short amount of time Classroom-based interventions compared results with historical trends
Results
For More Information
Targeted Interventions Improve College Readiness Anne Arundel Community College (MD) Anne Arundel CC partnered with local high school to provide an Intervention Math course 30 students assigned to an intervention math class were atrisk with IEPs, 504 plans, or limited English proficiency 60 students assigned to control group with no intervention Pre- and post-testing with EA and CLM for all students Intervention class took Diagnostics for data to guide instruction
Targeted Interventions Improve College Readiness Anne Arundel Community College (MD) 60% 50% 51.3% 50.0% 40% 33.3% 35.1% 30% Control Group 20% 16.7% 13.0% Diagnostic Group 10% 0% Decrease in Placement Same Placement Increase in Placement
Proving the Value of Test Prep Community College of Aurora (Colorado) Community College of Aurora (Colorado) designed 2 hour long Math Workshops for students to review knowledge and skills prior to initial testing or re-testing with ACCUPLACER More than half of re-testers placed in higher course levels 98 of re-testers earned a C or higher 450 Students Attended Math Workshop 109 Enrolled in a Math Course 98 passed with a C or higher
Enhancing Developmental Ed Mississippi Institutions of Higher Learning Audience: Incoming Freshmen Time: 9 weeks Measure: Increase in Placement Scores Implementation: Academic Support Lab Average Score Gains Pretest Posttest Gain Elementary Algebra 39.82 56.83 17.01 Reading Comprehension 50.68 53.11 2.43 Sentence Skills 64.43 70.19 5.76
Improving Course Placement East LA College Summer Boot Camp (CA) Audience: Entering college freshmen Time: 2 weeks (10 hours total) Measure: Increase in placement test scores and course placement Results: 38% of students using ACCU//MFL increased placement by 1 or more courses
Enhancing Achievement in CTE Jefferson Community College (NY) Audience: High school students in CTE programs Time: 36 weeks Measure: Increase in Diagnostic domain scores Results: Various implementation models were used with all achieving gains. The Lab format provided the strongest gains.
Enhancing Achievement in CTE Jefferson Community College (NY) Percentage of Students with Improved Scores 100% 80% Independent Study: Culinary Arts Independent Study with Guidance: New Visions Lab: Criminal Justice 60% 40% 20% 0% Reading Sentence Skills Arithmetic Elementary Algebra
Enhancing College Readiness Montgomery County Public Schools (MD) Audience: 11 th graders Time: 36 weeks Measure: Increase in placement test scores and course placement Results: Improved placement scores for students using AMFL vs. other resources 80% 70% 60% 50% 40% 30% 20% 10% 0% RC SS EA CLM ACCU//MFL Other
Enhancing College Readiness Delaware DOE Audience: 11 th -12 th graders in high schools across the state Time: 10 weeks Measure: Increase in Diagnostic domain scores Results: Improved domain scores even with a small amount of time devoted to intervention Available Topics Average Number of Topics Assigned Average Number of Topics Completed Average Time Per Student Math 77 61 15.0 4:07 Reading/Writing 40 26 1.0 2:30 All 117 90 1.5 4:54
Enhancing College Readiness Delaware DOE
Developing and Planning Interventions
Developing an Intervention Defining Goals Defining Instructional Strategies Roles and Responsibilities Technical Considerations Educator Professional Development Timeline
Defining Goals Identifying students to be served Facing placement testing Not meeting PAT/NMSQT or SAT college-readiness benchmarks Academic difficulty grades or state test scores Not planning on college for now Younger students planning for dual enrollment Measures of success Pre- to post-testing with placement o Score gains o Placement gains Pre- to post-testing with Diagnostics Success in subsequent courses
Defining Instructional Strategies Independent Study Independent Study with Guidance Computer Lab Blended Learning Existing Dev Ed or high school coursework High school college readiness course Summer Boot Camp After school or Saturday programs Independent study
Roles and Responsibilities ACCUPLACER account management Testing Registering students in MyFoundationsLab Instruction Re-testing Classroom teachers / instructors Counselors / advisors Testing center personnel Administration / leadership Students / parents
Technical Considerations Testing Sufficient hardware access Testing time varies System Requirements check Intervention Sufficient hardware access in computer lab, classroom, and/or home System requirements check
Professional Knowledge ACCUPLACER Account setup Testing Reporting MyFoundationsLab Activating student accounts Student experience Instructor Grade Book Data analysis and instructional decisions
Timelines Account setup Activation of MyFoundationsLab Professional Development/Training Testing Time ACCUPLACER untimed Implementation design may require additional testing such as Placement > Diagnostics > Placement MyFoundationsLab Topics require 1 1 ½ hours each Math: 77 topics / Reading: 20 topics / Writing: 20 topics
Resources
Questions? ACCUPLACER Resources Page accuplacer.collegeboard.org/professionals/accuplacer-resources Keith Henry Sr. Assessment Manager khenry@collegeboard.org