Current Range MATH-61. Current Range. Prior Range to to to to to 34.

Transcription

1 Sept 2010 Mathematics Placement Follow-Up Study Placement Recommendations After Changes to Branching Profile And Subsequent Mathematics Course Performance Outcomes Controlling for Students Placed at Higher Mathematics Course Levels In Order to Generate Educational Background Predictor Variables Overview: As a follow-up to a previous study, the Office of Institutional Research once again examined the relationship between placement recommendations generated in the year before (August 16 th, 2008 through August 15 th, 2009) and after (August 16 th, 2009 through August 15 th, 2010) changes were made to the mathematics test branching profile and subsequent mathematics course outcomes. However, in the current study the Office of Institutional Research controlled for students assessed after changes were made to the mathematics test branching profile who received higher course placement recommendations in order to collect data to validate educational background predictor variables. Methodology: In prior conversations with the Mathematics Department, the then Senior Research Analyst discussed allowing students who generated assessment test scores in proximity to faculty identified mathematics course cut-scores to receive placement recommendations for the next higher mathematics course. In allowing students who scored slightly below existing mathematics course cut-scores to access the next higher level course in the mathematics course sequence, the intent was to capture additional data to produce more accurate educational background predictor variables. As mandated by the California Code of Regulations Title 5, 55521(a), California Community Colleges must use multiple measures (other than two or more highly correlated instruments) to generate student placement recommendations. Capturing additional data was intended to strengthen this process, affording researchers with the opportunity to collect and analyze sufficient data to determine multiple measures that predicted optimal course success and non-success. Table 1 on the following page identifies faculty determined mathematics course placement cutscore ranges prior to and after changes to cut-score placement recommendations were implemented. In comparing previous and current cut-score ranges, table 2 identifies the specific cut score ranges by assessment test that were impacted, how these changes effected course placement recommendations, and the number and percentage of students within each score range that were affected. For the purposes of this follow-up study, the number and percentage of students who were affected by cut-score changes reflect only students who subsequently earned a grade on record in a mathematics course within a year of generating a mathematics assessment placement recommendation (see the initial study (Math Placement Study Sep10.doc) for details about the methodology and determination of placement recommendations and relationship to subsequent course-taking behavior). 1

2 Table 1: Faculty Determined Mathematics Cut-Score s to and After Changes to Cut-Score Placement s Faculty Determined Mathematics College Level Math Test Test Arithmetic Test Cut-Score s to to to to to to College Level Math to to to to to to Arithmetic Test to to to to to to Table 2: Cut-Cut Changes by Assessment Test - Changes to Mathematics Recommendation and Number of Effected Students Effected Cut- Placed Students Into... Student Effected Assessment Test Score Formerly ly N % to Take CLM to to Take Arith Test Arithmetic Test to Take EA n/a n/a Arithmetic Test to Arithmetic Test to TOTAL In examining data displayed in table 2, the 26 students who generated an (EA) assessment test score in the to range all subsequently took the College Level Math (CLM) test and generated course placement recommendations. Under the prior branching profile, these students would have received placement recommendations. The table similarly identifies students who scored in the to range on the EA test ( course placement recommendation; formerly would have received a placement recommendation); the to range on the EA test 2

3 ( course placement recommendation; formerly would have received a recommendation to take the Arithmetic test); the to range on the Arithmetic test ( course placement recommendation; formerly would have received a placement recommendation); and the to range on the Arithmetic test ( course placement recommendation; formerly would have received a placement recommendation). Students who scored in the to range on the Arithmetic test were not affected; in making changes to the test branching profile, no students were identified who started on the Arithmetic test (the previous study identified that 99.3% of students started with the test after changes were made to the mathematics test branching profile on August 16 th, 2009) and received an Arithmetic score that would have branched them to take the test. Findings: Controlling for the 141 enrollments that would have resulted in lower mathematic course placement recommendation if previous cut-score ranges had been applied, table 3 on the following page provides a detailed analysis of course success and retention rates by specific course placement recommendation, identifying statistically significant success and retention rate differences between groups at the course level. Even after controlling for the 141 enrollments that might have resulted in students being placed into higher mathematics courses, outcomes closely parallel the findings indentified in the initial study. Statistically significant differences were noted in the same areas identified in the initial study (e.g., retention rate; success rate; aggregate math success and retention rates). Table 4 compares findings from the initial study to the current study. 3

4 Table 3: Mathematics Course Performance Outcomes of Students Who Assessed Before and After Changes to the Branching Profile by Course Level Controlling for Students Placed at Higher Course Levels in Order to Generate Educational Background Predictor Variables Overall Math Course Course Performance Outcomes in: Assessment Group by Math Performance Outcomes Lower Math Courses Recommended Math Course Higher Math Courses Recommendation Level N Succ Ret N Succ Ret N Succ Ret N Succ Ret Assessed Before Changes * Assessed After Changes Assessed Before Changes Assessed After Changes Assessed Before Changes Assessed After Changes n/a n/a Assessed Before Changes * * 87.0* Assessed After Changes Assessed Before Changes Assessed After Changes Assessed Before Changes Assessed After Changes Assessed Before Changes Assessed After Changes Assessed Before Changes n/a n/a Assessed After Changes n/a n/a TOTAL Assessed Before Changes 1, * 88.7* * 88.0* * Assessed After Changes * Statistically significant differences observed (p.05) 4

5 Table 4: Mathematics Course Performance Outcomes of Students Who Assessed Before and After Changes to the Branching Profile by Course Level Comparison of Findings From Initial Study and Study That Controls for Potential Higher Course Placement Recommendations Assessment Group by Math Findings from Initial Study Findings From Study Recommendation Level N Succ Ret N Succ Ret Assessed Before Changes * * Assessed After Changes Assessed Before Changes Assessed After Changes Assessed Before Changes Assessed After Changes Assessed Before Changes * * 88.4 Assessed After Changes Assessed Before Changes Assessed After Changes Assessed Before Changes Assessed After Changes Assessed Before Changes Assessed After Changes Assessed Before Changes Assessed After Changes TOTAL Assessed Before Changes 1, * 88.7* 1, * 88.7* Assessed After Changes * Statistically significant differences observed (p.05). 5