Chaffey College Performance Outcome Data Online to College (OTC) vs. non-otc Students Fall 2008 and Fall 2009 Cohorts

Transcription

1 February 2011 Chaffey College Performance Outcome Data Online to College () vs. non- Students Fall 2008 and Fall 2009 Cohorts Overview: In support of the Online to College () Program, the Chaffey College Office of Institutional Research examined 2008 and 2009 Montclair High School 12 th graders who: a) earned a high school diploma or equivalent (i.e., GED or certificate of California high school proficiency); and b) subsequently enrolled at Chaffey College in the fall semester immediately following high school graduation. In examining Chaffey College performance outcome data for these two cohorts, the Office of Institutional Research further disaggregated data and compared performance outcome differences between Online to College () and non-online to College students. Methodology: As part of the larger scope of the Online to College () Program, the Office of Institutional Research identifies 5 th grade students who enroll at Kingsley, Lehigh, and Monte Vista Elementary Schools and tracks these students through middle school (Serrano and Vernon Middle Schools) and high school (Montclair High School). This tracking system serves as the primary vehicle that is used to identify students who attend Montclair High School. To maintain Program eligibility, a student must be continuously enrolled in 5 th through 12 th grades at one of the three elementary schools, two middle schools, and Montclair High School. As prospective students apply to Chaffey College a number of data elements are captured, including year of high school graduation, student education status, and County-District-School (CDS) code of institution last attended. Chaffey College applicants who: a) last attended Montclair High School; b) received a high school diploma or equivalent (GED or certificate of California high school proficiency); and c) graduated in one of the two cohort years (2008 or 2009) were identified for possible inclusion in the current study. Specific to Online to College students who apply to Chaffey College, the code is entered into the STA.OTHER.COHORT.GROUPS field in the STUDENT.ACAD.LEVLES file in Colleague (Chaffey Colleges administrative computing system) to identify students who have met the Online to College Program criteria. Extracting all applicants with a code equal to, and Online to College Program applicants were identified. A number of applications generated by both Online to College and non-online to College students are submitted to Chaffey College through CCCApply, the California Community College System application process. Since CCCApply allows prospective students to forward their applications to multiple community colleges, applicants in the 2008 and 2009 cohort groups were processed through the district s enrollment history database to determine whether applicants actually enrolled in courses at Chaffey College. The Office of Institutional Research maintains a database of all unitary enrollments generated by students since the Summer

2 semester, a total of 4,351,475 individual enrollment records. Selecting out 2008 and 2009 cohort group students who generated one or more enrollments that were recorded in the enrollment history database, a total of 270 students were identified (126 in 2008; 144 in 2009), including 83 students (37 in 2008; 46 in 2009). Controlling for enrollments in cancelled sections and student withdrawals that occurred prior to the start date of the semester, 2,039 unique enrollments were identified (1,014 enrollments by students in the 2008 cohort group; 1,025 enrollments by students in the 2009 cohort group). A number of operational definitions have been developed by the California Community College Research & Planning (RP) Group and adopted by the California Community College Chancellor s Office to examine performance outcomes. As it relates to the current study, the following operational definitions have been employed: Successful grades earned on record: Defined as any A, B, C, or P (pass) grade earned on record Retained grades earned on record: Defined as any A, B, C, P, D, F, FW, NP (no pass), or I, grades earned on record. Grade earned on record: Defined as any A, B, C, P, D, F, FW, NP, I, or W grades earned on record. Success rate: Defined as the number of A, B, C, and P grades earned on record, divided by the number of A, B, C, P, D, F, FW, NP, I, and W grades earned on record. Retention rate: Defined as the number of A, B, C, P, D, F, FW, NP, and I grades earned on record, divided by the number of A, B, C, P, D, F, FW, NP, I, and W grades earned on record. Term-to-Term : Defined as the number of students who earn a grade notation ( A, B, C, P, D, F, FW, NP, I, or W grades earned on record) in at least one course in the first term (either Fall 2008 or Fall 2009 in the current study) who subsequently earn a grade notation in at least one course in the next term. First term enrollment serves as the denominator; subsequent term enrollment serves as the numerator. Thus, persistence rate is operationally defined as the numerator divided by the denominator, multiplied by 100. In the current study, two measures of next term were examined: o Fall-to-Spring semester persistence rate (i.e., enrollment in the subsequent term within the same academic year) o Fall-to-Fall semester persistence rate (i.e., enrollment in the subsequent academic year) As the aforementioned operational definitions illustrate, enrollments that result in withdrawal prior to earning a grade on record are not included in the identification of successful or retained grades, the calculation of success and retention rates, or the calculation of Fall-to-Spring or fallto-fall persistence rates. In recognizing the importance of examining enrollment behavior that might not result in a grade earned on record, the Office of Institutional Research also examined status codes and status dates. Unique to Colleague, the following codes are mapped to each Chaffey College enrollment: Status code of N. This code reflects enrollments that occurred prior to the start date of the section. Students who generated this enrollment status code were retained in the section (i.e., completed the semester) and earned a grade on record in the section other than W (withdraw). Status code of A. This code reflects enrollments that occurred after the start date of the section. Students who generated this enrollment status code were retained in the 2

3 section (i.e., completed the semester) and earned a grade on record in the section other than W (withdraw). Status code of D. This code reflects enrollments that resulted in student withdrawals after the start date of the section. Examining enrollments that generated a D status code, two unique patterns exist: o o Students who withdraw prior to earning a grade on record: While a local status code of D is generated for this population, comparing status code date to last day to withdraw without earning a grade on record (usually the Friday of the fourth week in weekly census procedure courses) allows researchers the opportunity to identify enrollments that do not result in a grade earned on record. Students who earn a W (Withdrawal) grade: The W grade reflects students who withdrew from a section after the last day to withdraw without a earning a grade on record but prior to the 75 th percentile of the course (i.e., on or before the last day to withdraw from a section). In examining a combination of Chancellor s Office adopted operational definitions and status codes and dates associated with enrollments that fall outside of adopted performance outcome measures, a complete picture of Online to College and non-online to College student performance outcomes begins to emerge. Findings: The table on the following page identifies enrollment and performance outcome data for the 2008 and 2009 cohorts, comparing Online to College and non-online to College students. First semester enrollments, grades earned on record, and success and retention rates are identified, as well as Fall-to-Spring semester persistence rates. As applicable, for students who persist from Fall-to-Spring, second semester performance outcome data (enrollments, grades on record, success and retention rates) is presented. Finally, Fall-to-Fall persistence rates are presented, along with corresponding performance outcomes in the Fall semester following initial Fall enrollment (i.e., enrollment one year later). 3

4 Table 1 ENROLLMENT AND PERFORMANCE OUTCOME DATA Online to College Students Compared to Non-Online to College Students 2008 and 2009 Cohorts Cohort Students Who Enrolled at Chaffey Following MHS Grad. Total Enrollments FIRST SEMESTER (First Fall Semester) Grades Earned on Record Success Retention Fall-to- Spring (Enroll in Fall/Enroll in Spring) Total Enrollments SECOND SEMESTER (Following Spring Semester) Grades Earned on Record Success Retention Fall-to- Fall (Enroll in Fall/Enroll in Fall) Total Enrollments THIRD SEMESTER (Fall Semester, Following Year) Grades Earned on Record Success Retention 2008 Cohort Non Cohort Non

5 Performance outcome differences (percentage of students who earned a grade on record, success rate, retention rate, and persistence rate) between Online to College and non-online to College Students were examined in a number of different ways: Aggregate data for the three-semester period, both cohort groups combined By semester (first fall, spring semester, and second fall semester), both cohort groups combined Aggregate data for the three-semester period, by cohort group (2008 and 2009 cohort groups examined separately) By semester (first fall, spring semester, and second fall semester), by cohort group (2008 and 2009 cohort groups examined separately) In examining data, minimal statistically significant differences were observed. All observed statistically significant differences were in two areas: the percentage of students who earned a grade on record and persistence rates: In examining aggregate data for the three-semester period, both cohort groups combined, a statistically significantly higher percentage of Online to College students earned a grade on record than non-online to College students (91.3% vs. 87.1%, respectively) In examining Spring semester data for both cohort groups combined, a statistically significantly higher percentage of Online to College students earned a grade on record than non-online to College students (92.0% vs. 87.4%, respectively) In examining aggregate data for the three-semester period for students in the 2008 cohort group, a statistically significantly higher percentage of Online to College students earned a grade on record than non-online to College students (93.5% vs. 87.3%, respectively) In examining Fall 2009 semester data for students in the 2008 cohort group, a statistically significantly higher percentage of Online to College students earned a grade on record than non-online to College students (92.7% vs. 84.0%, respectively) In examining both cohort groups combined, a statistically significantly higher percentage of Online to College students than non-online to College students persisted from the first fall semester to the spring semester (97.5% vs. 72.2%, respectively) In examining both cohort groups combined, a statistically significantly higher percentage of Online to College students than non-online to College students persisted from the first fall semester to the following fall semester (79.0% vs. 59.8%, respectively) In examining the 2009 cohort group, a statistically significantly higher percentage of Online to College students than non-online to College students persisted from the first fall semester to the spring semester (100.0% vs. 62.1%, respectively) In examining the 2009 cohort group, a statistically significantly higher percentage of Online to College students than non-online to College students persisted from the first fall semester to the following fall semester (79.5% vs. 48.3%, respectively) Recognizing that observed statistically significant differences might be an artifact of sample size (with larger samples, only minimal differences can sometimes produce statistically significant results; conversely with small samples large outcome differences may not be identified as statistically significantly different), the Office of Institutional Research also examined effect size differences between groups. In essence, effect size measures the strength of the relationship between two variables, controlling for the influence of sample size. While interpretations vary, the most commonly accepted interpretations suggest that an effect size of 0.20 indicates a small 5

6 effect, 0.50 a medium effect, and 0.80 or higher a large effect. Recognizing the difficulty in identifying a relationship between two variables in a quasi-experimental environment like postsecondary education, most educational research studies consider an effect size of 0.25 or higher to be significant. Table 1 identifies areas where the observed effect size difference between Online to College and non-online to College students was.25 or higher Table 1: Online to College vs. non-online to College Students: Effect Size Differences Observed Between Groups Data Online to College Non-Online to College Effect Examined Metric N Mean SD N Mean SD Size 2008 Cohort Group, Fall Grade Earned on Semester Record Combined Fall-Spring Cohort Groups Combined Fall-Fall Cohort Groups Cohort Fall-Spring Group 2009 Cohort Group Fall-Fall Summary: In examining performance outcome differences between 2008 and 2009 Online to College and non-online to College cohort groups, it appears that: Online to College students are statistically significantly more likely than non-online to College students to earn a grade on record, suggesting that non-online to College students are far more likely to withdraw early in the semester, prior to earning a grade on record. Since earning a grade notation factors into the formula to compute persistence rate, Online to College students are also statistically significantly more likely to persist across semesters, both from initial Fall enrollment to Spring semester and from Fall-to- Fall. No statistically significant differences were observed in between Online to College and non-online to College students in regard to course outcomes (i.e., success and retention rates), although Online to College students generated slightly higher success and retention rates than non-online to College students. 6