Running Head: Promoting Student Success: Evaluation of a Freshman Orientation Course

Transcription

1 Running Head: Promoting Student Success: Evaluation of a Freshman Orientation Course Promoting Student Success: Evaluation of a Freshman Orientation Course Mary A. Millikin, PhD Abstract Many first-time college students arrive on campus unprepared to succeed in college. According to a survey of degree-granting institutions by the National Center for Education Statistics, 20 percent of entering students at public four-year institutions and 42 percent of entering first-time students at two-year colleges require at least one remedial course. This study investigated the effectiveness of a student success course designed to address the readiness gap for first-time freshmen at a public two-year college. Formative and summative analyses were conducted using two first-time freshman cohorts for the experimental and control groups. Results indicate higher persistence and graduation rates for the experimental group (alpha <.01) as a result of enrollment in the student success course. Additionally, the experimental group completed seven of 15 remedial and gateway courses with significantly higher course grades than did the control group. These results suggest that a student success course designed to address specific barriers common to first-time freshmen can be highly beneficial. Implications for academic policy are considered.

2 Promoting Student Success: Evaluation of a Freshman Orientation Course 2 Many first-time college students arrive on campus unprepared to succeed in college. According to a survey of degree-granting institutions by the National Center for Education Statistics (2008), 20 percent of entering students at public four-year institutions and 42 percent of entering first-time students at two-year colleges require at least one remedial course. Underpreparation is often viewed as deficiencies in basic academic skills, specifically in mathematics, reading, and writing (Zeidenberg, Jenkins & Calgagno, 2007). However, research indicates that entering students often lack critical non-academic skills for college success, including adequate study skills, college- and career goal-setting skills, and the ability to adjust to college life in a timely fashion. These non-academic factors may affect student persistence at least as much as academic preparedness (Gonzalez, Woodruff, & Millikin, 2009; Millikin & Woodruff, 2010). Historically, the most commonly used conceptual models of college persistence were based on Tinto s Student Integration Model (1993) and Bean s Student Attrition Model (1985). These frameworks have been the impetus for a plethora of research and conceptual development, and they suggest that college students learn best when faculty and administrators foster both academic and social engagement. Orientation courses have been one means by which universities have promoted student success; however, until recently two year colleges were slow to adopt this policy. Through national reform movements for student success in higher education, such as Achieving the Dream, Inc. (2011), many community colleges have adopted the practice of mandatory orientation and student success courses for entering freshmen. These courses are designed to teach students how to write notes, take tests, and manage their time. Additionally, special attention is given to the exploration of student learning styles and the developmental of plans for college goals and career goals.

3 Promoting Student Success: Evaluation of a Freshman Orientation Course 3 Despite the growing requirement of student success courses for entering freshmen, until recently little research has been conducted as to their effectiveness. Over the last several years, efficacy research in this area has been energized through collaboration between associate degreegranting institutions and the Achieving the Dream initiative (ATD, 2011; Lincoln, 2010). The purpose of this paper is to describe the evaluation of the effectiveness of a student success course for entering freshmen at a community college. Statistical models were used to determine if positive outcomes were evident even after controlling for entering student preparedness. Data and Methodology To examine the effects of a student success course on entering freshmen persistence and gateway course success, data were used from the institutional database at a large Oklahoma community college. Subjects were first-time freshmen during fall 2008 and fall 2009 and were tracked through fall The experimental group consisted of 2,995 first-time freshmen who completed the student success course within their freshman year. The vast majority (93%) of the experimental group were mandated to complete the course as part of a local tuition waiver program. The control group consisted of 3,960 first-time freshmen who did not enroll in the student success course during this same period. Table 1 displays the first-time freshman cohort size by entering student class. Table 1: Fall 2008 and 2009 First-time Freshman Cohort Cohort TOTAL in Cohort (Live Data System) Fall 2008 First-time Freshmen Cohort 3,655 Fall 2009 First-time Freshmen Cohort 3,840

4 Promoting Student Success: Evaluation of a Freshman Orientation Course 4 The student success course, titled, was developed through a collaborative effort of faculty from four campuses of the institution. Curriculum was developed based upon a comprehensive review of the literature, results from 12 student focus groups and seven faculty and staff focus groups, and robust experience from seasoned faculty. A formative assessment of the course was conducted using the Learning and Study Strategies Inventory (LASSI), a 10-scale, 80-item measure designed to assess students' awareness about and use of learning and study strategies related to skill, will and self-regulation components of strategic learning. The LASSI was implemented as a pretest and posttest to measure study skills and affective development, and a locally developed cognitive exam was used to evaluate cognitive gain. Summative assessment was conducted with student success as the dependent variable and enrollment/non-enrollment in as the independent variable. Student success was operationally defined as: [1] fall-to-spring retention; [2] fall-to-fall retention; [3] two year retention or graduation with an associate s degree or certificate; [4] C or better in eight gateway courses; [5] C or better in seven remedial courses; and [6] overall GPA. The hypotheses tested were as follows: Ho: M 1 < M 2; There is no increase in student success as a result of enrollment in Ha: M 1 > M 2; There is an increase in student success as a result of enrollment in where: M 1 = mean student success score for students who enrolled in and M 2 = mean student success score for students who did not enroll in

5 Promoting Student Success: Evaluation of a Freshman Orientation Course 5 Results Formative Assessment To measure affective development of study skills within, a dependent t test was conducted using LASSI pretest and posttest scores. Statistically significant increases in mean scores resulted for all ten scales of the instrument. Greatest gain resulted in Information Processing, Self-Testing, Selecting Main Ideas, and Test Strategies. Student scores on the Test Strategies scale resulted in the second highest mean posttest score, and the Information Processing scale resulted in the highest mean posttest score. Although statistically significant differences in the positive direction were realized for all scales, student scores on the Motivation Scale showed the smallest increase from pre- to posttest, with the Attitude scale resulting in the lowest mean posttest score. Table 2: LASSI Scale Pretest and Posttest Scores for LASSI Scales Mean Std. Deviation t-value Sig. Anxiety Pretest Posttest Attitude Pretest Posttest Concentration Pretest Posttest Information Pretest Processing Posttest Motivation Pretest Posttest Self-Testing Pretest Posttest Selecting Pretest Main Ideas Posttest Study Aids Pretest

6 Promoting Student Success: Evaluation of a Freshman Orientation Course 6 LASSI Scales Mean Std. Deviation Posttest Time Pretest Management Posttest Test Pretest Strategies Posttest Critical value of t = t-value Sig A cognitive assessment of critical thinking skills, developed by faculty experts, was administered during the final week of the course. Student scores ranged from 0 to 3, with a mean score of 2.37 and just over half of students earning the highest score of 3. When scores for the cognitive measure were correlated with LASSI posttest scores, the Concentration scale (r =.91) and Motivation scale (r =.89) were found to be positively correlated (alpha <.01). Summative Assessment A summative assessment was conducted to determine if enrollment in affected student success as measured by [1] fall-to-spring retention; [2] fall-to-fall retention; [3] two year retention or graduation with an associate s degree or certificate; [4] C or better in 15 remedial and gateway courses; and [5] cumulative GPA. Chi-squared tests were conducted to test for differences in fall-to-spring retention, fall-to-fall retention, and two year retention or graduation. Analysis of Variance (ANOVA) tests were conducted to compare differences in subsequent course grades, and an independent t test was used to determine if cumulative GPA increased as a result of enrollment in. Results showed that students who enrolled in persisted from fall to spring at a significantly higher rate than did first-time freshmen who did not enroll in the course.

7 Promoting Student Success: Evaluation of a Freshman Orientation Course 7 Statistical significance also resulted for fall to second fall persistence, and for fall to third fall persistence and/or graduation. Table 3 presents the results of these analyses. Table 3: Persistence and Graduation as a Function of Enrollment in First-time Freshmen Cohort (2008 and 2009 Combined) Fall to First Spring Students Who Enrolled in SAS* Students Who Enrolled in Neither SAS or College Survival X 2 Sig. Fall 2008 First-time Freshmen 85% 59%.001 Fall 2009 First-time Freshmen 89% 61%.001 Fall to Second Fall Fall 2008 First-time Freshmen 61% 41%.001 Fall 2009 First-time Freshmen 57% 37%.001 Fall to Third Fall (includes graduation in persistence) Fall 2008 First-time Freshmen 45% 31%.001 Course grades for seven developmental courses and eight gateway courses with highest enrollment were evaluated with respect to enrollment. ANOVA tests were conducted using course GPA (0 to 4.0) for all 15 courses as a dependent variable and enrollment/non-enrollment in as the independent variable. Results indicate that students were significantly more likely to succeed with a C or better than were students who did not enroll in for seven gateway and developmental courses. Grade distribution for is presented in Table 4, and grade distributions for all other courses are presented in Tables 5-11.

8 Promoting Student Success: Evaluation of a Freshman Orientation Course 8 Table 4: Course Outcomes for Strategies for Academic Success, ENGL 1003 Strategies for Academic Success Students Who Enrolled in Grade Distribution Number Percent A % B % C % C or Better 1, % D % F % AW, W, or N % I 8 0.4% Total 2, % Note: Columns may not add up to 100% due to rounding. Table 5: Course Outcomes for Basic Math, MATH 0003 Basic Math Students Who Enrolled in Students Who Did Not Enroll in Grade Distribution Number Percent Number Percent DA % % DB % % DC % % C or Better % % DD % % DF % % AW, W, or N % % I 0 0.0% 3 0.4% Total % % *Significant at the 99% confidence level. (ANOVA: DV = course GPA) Note: Columns may not add up to 100% due to rounding.

9 Promoting Student Success: Evaluation of a Freshman Orientation Course 9 Table 6: Course Outcomes for Reading II, ENGL 0913 Reading II Students Who Enrolled in Students Who Did Not Enroll in Grade Distribution Number Percent Number Percent DA % % DB % % DC % % C or Better % % DD % % DF % % AW, W, or N % % I 0 0.0% 1 0.2% Total % % *Significant at the 99% confidence level. (ANOVA: DV = course GPA) Note: Columns may not add up to 100% due to rounding. Table 7: Course Outcomes for Writing I, ENLG 0923 Writing I Students Who Enrolled in Students Who Did Not Enroll in Academic Strategies Grade Distribution Number Percent Number Percent DA % % DB % % DC % % C or Better % % DD % % DF % % AW, W, or N % % I 0 0.0% 0 0.0% Total % % *Significant at the 99% confidence level. (ANOVA: DV = course GPA) Note: Columns may not add up to 100% due to rounding.

10 Promoting Student Success: Evaluation of a Freshman Orientation Course 10 Table 8: Course Outcomes for Freshman Composition I, ENGL 1113 Fresh Comp I Students Who Enrolled in Students Who Did Not Enroll in Academic Strategies Grade Distribution Number Percent Number Percent A % % B % % C % % C or Better % % D % % F % % AW, W, or N % % I 1 0.1% 2 0.1% Total % % *Significant at the 99% confidence level. (ANOVA: DV = course GPA) Note: Columns may not add up to 100% due to rounding. Table 9: Course Outcomes for Biology for Non-majors, BIOL 1114 Biology for Nonmajors Students Who Enrolled in Students Who Did Not Enroll in Grade Distribution Number Percent Number Percent A % % B % % C % % C or Better % % D % % F % % AW, W, or N % % I 1 0.3% 1 0.4% Total % % *Significant at the 99% confidence level. (ANOVA: DV = course GPA) Note: Columns may not add up to 100% due to rounding.

11 Promoting Student Success: Evaluation of a Freshman Orientation Course 11 Table 10: Course Outcomes for Introduction to Psychology, PSYC 1113 Intro to Psychology Students Who Enrolled in Students Who Did Not Enroll in Grade Distribution Number Percent Number Percent A % % B % % C % % C or Better % % D % % F % % AW, W, or N % % I 1 0.1% 1 0.1% Total % % *Significant at the 99% confidence level. (ANOVA: DV = course GPA) Note: Columns may not add up to 100% due to rounding. Table 11: Summary of Course Grade Comparison Course Students Who Enrolled in Students Who Did Not Enroll in Basic Math Significant Difference Significant Difference Beginning Algebra Intermediate Algebra Reading I Significant Difference* Significant Difference* Reading II Significant Difference Significant Difference Writing I Significant Difference Significant Difference Writing II Freshman Comp I Significant Difference Significant Difference College Algebra Biology for Non-majors Significant Difference Significant Difference Biology for Majors American Federal Government US History 1492 to the Civil War Era US History Civil War Era to the Present Introduction to Significant Difference Significant Difference Psychology *Significant difference for developmental version of. Reading I students are ineligible to enroll in until completing College Survival.

12 Promoting Student Success: Evaluation of a Freshman Orientation Course 12 An independent t test was conducted to determine if cumulative GPA increased as a result of enrollment in. No significant difference in overall GPA was found for these two groups. Table 12 presents the results. Table 12: Comparison of Cumulative GPA Cum GPA By Second Fall Students Who Enrolled in Mean N Std Dev Students Who did Not Enroll in Mean N Std Dev No significant difference at the 95% confidence level. (T test; alpha <.01) Conclusions Formative assessment revealed that student scores for all ten LASSI scales increased significantly at the 99% confidence level from pretest to posttest. Greatest gain resulted in Information Processing, Self-Testing, Selecting Main Ideas, and Test Strategies. These results hold true for both first-time freshman cohorts and suggest that study strategies are being effectively taught in, with motivation for learning and attitude about learning having a more restricted range for improvement. In evaluating summative results, seven of 15 developmental and gateway courses with the highest enrollment resulted in statistically significant increases in course success (C or better) for students who enrolled in. Further, students who enrolled in Academic Strategies were more likely to persist from fall to spring, fall to fall, and through two full years than were other groups. However, cumulative GPA was not significantly different within this time period.

13 Promoting Student Success: Evaluation of a Freshman Orientation Course 13 These results suggest that freshmen orientation courses can be effective in increasing student persistence within the first two years. Specifically, enrollment in such a course can affect success in critical remediation courses, including developmental reading, developmental writing, and basic math. Orientation courses can also affect essential college gateway course success, including Freshman Composition, Biology for Non-majors, and Introduction to Psychology. This supports the claim that both academic and non-academic skills are strengthened as a result of enrollment in a well-developed student success orientation course. Community colleges may do well to follow the lead of universities in requiring such courses for entering freshmen to promote student success.

14 Promoting Student Success: Evaluation of a Freshman Orientation Course 14 References Achieving the Dream. (2011). Promising Practices: 2011 Leader Colleges. Bailey, T., Calcagno, C., Jenkins, D., Kienzl, G., & Leinbach, T., (2005). Community College Student Success: What Institutional Characteristics Make a Difference? Community College Research Center, Teachers College, Columbia University. Bean, J. P. (1985). Interaction effects based on class level in an explanatory model of college student dropout syndrome. American Educational Research Journal, 22(1), Gonzalez, K., Woodruff, J. and Millikin, M. (2009). Focus Groups: A Powerful Tool to Increase Student Success. Presented at the Achieving the Dream Strategies Institute. San Francisco, CA. Lincoln, C. (2010). Institutional change for Student Success. Millikin, M. & Woodruff, J. (2010). Focus on diagnosis: Persistence at Tulsa Community College. Presented at Achieving the Dream Strategies Institute. Charlotte, NC. National Center for Education. (2008). U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics. Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition. Chicago: University of Chicago Press. Zeidenberg, M., Jenkin, D., & Calgagno, J.C. (2007). Do student success courses actually help community college students succeed? Community College Research Center Brief (36), 1-6.