High-Impact Educational Practices as Promoting Student Retention and Success

203 High-Impact Educational Practices as Promoting Student Retention and Success Sunny Moon Institutional Research and Analytical Studies California State University, Fullerton hmoon@fullerton.edu Sean Walker College of Natural Sciences and Mathematics California State University, Fullerton swalker@fullerton.edu Cathy Fernandez College of Natural Sciences and Mathematics California State University, Fullerton cfernandez-weston@fullerton.edu Edward Sullivan Institutional Research and Analytical Studies California State University, Fullerton esullivan@fullerton.edu Martin Bosangue College of Natural Sciences and Mathematics California State University, Fullerton mbonsangue@fullerton.edu Raman Unnikrishnan College of Engineering and Computer Science California State University, Fullerton runnikrishnan@fullerton.edu James Hershey Institutional Research and Analytical Studies California State University, Fullerton jrhershey@fullerton.edu Mark Filowitz College of Natural Sciences and Mathematics California State University, Fullerton mfilowitz@fullerton.edu Victor Delgado College of Engineering and Computer Science California State University, Fullerton vdelgado@fullerton.edu Abstract - Student retention and timely graduation are important topics under discussion on a campus-, system-, and nation-wide basis. California State University, Fullerton, focused on enhancing student success to improve student persistence and graduation rates and to narrow achievement gaps for underrepresented students as an institutional priority. The present study investigated the effectiveness of high-impact educational practices on student learning outcomes at the program and course levels. Supplemental Instruction is a peer-facilitated review session that takes place in a comfortable collaborative learning setting in which students can openly discuss lecture materials with an instructional leader and with their peers. The instruction has been implemented in bottleneck science courses and has raised course success and course grades, thereby narrowing the achievement gap. The Freshman Program provides a community of students, faculty, and professional staff to assist first-year students to make a smooth transition from high school into college and to make the most of the college experience. They create their own small academic and social community environment, within the larger university setting. Analysis of longitudinal data indicated the positive effects of the Freshman Program on achievement of underrepresented students. Introduction Retaining students and advancing them toward successful graduation is a central mission of higher education institutions. To achieve this mission, it is vital to identify and understand factors related to student retention and success. Tinto (2002) emphasized that the higher education institution is ultimately responsible for student retention. Failure to provide an effective institutional environment that fosters student engagement in college activities and to comprehensively understand student characteristics and cultural background, may bring about high attrition rates, and works against the fundamental role of higher education. For the mission of higher education institutions to be accomplished, we need to understand the relationship between student characteristics, particularly cultural background, student engagement, and learning outcomes. Researchers at all levels of higher education institutions have been investigating factors related to college student retention and timely graduation (Astin, 1985, 1993; Astin & Dey, 2001; Bean & Easton,

204 2002; Moon & Hagedorn, 2006; Pascarella & Terenzini, 2005; Tinto, 1993, 2002, 2007). Student success is one of the major topics being discussed, nation-wide, system-wide, and campus-wide as college education is a key to our country s economic advancement in the world. As such, President Obama (2010) has made increasing the number of college graduates among young adults from 40% to 60% in next 10 years a national priority. Restoring America to first in the world in college degree attainment and improving America s STEM (Science, Technology, Engineering, and Mathematics) education have been adopted by White House as important strategies for American innovation (White House, 2010). The United States is a nation of diversity, and there is more ethnic diversity in the state of California than in any other state. According to the US Census Report (2011), 50% of California s population is comprised of minority individuals. In particular, Hispanics contributed to over 90% of California s population growth between 2000 and 2010 and to approximately 50% of the total US population growth during the same period (US Census, 2013a; Appendix). However, only 15% of Hispanics aged 25 years and over have completed a bachelor s degree or higher, whereas 31% of individuals from all races in the same age range have earned a bachelor s degree (US Census, 2013b; Appendix). Further, as of 2011, the average income of Hispanics 18 years or older is lowest among all the races (US Census, 2011). The CSU (California State University System) Graduation Initiative (http://graduate.csuprojects.org/) is a prime example of a nation-wide effort to increase system-wide graduation rates and to narrow the achievement gap between underrepresented and non-underrepresented students. With an enrollment of over 400,000 students across 23 campuses, CSU is one of the largest and most diverse public university systems in the nation. The goals of the system-wide Graduation Initiative are to improve overall graduation rates by eight percent and cut the graduation gap for underrepresented students in half. California State University, Fullerton, is a large, four-year public institution with an enrollment of nearly 38,000 students, of whom close to one-third are Hispanic. As one of the most diverse campuses in the state, as well as in the country, California State University, Fullerton, has been serving a substantial number of Hispanic students as an HSI (Hispanic Serving Institution) in recent decades. Hispanic enrollment has increased from 7% in 1980 to 33% in 2012 (Moon, 2012). The growth of Hispanic students is seen in undergraduates, graduates, first-time freshmen, and transfer enrollments. Annually, California State University, Fullerton, is recognized as being among the top institutions for bachelor s degrees awarded to Hispanic students (Hispanic Outlook in Higher Education, 2009, 2010, 2011, 2012, 2013). More than 40% of the incoming freshmen are underrepresented students, including Hispanics, Blacks, and Native Americans. First-generation students (neither parents attained a bachelor s degree) comprise 55% of the freshmen cohort. However, the achievement gap in six-year graduation rates for underrepresented students at California State University, Fullerton, has been 10% in recent years (http://www.fullerton.edu/analyticalstudies/). Thus, this institution has recognized a need to focus efforts on underrepresented student success and providing intervention strategies to close the achievement gap. Joining in the CSU Graduation Initiative which is a system-wide effort, Cal State Fullerton identified enhancing student success to improve student persistence and graduation rates as one of the top priorities of the institution moving forward. As a core goal of the institutional strategic plan, Improve student persistence, increase graduation rates University-wide, and narrow the achievement gap for underrepresented students has been adopted aiming to become a model comprehensive university (http://planning.fullerton.edu/planning/index.asp). The implementation of this goal is established on educational practices which promote active student engagement, so called, High Impact Practices (HIPS). The present study seeks to evaluate the effectiveness of those pedagogical approaches and interventions that our institution has been practicing at the course level as well as the program level. Encompassing analysis and assessment of our educational practices will advance our deeper understanding of student cultural characteristics, student engagement in academic and supportive programs, and its relationship with learning outcomes including grade, retention, and graduation. Accountability of effective educational practices will be imperative for achieving our priorities. Data-

205 driven, evidence-informed practices based on the accountability are convincing campus constituencies and it is in their interest to enhance student success. Retention Theories and College Impact Models One of the most significant means by which students, faculty, and stakeholders evaluate a college is student persistence/retention. The terms retention and persistence are often used interchangeably in the higher education context (Hagedorn, 2004). NCES IPEDS (2013) defines retention rate as a measure of the rate at which students persist in their educational program at an institution, expressed as a percentage. For four-year institutions, this is the percentage of first-time degree-seeking undergraduates from the previous fall who are again enrolled in the current fall (NCES IPEDS, Glossary of Retention Rates). Retention is usually defined as an institutional measure, whereas persistence is generally a studentoriented measure. Therefore, students persist at a college, while the college retains those students. Astin s I-E-O Model and Theory of Involvement (1985, 1993, p365) describes college outcomes as a function of three components including Input-Environment-Outcomes. Students enter college with certain pre-collegiate inputs such as demographics and previous experiences. Student involvement in institutional environment along with their pre-collegiate factors shapes environmental experiences and learning outcomes such as retention and successful graduation. Involvement with scholastic activities, involvement with faculty, and involvement with peers are all related to positive student development in higher education. Assessment of the college impact on students consists of a tri-fold process including understanding student change as a result of college education, elaborating the conceptual model for student learning outcomes, and analyzing college impact on student learning (Astin, 1993). Academic and social integration have been postulated as primary factors to increase student retention in Vincent Tinto s Model of Institutional Departure (1987, 1993). Tinto s integration model posits that student persistence is related to college experiences through interfacing between student commitment and institutional environment. Therefore, a good match that encourages students to be academically and socially integrated in college campuses is more likely to retain students and to advance them to successful graduation. In other words, marginalization and mismatching an individual from academic and social environment on campus can be critical factors leading to student withdrawal. Based on Tinto s theory, Bean and Easton (2002) advanced a Psychological Model of Retention that explicates how social and academic integration develop. The institutional environment assists the student to develop certain psychological attributes, such as a positive attitude toward the school and toward academic work that can steer students to be successful in their college life. As a precursor to academic and social integration into college, students should believe that they are effective in learning and are accountable for their own learning outcomes. The theory presents the contribution of student efforts and the student-institution interaction to retention. In this regard, through a variety of retention programs, including learning communities, service learning, freshman seminars, and mentoring programs, students are able to interact positively with the institution, which facilitates their psychological growth. Five components in Pascarella s (1985) Causal Model of Assessing College Student Change consists of institutional characteristics, student pre-collegiate factors, interaction with agents of socialization in college, institutional environment, and student efforts. In his theory, learning and cognitive outcomes are instigated as direct and indirect effects of institutional characteristics and environment. Affective learning outcomes including attitudes and values, psychosocial changes, and moral development have been added as college learning outcomes in his later research (Pascarella & Terenzini, 2005). Culturally-Responsive Practices Many popular models and theories are based on the roles of traditional college students, at an individual level. Those individualistic models largely failed to embrace cultural characteristics of underrepresented college students and did not focus on a student s position as a member of specific cultural group. The individualistic nature of student departure in Tinto s model was criticized by

206 researchers (Tierney, 1992; Guiffrida, 2006; Rendon, Jalomo, & Nora, 2000). Without understanding cultural components, we are not able to apply theoretical models to educational practices in higher education institution enrolling students from diverse background. Most underrepresented students come from collectivistic cultures in which people view themselves as members of groups. The main characteristics of collectivistic cultures focus on valuing the needs of family or group as opposed to those of individuals (LeMonda, 2008). Their cultures tend to emphasize noncompetitive, cooperative living, with a strong sense of community and extended family. This cultural consideration may be a key factor in retention for underrepresented students, most notably Hispanics and Blacks. Therefore, increased sensitivity to cultural differences in college students is pre-requisite to developing institutional practices to promote student persistence and successful degree attainment. Meaningful connection and consistency between college experiences and student home culture will strengthen retention theories applied to institutional practices, which finally steer toward successful role of higher education. In an early inquiry on multicultural classroom, researchers introduced collaborative learning as an effective educational practice since it promotes student class achievement, prepares students to deal with the real world of individual differences, as well as improves the social climate in the classrooms (Slavin, 1990, 1991a, 1991b; Zanger, 1990). Students study together within culturally diverse groups becoming more accustomed to working toward a common goal. Those who are exposed to the collaborative learning environment are more likely to create friendly structures external to their own cultural background than those exposed only to traditional classrooms (Slavin, 1990, 1991b). In a research on campus pluralism and unity, educational researchers discussed collaborative learning as an effective teaching and learning method with dual goals (1) enhancing inter-ethnic relations and (2) increasing achievement gains (Hagedorn, Moon, Buchanan, Shockman, Jackson, 2001). Research consistently indicates that underrepresented students are collaboratively oriented in terms of their learning preferences (Avellar & Kagan, 1976; Cabrera et al., 2002; Johnson & Johnson, 1987; Kagan & Madison, 1971). When engaging in collaborative learning, underrepresented students report more satisfaction with group work and perceive greater class cohesion, fewer conflicts, and more peer academic support than do white students. Research also demonstrated that collaborative learning strategies tend to increase learning outcomes of all students, regardless of race or gender (Cabrera et al., 2002; Tinto, 1997), although to a greater extent for underrepresented students (Slavin & Oickle, 1981). College students who engage in collaborative learning tend to be more open to diversity than those who do not (Cabrera et al., 2002). Those who exhibit more openness to diversity are likely to have better achievement in college than those who do not (Bowman, 2013). From a cognitive perspective, academic performance may be enhanced when students work in collaboration with more-capable peers. This aspect of collaborative learning supports Vygotsky s notion of the zone of proximal development (1978). Interestingly, studies of high-achievers in collaborative learning environments indicated that they tended to perform better than in traditional classrooms (Slavin, 1991a). Thus, while collaborative learning may provide low-achieving students with special benefits, it is not done at the sacrifice of the high achievers. Student Engagement and High Impact Practices at California State University, Fullerton Engagement is the term usually used to represent constructs such as quality of effort and involvement in productive learning activities (Kuh, 2009a, p. 6). Student engagement and HIPS can be understood through retention theories and their constructs, as described in the previous section. HIPS are defined as educationally purposeful and effective practices that deepen student engagement and learning that lead to college student success. In research on High-Impact Educational Practices to promote student learning, Kuh (2008) identified ten teaching and learning activities, including: (a) first-year seminars and experiences; (b) common intellectual experiences; (c) learning communities; (d) writingintensive courses; (e) collaborative assignments and projects; (f) undergraduate research; (g) diversity/global learning; (h) service learning, community-based learning; (i) internships; and (j) capstone courses and projects. These activities foster student academic and social integration, involvement in the college activities, and interaction with peers and faculty, which, in turn, produce positive college learning outcomes.

207 Historically, underrepresented students are more likely to gain benefit from engaging in highimpact practices than are non-underrepresented students (Kuh, 2008; Kuh, Cruce, Shoup, Kinzie, & Gonyea, 2008; Pascarella & Terenzini, 2005). A greater increase in a freshman's likelihood of enrolling in a second year is seen among underrepresented students who participate in high-impact college activities as compared to non-underrepresented students who participate. Similarly, improvement in grades among underrepresented students was larger in their freshman year than that of nonunderrepresented students. Kuh (2008) recommended a student success strategy that involves students participating in two highly effective and educationally purposeful activities among ten listed above, while they are in college. For a student who is a first-time college entrant, first-year seminars and experiences or a freshman learning community will be an ideal choice. The compensatory effects of HIPS, which boost the performance of historically underrepresented groups, such as students of color and first-generation students (Kuh, 2008, p. 17), are of particular interest to California State University, Fullerton, where almost 40% of the undergraduates are underrepresented students. Shared characteristics of retention models and student engagement in HIPS suggest significance of policy and practices forming institutional environment to produce constructive student change during schooling. Developing and implementing institutional practices to maximize college impacts on student learning outcomes is critical in higher education. In the present study, we investigate the impacts of supplemental instruction and freshman learning community as culturally-responsive practices on student learning outcomes at an HSI. Both practices of supplemental instruction and freshman learning community are theoretically based on collaboratively-disposed culture of historically underrepresented students. Supplemental Instruction is a peer-facilitated review session conducted in a collaborative group learning setting in which students can openly discuss lecture materials with supplemental instruction leaders and other peers (Blanc & Martin, 1994; UKMC, 2013). The supplemental instruction leader is an undergraduate student who has successfully completed the course and has strong content knowledge. We explored how students with shared experiences and interactions in the SI session progress toward course performance and completion. One theoretical framework of supplemental instruction is the collaborative learning approach, which values group interaction (Arendale, 1993). Extensive research has shown the effectiveness of SI for teaching and learning (Blanc, DeBuhr, & Martin, 1983; Congos, Langsam, & Schoeps, 1997; Dizinno, Crisp, & Wilkerson, 2013; Ogden, Thompson, Russell, & Simons, 2003; Rath, Peterfreund, Xenos, Bayliss, & Carnal, 2007; Stone & Jacobs, 2006). Our institution also focuses on narrowing the achievement gap between underrepresented and non-underrepresented students in high-risk gateway courses. A learning community consists of a variety of educational approaches in which the curriculum is intentionally reconfigured around a communal theme, and a group of students take courses that are linked. The goal is to promote a shared learning experience as well as academic and social cohesion (Smith, MacGregor, Matthews, Faith, & Gabelnick, 2004). Students make reflective connections with other peers, faculty, and staff in the same learning community, which advances collaborative learning. A student cohort in a learning community builds a strong sense of community. A learning community is one of the most effective educational practices that fosters student engagement and maximizes college impact (Kuh, 2008). Tinto (2003) described learning communities as an effective educational practice due to its collaborative pedagogy, active student involvement, and mutual responsibility for group members learning. Purpose of Study The effects of two educational practices that have been adopted as culturally sensitive strategies to maximize college impact will be examined to assess how these practices influence student learning outcomes. We are particularly interested in culturally responsive practices that help close the

208 achievement gap between underrepresented and non-underrepresented students. The theoretical foundation for both practices is student engagement and the collaborative learning approach, which are congruent with the cultural characteristics of underrepresented students. We seek to discover the effects of supplemental instruction on student learning at the course level, while exploring the short-term and long-term effects of the freshman learning community at the program level. Both practices require active student engagement in the course and program. Based on the previous research on college impact, student learning outcomes, and the cultural characteristics of our diverse students, the following research questions were developed: 1. Does supplemental instruction, as an effective high-impact practice, improve student course grade and retention in Biology gateway courses? 2. Does supplemental instruction, as an effective high-impact practice, contribute to narrowing the achievement gap in Biology gateway courses between underrepresented and nonunderrepresented students? 3. Does a freshman learning community, as an effective high-impact practice, improve student 1- year and 2-year retention, 6-year graduation, and GPA? 4. Does a freshman learning community, as an effective high-impact practice, contribute to narrowing the achievement gap in 1-year and 2- year retention, 6-year graduation rates, and campus GPA between underrepresented and non-underrepresented students? Methodology Data Source and Sample Supplemental Instruction The sample consists of 1,211 undergraduate STEM major students who were required to take Biology 171 (Biodiversity and Evolution) between fall 2007 and fall 2010 (Table 1). Supplemental instruction has been implemented since fall 2007 in Biology 171, in the first course required for Biology majors, to improve student performance. Historically, Biology 171 is one of the key science gateway courses and for which students have had a low level of success and earned poor grades (Figures 1 and 2). The average course grade for the past 7 years was approximately C, and only 63% of students pass the course with a C or better. Of the 1,211 course takers, 486 students participated in supplemental instruction, whereas 725 students did not. As provided by a collaboration of the College of Natural Science and Mathematics and the Office of Institutional Research, student participation data collected in class sessions were connected with the institutional database of student demographics, course grades, and pre-collegiate GPA, including high school or transfer GPA. Historically underrepresented minority (URM) consisted of Hispanic, Black, and American Indian students. We excluded international students, unknown ethnicity, and multi-race students from the analysis as we were interested in investigating differential effects of supplemental instruction between underrepresented and non-underrepresented students. URM SI Participation # of Students Success Rate Course Grade Previous GPA Non-Participant 298.48 1.67 3.11 URM Participant 183.80 2.67 3.25 Total 481.60 2.07 3.17 Non-Participant 427.64 2.24 3.16 Non- Participant 303.88 2.93 3.24 URM Total 730.74 2.53 3.19 Non-Participant 725.57 2.01 3.14 Total Participant 486.85 2.83 3.24 Total 1211.68 2.35 3.18 Table 1: The Effects of SI on Course Success Rates and Course Grades

209 Biology 171 Course Grade ------- Course Grade, ------- High School GPA 4 3 2 1 0 3.22 3.18 3.22 3.15 3.19 3.16 3.17 3.16 3.18 3.07 3.23 3.16 3.26 3.19 2.15 2.06 2004 2005 1.71 1.72 1.94 1.87 2005 2006 2006 2007 2.27 2.19 2.31 2007 2008 2008 1.62 2009 2.48 2.58 2.68 2009 2010 2010 2.14 Total Figure 1: Historical Course Grade in Biology 171 (A: 4.0, B: 3.0, C: 2.0, D: 1.0, F: 0) Biology 171 Pass Rate 1 0.8 0.6 0.4 0.2 0 0.67 2004 0.76 2005 0.52 0.48 2005 2006 0.55 2006 0.48 2007 0.64 0.61 2007 2008 0.67 2008 0.46 2009 0.78 0.75 0.77 2009 2010 2010 0.63 Total Figure 2: Historical Course Success (Pass) Rates in Biology 171 Freshman Learning Community. A total of 27,564 first-time full-time students who entered between fall 2003 and fall 2010 were included in the analysis of 1-year/2-year retention rates and freshman/sophomore college GPA. The sample consisted of fall entrants only, and, among this sample, 3,069 participated in the freshman learning community, while 24,495 freshmen did not participate. A total of 12,198 URM freshmen comprised 12,309 Hispanic, 1,468 Black, and 150 American Indian students out of 27,564 first-time fulltime students (Table 2). International students, students of unknown ethnicity, and multi-race freshmen were excluded from the analysis. Approximately 54% of freshmen cohorts from fall 2003 through fall 2010 comprised first-generation students (Table 3). Similarly, 6-year graduation rates and college graduation GPA were obtained for the four appropriate fall cohorts from 2003 through 2006. Freshman Underrepresented Minority (URM) Non Underrepresented Minority (Non-URM) Program American URM Non-URM Black Hispanic Asian White Participation Indian Total Total Total No 122 1194 9451 10767 6106 7622 13728 24495 Yes 18 176 1237 1431 587 1051 1638 3069 Total 150 1468 12309 12198 6693 8673 15366 27564 Table 2: Demographic Composition of First-Time Full-Time Students between 2003 and 2010

210 Program Participation Parent Education Level Non-URM URM 4-year college Completed 7220 2405 Some College 3658 3318 Not Participate No College 1790 4373 Unknown 1060 671 Total 13728 10767 4-year college Completed 948 355 Some College 431 458 Participate No College 157 524 Unknown 102 94 Total 1638 1431 4-year college Completed 8168 2760 Some College 4089 3776 Total No College 1947 4897 Unknown 1162 765 Grand Total 15366 12198 Table 3: Parent Education of First-Time Full-Time Students between 2003 and 2010 Variables and Analysis Supplemental Instruction (SI) Two phases of analysis were performed, depending on the type of outcome being assessed. Course pass rate is a dichotomous variable (pass or fail) for the Biology course. In the first phase, logistic regression was conducted to assess whether the three independent variables and one interaction term between participation and URM status significantly predicted course retention and passing. The logistic regression analysis models event probability that predicts the presence or absence of an outcome based on a set of predictor variables. Pass or fail in the Biology 171 course is one of two learning outcomes. Precollegiate factors, such as high school GPA, transfer GPA, and URM status, were entered into the logistic regression analysis. Participation in SI also was used as a dichotomous independent variable to determine the impact of SI. Interaction effects of URM by SI participation also were investigated to determine the compensatory effects with HIPS on reducing the URM achievement gap. In the next phase, course grade was explored as the outcome variable. A 2 x 2 analysis of covariance was conducted, using student previous GPA as a covariate. SI participation and URM status were two independent variables with dichotomous values. Interaction effects of SI participation by URM status were determined to identify the uneven effects of SI on course grade between URM and non-urm students. Freshman Learning Community (FLC) The analysis to explore the effects of FLC was conducted in two phases: logistic regression and a 2 x 2 factorial analysis of covariance. In the first phase, 1-year retention, 2-year retention, and 6-year graduation rates were investigated as outcome variables. These three variables are dichotomous and were coded as yes or no, indicating retained/graduated or not. Logistic regression was employed to explore the factors that predicted 1-year retention, 2-year retention, and 6-year graduation. As independent variables, pre-collegiate factors such as high school GPA, URM status, and parent education were entered in logistic regression analysis. Freshman participation in the FLC also was used as an independent variable to determine its impact. The interaction terms of URM by learning community participation also were examined to identify the effects of HIPS in reducing the URM achievement gap. We were interested in determining how URM versus non-urm students responded to learning community participation with respect to retention and graduation rates. In phase two, freshman GPA, sophomore GPA, and graduation GPA were investigated as outcome variables. A 2 x 2 factorial analysis of covariance was employed to examine differences in FLC effects, depending on URM status and FLC participation. Interaction effects between FLC participation and URM status also were investigated to analyze the effects of the FLC on underrepresented students. High school GPA was included as a covariate to control for the effects of student ability and preparation before college.

211 Results Supplemental Instruction The descriptive statistics used to determine the effects of SI participation indicated large differences in success rates as well as in course grades between participants and non-participants (Table 4). The graphs in Figures 3 and 4 clearly show a considerable disparity in course success rates between participants and non-participants consistently across seven semesters. Course success was coded pass (C grade or above) versus fail (C- grade or below). There was a twenty-eight percentage point difference in course success rates between participants and non-participants; 85% of SI participants were likely to pass the course, while only 57% of non-participants were likely to pass. According to prior research, the strongest predictor of student learning variables, measured as graduation and college GPA, is student previous GPA (Abraham, Richardson, Bond, & Rod, 2012; Moon, Hershey, & Sullivan, 2010). Thus, student previous GPA was included in the logistic regression equation to separate out the unique effects of each predictor variable, particularly the unique effects of SI participation on course success and grade. URM Status SI Participation N Success Rate Course Grade Previous GPA Non-Participant 298.48 1.67 3.11 URM Participant 183.80 2.67 3.25 Total 481.60 2.07 3.17 Non-Participant 427.64 2.24 3.16 Non-URM Participant 303.88 2.93 3.24 Total 730.74 2.53 3.19 Non-Participant 725.57 2.01 3.14 Total Participant 486.85 2.83 3.24 Total 1211.68 2.35 3.18 Table 4: Success Rates and Course Grade by SI participation and URM % of Success (C or above) 1.00 0.80 0.60 0.40 0.20 0.00 Success Rate of SI Participants by Year Term 2007 2008 2008 2009 2009 2010 2010 Total 0.64 0.62 0.67 0.46 0.78 0.75 0.77 0.68 Non-Participant 0.53 0.47 0.62 0.38 0.69 0.60 0.65 0.57 Participant 0.91 0.78 0.95 0.68 0.91 0.82 0.88 0.85 Total Figure 3: Success Rate of SI Participants by Year and Term Multiple predictors were added in the analysis, including dichotomous variables (SI participation, URM status), continuous variables (high school GPA), and interaction terms (SI participation x URM status). The logistic regression results show that SI participation ( 2 =41.04, p <.001), followed by previous GPA ( 2 =26.70, p <.001) and URM status ( 2 =17.59, p <.001), are significant predictors of Biology course success rates (Table 5). The predicted odds of the independent variable, SI participation in course success, are 4.14, indicating that the odds of course success for SI participants are 4.14 times higher than for the non-participants. That is, holding all other independent variables constant, the odds of course success for SI participants is 314% more than the odds of course success for non-participants. Previous GPA also was a significant predictor, and the predicted odds of previous GPA were 2.25. That is, for every one-unit increase in previous GPA (so, for every additional 1 point of GPA), we expect a

212 2.25 increase in the odds ratio of course success, controlling for all other predictor variables. The odds of course success for URM were 58% less than the odds of course success for non-urm students. Effects Coefficient (B) Wald 2 P Odds ratio Previous GPA.811 26.699.000 2.25 SI Participation 1.421 41.039.000 4.14 URM -.654 17.585.000.52 SI Participation by URM -.061.302.841.94 Table 5: Logistic Regression: Analysis Model of Success Rates A 2 x 2 analysis of covariance was conducted to examine the effectiveness of SI participation in improving Biology gateway course grade (Table 6). SI participation and URM status were independent variables that predicted the dependent variable of course grade. Notable differences were observed in course grades between SI participants and non-participants, as indicated in Figures 4 and 5. Instead of a 2 x 2 analysis of variance, a 2 x 2 analysis of covariance was employed to compare group means more precisely, controlling for previous GPA. When we assessed the unique effects of SI, there was a need to use student previous GPA as a covariate that might be strongly related to student course grade. Thus, we reduced within-group error variance and isolated the differences attributable to the covariate and maximized the precision of between-group variance. Identification of the covariate was based on prior research findings that indicated previous GPA as the strongest correlate of learning outcome (Abraham et al., 2012; Moon et al., 2010). The dependent variable for a 2 x 2 analysis of covariance was course grade, which was converted to a number. For example, A was converted to 4.0, B was converted to 3.0, and so on. After controlling for the effects of the confounding variable, previous GPA, there was a significant interaction effect (F = 4.29, p <.05; Table 6) between SI participation and URM status. The effects of SI participation were not uniform across URM status, however. The URM and non-urm students responded differently to SI, while both groups significantly increased their course grades based on participating in SI. As shown in Figure 5, the URM students were likely to gain more benefits of SI than were non-urm students. Source Type III Sum of Squares df Mean Square F Sig. Previous GPA 83.243 1 83.243 69.163.000 Participation 160.661 1 160.661 133.485.000 URM 43.779 1 43.779 36.374.000 Participation * URM 5.160 1 5.160 4.287.039 Corrected Total 1721.432 1157 a. R Squared =.194 (Adjusted R Squared =.191) Table 6: Two-way (SI Participation X URM) Analysis of Covariance on Course Grades % of Success (C or above) 3.50 3.00 2.50 2.00 1.50 1.00 0.50 Course Grade of SI Participants by Year Term (4.0: A, 3.0: B, 2.0: C, 1.0:D, 0:F) 2007 2008 2008 2009 2009 2010 2010 Total 2.27 2.19 2.31 1.62 2.48 2.58 2.68 2.33 Non-Participant 1.97 1.79 2.14 1.33 2.21 2.21 2.26 1.99 Participant 2.95 2.60 3.10 2.28 2.86 2.75 3.03 2.82 Total Figure 4: Course Grade of SI Participants by Year Term

213 3 2.5 2 1.5 2.24 1.67 2.93 2.67 Non-URM URM 1 Non-participant Participant Figure 5: Interaction Effects (SI by URM ) in Course Grade: 07 10 Total Freshman Learning Community (FLC) The descriptive statistics used to determine the effects of FLC participation continually show benefits of FLC for 1-year and 2-year retention and 6-year graduation rates as well as for GPA between participants and non-participants (Table 7). FLC participants had five percentage point more 1-year retention than did non-participants. The retention gap is wider in 2-year retention rates, with 79% for participants and 72% for non-participants, and the graduation rates gap was eleven percentage points between FLC participants and non-participants. Substantial differences were seen in 6-year graduation rates between participants and non-participants. Specifically, 61% of FLC participants graduated in 6 years, while only 50% of non-participants graduated on time. These differences also were statistically significant, as shown in Table 8. We included previous high school GPA in the analysis to control for the pre-existing factors that influence retention and graduation. Logistic regression indicated that high school GPA, parent education, FLC participation, and URM status are all significant predictors of 1-year and 2- year retention and 6-year graduation rates, holding other variables constant. It is particularly noted that interaction effects were significant between FLC participation and URM status (F = 4.18, p <.05). The URM participants were likely to gain more advantages from FLC in terms of persisting and graduating in 6 years than were non-urm participants. That is, URM students received more benefits from FLC than did non-urm students in Figure 6. URM 1-Year 2-Year 6-Year Graduation Freshmen Program Status Retention Rates GPA Retention Rates GPA Graduation Rates GPA Non- Non-participant 0.83 2.85 0.74 2.86 0.54 3.01 URM Participant 0.86 2.96 0.80 2.92 0.62 3.07 URM Non-participant 0.76 2.63 0.69 2.65 0.43 2.88 Participant 0.84 2.80 0.77 2.76 0.59 2.99 Total Non-participant 0.80 2.76 0.72 2.77 0.50 2.96 Participant 0.85 2.89 0.79 2.85 0.61 3.04 Total 0.81 2.77 0.73 2.78 0.51 2.97 Table 7: Retention and Graduation Rates: Freshman Program by URM status 1-year 2-year 6-year Effects Wald odds Wald odds Wald odds B 2 p B ratio 2 p B ratio 2 p ratio High School GPA.86 429.49.000* 2.36.82 505.47.000* 2.26 1.1 480.46.000* 2.86 Parent Education.11 25.98.000* 1.11.08 20.58.000* 1.09.14 30.77.000* 1.15 Freshmen Program.22 8.71.003* 1.25.28 19.18.000* 1.33.32 17.78.000* 1.38 URM -.22 40.05.000*.80 -.19 36.16.000*.83 -.31 56.41.000*.733 Freshmen Program x URM.19 3.21.073 1.21.11 1.46.227 1.12.24 4.18.041* 1.28 Table 8: Logistic Regression for 1-yr Retention Rates, 2-yr Retention Rates, 6-yr Graduation Rates

214 0.80 0.70 0.60 0.50 0.40 0.54 0.43 0.62 0.59 Non-URM URM 0.30 Non-Participant Freshman Program Participant Figure 6: Figure 5: Interaction Effects (FLC by URM ) in Six-Year Graduation Rates A 2x2 analysis of covariance showed the effectiveness of FLC on 1-year GPA, 2-year GPA, and 6-year graduation GPA. We controlled high school GPA as a covariate to investigate effects of FLC and other independent variables on college GPA. Results in Table 9, Figure 7 and 8, revealed that there were interaction effects between FLC participation and URM status in 1-year college GPA and 2-year college GPA. URM and Non-URM students responded differently to the FLC experience with regard to their GPAs. The URM students tend to experience more benefits from FLC raising their college GPA at the 1 st and 2 nd year compared to non-urms. The interaction at the time of graduation was not significant, as the college GPA was calculated only for the persisters who have already been successfully retained for graduation, and the GPA gap between participants and non-participants decreased at graduation. Effects 1-year 2-year 6-year F ratio p F ratio p F ratio p High School GPA 5115.585.000* 4970.041.000* 1388.785.000* Parent Education 204.828.000* 183.252.000* 64.611.000* Freshmen Program 120.228.000* 42.930.000* 21.060.000* URM 115.274.000* 122.944.000* 9.651.002* Freshmen Program x URM 5.747.017* 4.073.044* 2.025.155 Table 9: 2x2 Analysis of Covariance for GPA at 1-year, at 2-year, at 6-year Graduation 3.00 2.90 2.80 2.70 2.60 2.50 2.96 2.85 2.80 2.63 Non-Participant Participant Freshman Program Non-URM URM Figure 7: Interaction Effects (FLC by URM ) in 1 st Year GPA

215 3.00 2.90 2.86 2.92 2.80 2.70 2.60 2.65 2.76 Non-URM URM 2.50 Non-Participant Freshman Program Participant Figure 8: Interaction Effects (FLC by URM ) in 2 nd Year GPA Conclusion and Implications The present study explored how college impacts were assessed in a comprehensive university that enrolls students from diverse cultural backgrounds. Two HIPS were employed at the course and program levels. Although there has been a great deal of research on college impacts for student success in US higher education, connections have not been made between and theory and practice in terms of student cultural characteristics, and cultural factors that influence the learning outcomes of underrepresented students have been overlooked. Because the culture of many underrepresented students has a collaborative nature, SI and FLC were found to be highly effective educational practices that reduced the achievement gap between underrepresented and non-underrepresented students and raised achievement for all. The first research question concerned the effects of SI on course success and grade in a Biology high-risk gateway course. Regardless of student ethnicity, SI significantly improved grades and success rates for both underrepresented and non-underrepresented students, which replicates prior research findings (Blanc et al., 1983; Congos et al., 1997; Dizinno et al., 2013; Ogden et al., 2003; Rath et al, 2007; Stone & Jacobs, 2006). SI in a collaborative group learning environment has been validated in this study as an effective high-impact practice that promotes student active engagement in learning, peer-topeer interaction, and peer-to-leader interaction, which results positive learning outcomes, including course retention, success, and high grades. For the second research question, which concerned narrowing the achievement gap, the results indicated the compensatory effects of SI benefits for underrepresented students but also the benefits for achievement in non-underrepresented students. These results are consistent with those of Kuh (2008, 2009b). SI has been found to be an effective educational approach that aligns well with the cultural characteristics of underrepresented students since SI has been implemented in collaborative learning environment of instructional session. The positive outcomes of SI were still present even after controlling for student previous GPA, which could be a strong correlate of student course performance and a confounding factor in the analysis. As such, SI was found to be indicative of a successful high-impact practice that maximizes college impacts that facilitate student learning outcomes as well as reduces the achievement gap for underrepresented students. Based on the collaborative learning and student active engagement employed in supplemental instruction, the third research question concerned the effects of FLC on student retention and timely graduation. FLC and SI are founded on the collaborative learning approach (Smith et al., 2004) and student engagement in college activities (Kuh, 2008). In this research, the longitudinal analysis of institutional data that tracked student progression toward graduation revealed significant and persistent effects of FLC through students continual persistence and successful graduation in six years. The effects of FLC in the first year became stronger as students progressed toward graduation. Significant group differences in both 1-year and 2-year retention were observed between participants and non-participants, and the disparity in 6-year graduation rates between participants and non-participants was higher than ten

216 percentage points. FLC participants persisted and completed a college degree better than did those who did not participate in an FLC. As noted, the positive findings were even obtained after controlling for student previous GPA to separate out the exclusive effects of the FLC. The interaction effects between underrepresented students and FLC participation are noteworthy, as California State University, Fullerton, serves more than 40% of historically underrepresented students who are enrolled in bachelor s degree programs. The underrepresented students gained more advantages from the FLC experience than did non-underrepresented students. However, all participants, regardless of ethnicity, who experienced the FLC showed significant improvement in graduation rates. In addition to resulting in increased retention and graduation rates, FLC encouraged students to achieve a high GPA throughout college. The students who participated in the FLC accomplished a significantly higher college GPA at 1-year and 2-year retention and 6-year graduation. The findings in this study are promising, as they take into account the cultural backgrounds of underrepresented students in the analysis of connection between retention models and higher education practices. Prior research has shown that experience in a learning community is directly related to students active engagement in the learning process and to their satisfaction with their college life (Zhao & Kuh, 2004). In this study, FLC and SI were meaningfully supported as HIPS for college success. The findings of this study have implications and lead to certain recommendations. First, collaboration of campus constituencies is vital to achieve maximum college impact and enhance student success. Students need the attention of a holistic university community that includes faculty, staff, and parents as well as other students. A strong sense of community in student college life promotes a positive learning environment and encourages students in active participation and engagement in learning. Collaboration and a connection between academic affairs and student affairs also is beneficial for student success. For underrepresented students whose parents do not have college experience or who are from low-income families, student support services are important to their success. Second, the role of faculty development should be re-assessed and should include more direct involvement in teaching and learning. What faculty think and value makes a difference with regard to the likelihood that students will participate in educationally effective practices (Kuh, 2009b, p. 690). To make courses more effective, the faculty development center should encourage faculty to use innovative teaching strategies including methods that employ collaborative learning and facilitate student active engagement in learning. Thus, incentives that encourage such innovative strategies for course redesign through faculty development are needed. Third, students cultural characteristics should be taken into account in policymaking and decision processes in college. A lack of connection between student cultural attributes and learning may cause underrepresented students to feel isolated and to drop out. Creating a learning environment that works with students home cultures will facilitate student success in college. In conclusion, not only should students be actively engaged for college success, but we all should be actively engaged as educational researchers, practitioners, and decision makers. Our collaborative efforts will enhance the learning opportunities of our students and increase their likelihood for success.

217 References Abraham, C, Richardson, M., & Bond, R. (2012). Psychological correlates of University students' academic performamce: a systematic review and meta analysis. Psychological Bulletin, 138(2), pp. 353-87. American Psychological Association. Arendale, D. (1993). Foundation and theoretical framework for Supplemental Instruction, In D. C. Martin & D. Arendale (Eds.), Supplemental Instruction: Improving first-year student success in high-risk courses. 2nd ed., (pp. 19-26). Monograph Series Number 7. Columbia, SC: The National Resource Center for The First Year Experience and Students in Transition. Available: ERIC Document Reproduction Service ED 354 839. Astin, A. W. (1985). Achieving Educational Excellence. San Francisco, CA: Jossey-Bass. Astin, A. W. (1993). What matters in college? : Four Critical Years Revisited. San Francisco, CA: Jossey- Bass. Astin, A. W. & E. L. Dey. (2001). Causal Analytical Modeling with Blocked Regression Analysis (CAMBRA). Los Angeles: Higher Education Research Institute, UCLA, 2001. Avellar, J., & Kagan, S. (1976). Development of competitive behaviors in Anglo-American and Mexican- American children. Psychological Reports, 39, 191-198. Bean, J. P., & Easton, S. B. (2002). A psychological model of college student retention. In J.M. Braxton (Ed.), Reworking the student departure puzzle (pp. 48-61). Nashville, TN: Vanderbilt University Press. Blanc, R., & Martin, D. C. (1994). Supplemental Instruction: Increasing Student Performance and Persistence in Difficult Academic Courses. Academic Medicine, 69(6), 452-454. Blanc, R. A., DeBuhr, L., & Martin, D. C. (1983). Breaking the attrition cycle: The effects of Supplemental Instruction on undergraduate performance and attrition. Journal of Higher Education, 54(1), 80-89 Bowman, N. (2013). Openness to Diversity as a Predictor of Student Engagement and Success. Paper presented at the Annual Meeting of the Association of Institutional Research, Long Beach, California. Cabrera, A. F., Crissman, J. L., Bernal, E. M., Nora, A., Terenzini, P. T., & Pascarella, E. T. (2002). Collaborative learning: Its impact on college students development and diversity. Journal of College Student Development, 43, 20-31. Congos, D. H., Langsam, D. M., & Schoeps, N. (1997). Supplemental Instruction: A successful approach to learning how to learn College Introductory Biology. The Journal of Teaching and Learning, 2(1), 2-17 Dizinno, G., Crisp, G., & Wilkerson, S. (2013). Evaluating SI Participation: Theoretical and Methodological Considerations. Paper presented at the Annual Meeting of the Association of Institutional Research, Long Beach, California.

218 Guiffrida, D. (2006). Toward a Cultural Advancement of Tinto's Theory. The Review of Higher Education, 29(4), pp. 451-472. Hagedorn, L. (2004). How to define retention: A New Look at an Old Problem. Unpublished Document. Center for Higher Education Policy Analysis. University of Southern California. Hagedorn, L.S., Moon, H.S., Buchanan, D., Shockman, E., & Jackson, M. (2001). Cooperative learning and unity: The perspectives of faculty, students, and TA s. (ERIC Reproduction Service No. ED 443 853). Hispanic Outlook in Higher Education Magazine (2009, 2010, 2011, 2012, 2013). Retrieved May 2009, May 2010, May 2011, May 2012, May 2013from http://www.hispanicoutlook.com/. Johnson, D. W., & Johnson, R. T. (1987). Learning together and alone: Cooperation, competition, and individualization (2 nd ed.). Englewood Cliffs, New Jersey: Prentice Hall. Kagan, S., & Madison, M. (1971). Cooperation and competition of Mexican, Mexican-American, and Anglo-American children of two ages under four instructional sets. Developmental Psychology, 5(l). Kuh, George D. (2008). High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. Washington, DC: Association of American Colleges and Universities. Kuh, G. D., Cruce, T. M., Shoup, R., Kinzie, J., &Gonyea, R. M. (2008). Unmasking the effects of student engagement on first-year college grades and persistence. The Journal of Higher Education, 79, 540 563. Kuh, G. D. (2009). The National Survey of Student Engagement: Conceptual and empirical foundations. In R. GonyeaandG.Kuh (Eds.), Using student engagement data in institutionalresearch. New Directions for Institutional Research, No. 141(pp. 5-20). San Francisco: Jossey-Bass. Kuh, G. D. (2009b). What Student Affairs Professionals Need to Know About Student Engagement. Journal of College Student Development, 50(6), pp. 683-706. LeMonda, C. S. (2008). Parents' Goals for Children: The Dynamic Coexistence of Individualism and Collectivism in Cultures and Individuals. Social development, 17(1), 183-209. MacGregor, B., MacGregor, J., Matthews, R., & Gabelnick, F. (2004). Learning Communities : Reforming Undergraduate Education. San Francisco, CA: Jossey-Bass. Moon, H.S. (2012). Relative Ranking of Hispanic Student Enrollment: Based on IPEDS Peer Analysis for Hispanic Students Enrollment. Unpublished Document. Institutional Research and Analytical Studies. California State University, Fullerton. Moon, H.S. & Hagedorn, L. (2006). Emancipated Foster Youth and Community Colleges: Snapshots of Achievement. Paper presented at the Annual Meeting of the Association of Institutional Research, Chicago, Illinois.

219 Moon, H., Hershey, J., & Sullivan, E. (2010). Well, I already knew that : Can you go beyond the quantitative findings on admissions available graduation rate variables? Based on Graduation Rates of First-Time Full-Time Freshmen and Full-/Part-time Transfer Student. Paper presented at the Annual Meeting of the California Association of Institutional Research, San Diego, California. National Center for Education Statistics (2013).The Integrated Postsecondary Education Data System (IPEDS). Retrieved May 10, 2014, from http://www.nces.ed.gov/ipeds/glossary/?charindex=r Obama, B. (2010). ABC News. Retrieved May 2013from http://abcnews.go.com/wn/president-barackobama-outlines-college-education-goal-university/story?id=11359759. Ogden, P., Thompson, D., Russell, A., & Simons, C. (2003). Supplemental Instruction: Short- and longterm impact. Journal of Developmental Education, 26(3), pp. 2-8. Pascarella, E. (1985). College environmental influences on learning and cognitive development: A critical review and synthesis. In J. Smart (Ed.), Higher Education: Research, Vol. 1 New York: Agathon Press and the American Educational Research Association (Division J), 1985, pp. 1-61, Reprinted in G. Kuh, et al., ASHE Reader on College Students, Association for the Study of Higher Education, 1989. Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research. San Francisco, CA: Jossey-Bass. Rath, K.A., Peterfreund, A.R., Xenos, S., Bayliss, F., Carnal, N. (2007). Supplemental Instruction in Introductory Biology I: Enhancing the Performance and Retention of Underrepresented Minority Students. CBE Life Sciences Education. Vol. 6, 203 216, 2007 Rendón, L., Jalomo, R., A. Nora. (2004). Theoretical considerations in the study of minority student retention in higher education. In Reworking the student departure puzzle ed. J. M. Braxton. Nashville, TN: Vanderbilt University Press. Slavin, R. E. (1990). Cooperative Learning: Theory, Research, and Practice. Englewood Cliffs, New Jersey: Prentice Hall. Slavin, R. E. (1991a). Student Team Learning (3rd ed.). Washington, D.C.: National Education Association. Slavin, R. E. (1991b). Synthesis of research on cooperative learning. Educational Leadership, Feb, 71-77. Slavin, R. E., & Oickle, E. (1981). Effects of cooperative learning teams on student achievement and race relations: Treatment by race interactions. Sociology of Education, 54, 174-180. Stone, M., & Jacobs, G. (2006). Supplemental Instruction: New Visions for Empowering Student Learning: New Directions for Teaching and Learning, Number 106. Tierney, W. (1992). An anthropological analysis of student participation in college. Journal of Higher Education, 63, 603-618. Tinto, V. (1987). Leaving College: Rethinking The Causes and Cures of Student Attrition (1 st ed.), Chicago: University of Chicago Press.

220 Tinto, V. (1993). Leaving College: Rethinking The Causes and Cures of Student Attrition (2 nd ed.), Chicago: University of Chicago Press. Tinto, V. (1997). Classrooms as communities: Exploring the educational Character of student persistence. The Journal of Higher Education, 68, 599-623. Tinto V. (2002). Establishing conditions for student success. Paper presented at the 11th Annual Conference of the European Access Network. Monash University, Prato, Italy. Tinto V. (2003). Learning Better Together: The Impact of Learning Communities on Student Success Higher Education Monograph Series, 2003-1, Higher Education Program, School of Education, Syracuse University Tinto, V. (2007). Research and Practice of Student Retention: What Next. Journal of College Student Retention: Research, Theory and Practice: 8(1), 1-19. UMKC International Center for Supplemental Instruction. Overview. Retrieved May 2013 from http://www.umkc.edu/asm/si/index.shtml. US Cenus. (2011). Intercensal Estimates of the Resident Population by Sex, Race, and Hispanic Origin for States: April 1, 2000 to July 1, 2010. Retrieved May 2013 from http://www.census.gov/popest/data/intercensal/state/st-est00int-03.html. US Cenus. (2013a). Retrieved May 2013 from http://www.census.gov/hhes/socdemo/education/data/cps/historical/index.html. US Cenus. (2013b). Retrieved May 2013 from http://www.census.gov/hhes/socdemo/education/data/cps/historical/index.html. Vygotsky, L. S. (1978). Interaction between learning and development. In M. Cole, S. Scribner, V. John- Steiner, & S. Sonberman (Eds.), Mind in Society: Development of Higher Psychological Process. Cambridge, MA: Harvard University Press. White House, (2010). Retrieved May 2013 from http://www.whitehouse.gov/. Zanger, V. V. (1990). Drawing on Diversity: A Handbook for and by Boston Teachers in Multicultural Multiracial Classrooms. Boston: Boston Public Schools. (ERIC Document Reproduction Service Number ED 323 281). Zhao, C., & Kuh, G. (2004). ADDING VALUE: Learning Communities and Student Engagement. Research in Higher Education, Vol. 45, No. 2, March 2004.

221 Appendix