Protocol: Effects of College Access Programs on College Readiness and Enrollment

Size: px
Start display at page:

Download "Protocol: Effects of College Access Programs on College Readiness and Enrollment"

Transcription

1 Protocol: Effects of College Access Programs on College Readiness and Enrollment Eleanor L. Harvill, Rebecca A. Maynard, Hoa T. H. Nguyen, Claire Robertson-Kraft, Namrata Tognatta, Rachél Fester Submitted to the Coordinating Group of: Crime and Justice Education Disability International Development Social Welfare Other: Plans to co-register: No Yes Cochrane Other Maybe Date Submitted: 3 February 2011 Date Revision Submitted: 6 October 2011 Approval Date: 29 November 2011 Publication Date: 1 September The Campbell Collaboration

2 BACKGROUND The Problem, Condition or Issue Over the past decade, national high school graduation and college enrollment rates have increased slightly, yet approximately 30% of students are still not graduating high school and, among those that do graduate, only 70% are enrolling in higher education (Snyder & Dillow, 2010). Additionally, recent research has demonstrated that, even when students earn a high school diploma, a large proportion have not developed the skills necessary to succeed academically in college (Callan, Finney, Kirst, Usdan, & Venezia, 2006; Greene & Winters, 2005). Of the 75% of high school graduates who enroll in two or four year colleges, only about 35% earn a bachelor s degree (Carnevale & Fry, 2000), and almost half of students in four year institutions and slightly less than two thirds of students in two year institutions require remedial coursework (Kirst & Bracco, 2004). Perhaps most concerning is the fact that, although college enrollment rates have increased, students of lower socioeconomic status or minority backgrounds, and students whose parents did not attend college, are still considerably less likely than their peers to graduate high school, pursue post-secondary education, and persist upon entry (College Board, 2010). Practitioners and policymakers offer a variety of explanations for enduring gaps in educational opportunity. One common explanation is that inequities result from the fact that financial aid programs do not provide sufficient funds to compensate for rising college costs (Fitzgerald, 2004; St. John, Musoba, & Simmons, 2003). Although financial aid is undoubtedly important, this review focuses on lack of academic preparation, information about college access, and social supports, which also have been identified as impediments (Ellwood & Kane, 2000; Perna, 2004). To provide a framework for our systematic review, we begin with a review of the literature examining the predictors of college readiness and enrollment. These predictors are situated within two theoretical approaches economic and sociological most often used to explain students decision-making processes. We adopt an integrated conceptual model, discussed in Perna (2006), to frame our examination of precollege outreach programs impact on college readiness and enrollment outcomes. Economists employ a rational model for explaining college enrollment decisions. This approach assumes that individuals act in ways that maximize their net benefits (Becker, 1993; Paulsen, 2001). Accordingly, high school students will weigh the benefits of higher education increased monetary and other societal advantages against both direct and indirect costs. Patterns of college enrollment cannot be explained through a straightforward cost-benefit analysis, however, as individuals have varying levels of academic preparation and resources necessary for success in higher education. A significant body of literature has demonstrated 2 The Campbell Collaboration

3 that low-ses and minority students are much less likely to graduate high school with the requisite skill set for success in higher education (ACT, 2004; NCES, 2003); as such, differential enrollment across groups can be explained, in part, by differential levels of academic preparation, or college readiness (Adelman, 1999). On the supply side, the rising costs of higher education make it more challenging for low income students, in particular, to enroll in college (Heller, 1997). Students perceptions and the reality of financial aid contribute to their predispositions towards (or away from) college, as well as their choice to enroll (Bound & Turner, 2007; Monks, 2009). Though a purely economic approach provides insight into the decision-making process, it largely ignores the nature of the information received by potential college enrollees and their families, and the broader influences affecting their ultimate decision (Avery & Hoxby, 2004; DesJardins & Toutkoushian, 2005). Sociological-cultural approaches address these shortcomings by considering individuals cultural capital, the system of attributes such as language skills and cultural knowledge (Bourdieu, 1986; McDonough, 1997), as well as their access to the social capital necessary to build important social networks (Coleman, 1988). Research documents how sociological factors, such as student, peer, and parental expectations (Goyette, 2008; Kim, Kevin, & Teresa, 2008; Wildhagen, 2009) and receipt of appropriate counseling and other social supports in high school influence patterns of college enrollment (McDonough, 2005; Perna et al., 2008). Perna (2006) offers a conceptual model, which integrates aspects of both economic and sociological approaches. Though the economic approach remains at the core of the model, it is situated within multiple contexts influencing the decision-making process. Students educational decisions are affected by their individual habitus, or system of values and beliefs, which shapes their views and interpretations, as well as by the social and cultural capital made available through their specific school and community context (Paulsen & St. John, 2002). Outcomes are further influenced by components of the higher education system, as well as broader changes in the social, economic, and policy landscape. College Access Interventions For decades, college access programs in the United States have aimed to improve college enrollment rates, particularly for underrepresented populations. Beginning with the TRIO programs in the 1960s, the federal government has sponsored many of these policies and, in recent years, has considerably expanded its role. Efforts at the federal level have been supplemented by numerous programs run by state and local governments, districts, foundations, universities, and private businesses. For this review, we define college access programs to be pre-college interventions that explicitly identify increasing college readiness and/or college enrollment as a primary goal of the program. This review examines two broad categories of college access programs: (1) prepackaged whole school reform efforts; and (2) supplementary services provided at the student level. 3 The Campbell Collaboration

4 How the Intervention Might Work Whole school reform efforts alter the organizational structure of the school and tend to focus on two key predictors of college enrollment students academic preparation and social supports (Martinez & Klopott, 2005). Supplementary pre-college outreach programs provide services, typically outside of school hours, that encourage students to become interested in attending college and increase parental engagement in the process. Despite commonalities between and among program types, individual initiatives vary greatly in approach, duration, and intensity (Gandara, 2001; Perna, 2002). Though there are many ways to organize a discussion of pre-college outreach programs, we use the taxonomy presented in Gandara (2001). This typology is based on programs primary source of funding private nonprofit, university based, state government sponsored, federal government sponsored, community based, and K-12 school systems. Within each program type, components are classified along the following dimensions: (1) Counseling provides students with access to information on the college enrollment process (2) Academic enrichment focuses on improving academic preparation so students have the requisite skill set (3) Parent involvement involves parents in the college enrollment process (4) Personal enrichment and social integration broadens students understanding of available opportunities through activities such as speakers and fieldtrips (5) Mentoring provides students with one-on-one mentoring throughout the college enrollment process (6) Scholarships offers financial aid for students in need These components, either alone or in combination, address the key predictors of college enrollment. For example, academic enrichment directly targets inadequate academic preparation, counseling and parent involvement address shortcomings in families cultural and social capital, and personal enrichment targets students low educational aspirations (Gandara, 2001). 1 Though much is known about the predictors of both students academic behaviors and preparedness for college, we have limited information about which program components are most effective in improving college readiness and raising college enrollment rates. 1 Programs with different funding sources are responding to very different needs of their primary stake-holders (Gandara, 2001). When programs receive funding from multiple sources, we follow Gandara (2001) in classifying them based on which sector started the program and/or maintains primary responsibility for overseeing day to day operations. A list of primary funding sources for a number of large-scale programs is available in Gandara (2001, pp ). 4 The Campbell Collaboration

5 Outcomes of Primary Interest This review investigates the impact of college access programs on college readiness and enrollment. We focus on these outcomes because they are the most aligned with the goals of the interventions we are considering. Also, as the interventions are fielded to pre-college students, we expect that most studies will report either pre-college or early-college outcomes. We consider four measures of college readiness: completed coursework, mathematics achievement test scores, language arts achievement test scores, and high school graduation. Our preferred measure of college readiness is an indicator of completed high school coursework (Adelman, 2006; America Diploma Project, 2004; Rose & Betts, 2001). The academic intensity of high school coursework is the best pre-college predictor of college completion (Adelman, 2006). The remaining measures are less preferred, but more commonly reported. Even though current state assessments are typically not well aligned with the rigor of college entrance requirements (Martinez & Klopott, 2005), students math and language arts achievement scores can still provide some insight into their college readiness. Though a high school diploma does not guarantee college readiness (America Diploma Project, 2004), it is one of the three critical steps that determines the likelihood that a disadvantaged student will enter college (Cabrera & La Nasa, 2001). As part of our coding process, we will note the full range of outcomes reported in the candidate studies. If there are multiple high quality studies reporting a particular outcome measure, we will consider expanding our analysis to include that measure. Outcomes measuring completion of milestones in the college application process, like SAT completion, and those measuring persistence in and completion of college would particularly enrich our analysis. Why it is Important to do the Review Though several published reports have produced comprehensive inventories of college access programs (Gandara, 2001; Perna, 2002; Tierney, Bailey, Constantine, Finkelstein, & Hurd, 2009), no systematic review and meta-analysis of the evidence regarding their effectiveness exists. 2 This systematic review will fill that knowledge gap by systematically gathering, reviewing, and synthesizing the findings on the effectiveness of programs designed to improve college readiness and enrollment for disadvantaged populations. In so doing, it will provide guidance for policymakers and practitioners implementing college 2 For the What Works Clearinghouse Practice Guide on Helping Students Navigate the Path to College, a panel of experts identified promising practices high schools engage in to increase college access (Tierney et al., 2009). A comprehensive search of the literature and review of studies by the WWC was performed to determine the level of evidence supporting the specific recommendations of the panel (Tierney et al., 2009). In contrast, this paper reports the effects of college access programs systematically, without limiting attention to particular strategies for increasing access. 5 The Campbell Collaboration

6 access programs and will identify important gaps in the scientific evidence base that warrant further research. OBJECTIVES The purpose of this review is to summarize the evidence regarding the effectiveness of college access programs on college readiness and college enrollment. Specifically, this review explores the following questions: 1. What is the corpus of evidence to judge the effectiveness of programs aimed at increasing college readiness and enrollment? 2. What does that evidence base tell us about the effectiveness of various intervention strategies? More specifically: a. What types of programs have been rigorously evaluated? b. How representative are these programs of the range of programs that exist? c. What are the estimated average impacts of programs on college readiness outcomes within the following four domains: math achievement, language arts achievement, completed coursework, and high school graduation? d. What are the estimated average impacts of programs on college enrollment rates? e. What are the estimated average impacts of programs targeted at students in different grades or of different ages? f. What are the estimated average impacts of programs for males and for females? g. What are the estimated average impacts of programs targeted at different racial/ethnic groups? h. What are the estimated average impacts of programs targeted at students with different levels of family income or socioeconomic status? i. Do the estimated average impacts of programs vary across demographic groups, including race/ethnicity, gender, and socioeconomic status? j. Do the estimated average impacts of programs vary depending on entering achievement levels? METHODOLOGY Criteria for Inclusion and Exclusion of Studies in the Review The following twelve criteria will determine whether a study will be included in the review database for purposes of estimating program impacts: 6 The Campbell Collaboration

7 Study design: The review will include randomized controlled trials and well-controlled quasi-experimental designs. Regression discontinuity designs will be evaluated on a case-bycase basis. 3 Interventions to be included: The review will include only interventions that explicitly identify increasing college readiness and/or college enrollment as a primary goal of the program. These interventions can include services and activities such as academic instruction, academic advisement, counseling, mentoring, transmission of information about how to apply for college or apply for financial aid, provision of financial aid, and other services meant to mitigate factors impeding college readiness or college enrollment. Counterfactual treatment: The review will include only studies where the counterfactual condition (i.e., the control group) received normal college counseling and support services. A study will not be included if the counterfactual condition was another college access program. Furthermore, studies in which the estimated effect of the treatment is confounded with another factor (such as the school attended) will not be included. Target population: The review will include only studies of interventions that target students between grade 6 and 12, or students of comparable ages who have not yet graduated from high school or earned a General Education Development certificate. 4 The review will not include studies in which less than 75% of the sample falls within the target population for this review. Key outcomes: The review will include only studies that measure one or more of the following outcomes: (1) college readiness; and (2) college enrollment. College readiness outcomes may fall within one or more of four domains: (1) math achievement, (2) language arts achievement, (3) completed coursework, and (4) high school graduation. 5 Math and language arts achievement should be measured separately using standardized tests administered to the entire sample or to representative subsets of the sample. 6 Completed coursework may be measured in one of two ways: (1) by a composite measure of academic performance based on the high school transcript data, such as 3 Regression discontinuity designs estimate the impact of the intervention for individuals near the eligibility cutscore, not the average treatment effect for the entire population of interest. This caveat will be taken into consideration when determining whether to include an RDD study in the meta-analysis. It is worth noting that preliminary search results did not identify any RDD studies of college access programs. 4 If an insufficient number of high quality studies of interventions for middle school students are available, the reviewers will restrict attention to studies of high school students. 5 Studies that only report a composite measure of college readiness, such as the one developed by Greene & Winters (2005) or Adelman s Academic Resources variable (2006), and do not separately report outcomes within one of the five domains will be coded, and the findings will be reported. However, the impact findings will not be included in the meta-analysis. If a large number of studies of this sort are found, we will consider also reporting out results for a composite academic readiness measure. 6 The requirement that the tests be administered to the entire sample or to representative subsets of the sample is intended to insure that we avoid the selection issues raised when one observes scores only for students who are intending to apply for college. 7 The Campbell Collaboration

8 Adelman s Academic Intensity measure (Adelman, 2006); or (2) (if the composite variable is unavailable) by a measure of the highest level of math completed (Rose & Betts, 2001). The preferred measure of high school graduation is the binary outcome indicating whether the student received a standard high school diploma within four years of starting high school (Bozick & Lauff, 2007; DesJardins & Lindsay, 2008; Eccles, 2008). College enrollment is measured by whether a student enrolled in any two-year or four-year college following the date corresponding to on-time high school graduation (Johnson, 2008; Park, 2008; Snyder, Dillow, Hoffman, & National Center for Education Statistics, 2008). The review will include up to two measures of enrollment per study: (1) within two years following the on-time graduation date; and (2) by the time of the most recent followup (if that is later than the on-time date). 7 Given the outcome measures described above, neither reliability nor over-alignment of the intervention and measures should be concerns for this review. Requiring standardized tests to measure achievement should ensure adequate reliability of those measures. In general, we would not expect there to be concerns related to the reliability or over-alignment of the other measures, since they relate to observed behaviors. Still, the data-coding instrument, included in Attachment A, instructs reviewers to note any concerns they may have regarding overalignment and/or reliability of outcome measures. At times, studies report impact estimates for key outcomes based on subsamples defined by criteria that are endogenous to the intervention. For example, studies might provide the impact of the program on enrollment rates for students who were admitted to at least one college, ignoring the fact that the program potentially affects the probability that the student applies to college, as well as the probability of acceptance conditional on application. If outcomes are reported for a non-representative subset of the study sample (such as for high school graduates or for students admitted to college), these outcomes will be included in the review only if it is possible to transform them to measures that pertain to the full study sample (or a representative subset of the study sample). For example, if results of college enrollment are reported based on a sample of college applicants and the study reports the proportions of treatment and control group members who applied to college, we would transform the reported conditional means into unconditional means by dividing the conditional means by the proportions that applied to college. Setting and timing of interventions: The review will include only studies reporting on interventions fielded in the past twenty years, since 1990; those that were conducted in the United States or in developed countries with a similar secondary and higher educational systems; and those written in English. The nature and prevalence of college programs is sufficiently different in the period pre-1990 as to raise questions about the current relevance 7 When studies report 2- and 4-year college enrollment separately, we will construct a measure of overall college enrollment by summing the proportions in the two categories. In addition, we will record the measures separately. 8 The Campbell Collaboration

9 of evidence regarding effectiveness. Similarly, studies of programs conducted in countries with dissimilar educational systems are not likely to have relevance to the U.S. and other represented countries. We exclude studies reported in languages other than English simply because we do not have the capacity to translate such studies. Data collection: The review will include only studies for which there is reason to believe that there is no systematic reporting bias in the outcome measures between the program and the control groups. For example, we would exclude a study in which the data for the program group were collected by the program staff, while independent data collectors conducted surveys with the control group, unless the study provided convincing evidence that this difference in data collection method did not lead to differential survey response bias. Appropriate analytic methods and adequate reporting: The review will exclude studies for which it is not possible to construct unbiased estimates of the effects of the interventions on one or more of the outcomes of interest for the full (intention-to-treat) follow-up sample or for sample subgroups defined by characteristics that are independent of the intervention. 8 Attrition: The level of attrition of a randomized controlled trial will be judged based on both the overall level of attrition and the differential level of attrition. Overall attrition is defined as the proportion of individuals lost from the study between randomization and the postintervention measurement. Differential attrition is the absolute value of the difference between attrition from the treatment and the control groups. We will use the What Works Clearinghouse Version 2 Standards (U.S. Department of Education, 2009, p. 33) to judge whether the study has an acceptable level of attrition, given these two measures. These standards allow for the fact that the expected bias in impact estimates is affected by both the overall attrition rate and the differential attrition rate. Roughly, these standards accept overall attrition rates up to 40 percent, if the difference in attrition rates between the treatment and control groups is 2 percentage points or fewer; in contrast, they only accept overall attrition rates in the neighborhood of 10 percent or less if the difference in attrition rates between the treatment and control groups rises to 10 percent or more. A randomized control trial with a high level of attrition will be considered a quasiexperimental design and included in the review provided it meets the quality standards established for quasi-experimental designs by demonstrating baseline equivalence for the analytic sample. A quasi-experimental design will not need to meet attrition standards; rather it must demonstrate baseline equivalence for the analytic sample (see below). 8 Cluster randomized trials analyzed at the individual level will be included and their estimates will be adjusted to correct for clustering. Studies will only be excluded if it not possible to use standard analytic tools to correct effect sizes and standard errors. 9 The Campbell Collaboration

10 Baseline equivalence: Quasi-experimental designs and randomized controlled trials with high levels of attrition must establish equivalence of initial measures for the treatment and comparison groups in the analytic sample to be included in the review. The study will only be included in the review if average pre-intervention continuous measures differ between the intervention and comparison groups by less than 0.25 of the population standard deviation. 9 A similar standard will be imposed on dichotomous pre-intervention standards. 10 Baseline equivalence must be established for demographic characteristics of the two groups and for achievement measures. Concealment: For this review, we will not require allocation concealment. While in many studies concealment is important for avoiding reporting bias, studies of educational interventions rarely use blind or double-blind designs for practical reasons. Search Strategy for Identification of Relevant Studies The search for relevant research will include two phases: a preliminary search phase and a final search phase. We have completed a preliminary search for relevant research, which included a keyword search of electronic databases and additional searches to identify the grey literature. The final search phase will extend the preliminary search by using additional electronic databases, searching select databases using controlled language, and adding strategies for identifying research in the grey literature. The entire search strategy is presented below. The presentation does not separate the preliminary search phase from the final search phase for ease of exposition. The search strategy is followed by a description of our bibliographic management strategy. Last, we present the results of our preliminary search. Electronic Database Search Strategy We will search the following databases for articles published between 1 January 1990 and 1 September 2011 in English. This timeline was determined by the date the search and by our exclusion of interventions that were fielded prior to For all databases, we will perform free-text searches. For the subject specific databases ERIC, PsycInfo, and Sociology Abstracts, 11 we will also perform controlled language searches. 9 This requirement is consistent with accepted quality standards for quasi-experimental designs, such as those developed by the What Works Clearinghouse (U.S. Department of Education, 2009, p. 15). 10 To assess the baseline equivalence of dichotomous variables, the reviewer calculates the log-odds ratio effect size for the pre-intervention measure, translates this into a continuous effect size measure, and then compares this result to Both of these calculations are performed using simple algebraic transformations of formulae found in Lipsey & Wilson (Lipsey & Wilson, 2001, p. 202). (Note that the calculation of the odds-ratio effect size does not take the size of the sample into account. Therefore, the reviewer is instructed not to assess baseline equivalence for uncommon demographic categories.) 11 We believe that the controlled vocabulary available in the EconLit database will not be particularly helpful in identifying relevant studies. 10 The Campbell Collaboration

11 General bibliographic databases/full-text journals EBSCO MegaFile JSTOR Project Muse Subject specific bibliographic databases: ERIC EconLit PsycInfo Sociology Abstracts Dissertation and theses databases ProQuest Dissertations and Theses Grey literature databases PolicyFile SSRN elibrary Within each database, our free-text searches will be of the form: (topic keyword) AND (intervention keyword) AND (evaluation keyword). All such queries generated by the following list of keywords will be performed: TABLE 1: SAMPLE SEARCH STRATEGY Topic Intervention Evaluation pre-colleg* precolleg* college AND transition* college access college enrollment college readiness college preparation college AND outreach college AND bridge college going college attendance program intervention evaluation experiment effect* Controlled language searches will be performed for the ERIC, PsycInfo, and Sociology Abstract databases. To identify relevant descriptors, we will look up the records of relevant studies identified in the preliminary search and record the descriptors in a word document. 11 The Campbell Collaboration

12 We will use the database thesaurus to identify additional potentially relevant descriptors. The final set of descriptors will be recorded and presented in the review paper. Search Strategy for Other Sources Before beginning this phase of the process, we will complete the review of the studies identified through the database search and leverage those results to find yet more studies. Conference abstracts or proceedings: We will browse the online programs of the following conferences to identify potentially relevant studies. For the smaller conferences, we will scan the entire program. For the larger conferences, we will use the association s divisions and descriptors to limit our search. American Educational Research Association (AERA) Online Paper Repository ( contains abstracts for papers presented at the AERA Annual Meetings from 2005 to the present. For the 2010 and 2011 conferences, the full-text of papers can be downloaded if the authors chose to make the text available. Association for Public Policy Analysis and Management (APPAM) Annual Fall Research Conference Archives (https://www.appam.org/conferences/fall/archives.asp) Society for Research on Educational Effectiveness Spring Research Conference Programs (http://www.sree.org/conferences/2011/program/; These programs include links to abstracts. Online Repositories: We will browse the following online sources of research to find relevant research. The links are specific to the pages within the site that list relevant research. These pages will be read and relevant information will be downloaded and recorded. The US Department of Education s web site includes reports of research funded by the department ( In addition, we will search for all IES grants (http://ies.ed.gov/funding/grantsearch/index.asp ) within in the Post-Secondary & Adult Education program area to identify relevant research projects currently underway. The National College Access Network maintains a searchable database of publications, research reports, websites, and other resources related to improving college access and success for underserved populations. We will browse the entries in the database by topic, focusing on two topics: college access programs and college readiness. (http://www.pathwaystocollege.net/pcnlibrary/listarticlesbytopic.aspx?topicid=19; 12 The Campbell Collaboration

13 Citation indexes: We will use the ISI Citation Indexes to search for articles that cite or are cited by existing inventories of college access programs (e.g. Gandara, 2001; Perna, 2002; Tierney, Bailey, Constantine, Finkelstein, & Hurd, 2009). We will also use the ISI Citation Indexes to search for articles that cite or are cited by any studies that pass our quality screening. Web Searching: From the studies that pass our quality screening and from the existing inventories of college access programs, we will compile a list of the names of college access programs. We will then search the web for evaluations of these programs, using the name of the program followed by the word study as the search term: e.g. Gear Up study. These searches will be performed on both Google.com and Yahoo.com. Solicit Feedback from Subject Experts: After we have completed the searches described above, we will screen the identified studies and present the search results to experts in the field, again asking if they know of any work that is missing from the review. Management of References and Document Retrieval To manage bibliographic records for this review, we have created a RefWorks database. In addition to storing citation information, this program allows us to easily identify the source of the citation, track the outcome of the review process, and attach copies of the study and the coding instrument. The following procedure for managing citations was developed and refined during our preliminary search process, the results of which are described below. For the search results generated by a particular search of a particular database, the number of hits was recorded in an excel spreadsheet, and the results were saved to RefWorks with a tag indicating how the citation was found. 12 Immediately after adding these citations, we searched for and removed all duplicate citations in RefWorks. We were then able to use the tagging system in RefWorks to determine the number of unique citations found by each database. A similar process was used to add citations found through hand-searching. For each unique citation, we reviewed the title and abstract. If the paper seemed potentially relevant, we obtained the paper for review. To obtain the study for review, we will use the full-text feature of the referring database, if available in the University of Pennsylvania Library system. Otherwise, we will use the PennText Article Finder tool on the library s web site to find published articles and the library catalogue to find books. If the study is unpublished, we will search online to find a full-text paper. We will contact the study author via to request copies of unpublished dissertations and papers that are not available online. If these resources are not sufficient, we plan to use interlibrary loan to request the document. 12 The University of Pennsylvania Library s online resources allow us to directly export the references from the electronic database to RefWorks. After performing a search in one of the electronic databases, we choose select all and then save to RefWorks. 13 The Campbell Collaboration

14 Preliminary Search Results The preliminary search focused on identifying documents written or published between 1 January 1990 and 1 February The preliminary search utilized the following databases: Dissertation Abstracts, EBSCO Megafile, Econlit, ERIC, JSTOR, and Project Muse. Free-text searches using the strategies described above were performed. We feel that these strategies adequately covered the range of potentially relevant studies. We did not perform any controlled language searches and believe that these could identify additional studies in subject specific databases. This preliminary search process generated 2068 initial hits from the electronic databases alone, 1151 of which were unduplicated. A search for relevant studies from the U.S. Department of Education, the National College Access Network, and Google generated an additional 24 citations, bringing the total studies for initial review for relevance to Review of abstracts for the 1175 identified 192 studies that appeared potentially relevant and, thus, warranted a full-text review. Results from this search appear in the table below, broken down by electronic database and other searches. TABLE 2: SUMMARY OF PRELIMINARY SEARCH RESULTS (NUMBER OF STUDIES) Search Source Citations Identified Unique Citations Obtained for Review Electronic Databases EBSCO Megafile JSTOR Econlit ERIC Dissertation Abstracts Project Muse US Department of Education Hand Searches Total The Campbell Collaboration

15 Description of Methods Used in the Component Studies Both quasi-experimental and randomized controlled trials contribute to our knowledge about program effectiveness. A recent evaluation of the EXCEL program (Bergin, Cooks, & Bergin, 2007) is an example of a seemingly well-conducted randomized controlled trial in this area. Acceptable quasi-experimental designs generally range from analyses of existing survey or administrative data to studies that collected data from both program participants and a prospectively identified comparison group. For example, a recent evaluation of Talent Search (Constantine, Seftor, Martin, Silva, & Myers, 2006) demonstrates the propensity score matching approach to program evaluation using existing data sources, while a recent evaluation of GEAR UP (Standing, Judkins, Keller, & Shimshak, 2008) illustrates application of quasi-experimental methods in the context of prospective identification of the comparison group. The methods of these three studies are described below. 13 The recent EXCEL evaluation (Bergin et al., 2007) is a randomized field trial that stratified the sample based on achievement level, gender, and ethnic group before randomly assigning applicants to treatment and control groups. The authors collected the data for this evaluation and analyzed it using ANOVA to examine the overall impact of the program and the sensitivity of the program impacts to initial achievement level. In addition, raw tabulations are presented for binary outcomes and means and standard deviations for continuous outcomes. We can use these data to compute log-odds ratios and effect sizes. The Talent Search study (Constantine et al., 2006) is a quasi-experimental design that analyzed administrative records to evaluate the impact of the Talent Search program using a comparison group derived from propensity score matching. First, the propensity score was calculated. The first step in this method involved using logistic regression to estimate the probability that a student with a set of observed characteristics would participate in the program. Then, caliper matching was used to identify the comparison group. Based on the similarity of their estimated propensities to have participated, comparison students for each student who participated in the program were selected from those not in the participant sample. Finally, weights were constructed for each comparison individual to adjust for the number of treatment individuals for whom she or he served as a comparison, the number of comparison individuals with which each of those treatment individuals was associated, and the relative size of the treatment and control groups. The impact estimates were then generated using the treatment sample and the matched comparison group from ordinary least squares, weighting the contributions of comparison individuals. The regressions included covariates to increase the precision of the estimates and to control for any other measured differences between the treatment and comparison groups (Constantine et al., 2006, p. 81). 13 For a discussion of how the different analytical approaches illustrated by these studies will be synthesized, see the Statistical procedures and conventions section. 15 The Campbell Collaboration

16 The recent GEAR UP evaluation (Standing et al., 2008) is a quasi-experimental design study that first selected 20 projects receiving GEAR UP funding. It then selected two matched middle-schools within each project area, and, from each of the matched schools, drew a random sample of seventh graders. The comparison school was selected to match the participating school through a structured process that entailed the following steps: (1) a set of schools near the participating school was identified; (2) the similarity of each of these schools to the participating school was measured on a range of observable characteristics; and (3) the school most similar to the participating school on average was selected (Standing et al., 2008, pp. A-6 and A-7). Data analysis was conducted using a variety of methods, including hierarchical linear models and replicated counterfactual projection. Criteria for Determination of Independent Findings To guarantee that the pooled impact estimates are constructed from a set of independent measures, we will separately estimate the impacts for each outcome and domain. Each study may contribute only one impact estimate to each pooled impact estimate. 14 The final review will include findings for up to five outcome domains: (1) math achievement; (2) language arts achievement; (3) high school course completion; (4) high school graduation; and (5) college enrollment. As noted, we will include only those math and language arts achievement outcomes that pass the standards described above (i.e., the measure is a standardized test administered to the entire sample, and the study demonstrates comparability of the treatment and control groups either by meeting the differential and overall attrition standards for randomized controlled trials or by presenting evidence of baseline equivalence for the analytic sample). If we encounter cases where there are multiple measures that meet these criteria within a study, we will give priority to the test given closest to the expected date of graduation, and, if there is more than one such measure, we will select the test that is more commonly administered. For measures of completed coursework, we will only include measures based on the student s high school transcript as of the on-time date for graduation or later, which is four or more years after the student started high school. If multiple measures of completed coursework from this transcript are reported, we will give priority to the most recent transcript and to a composite measure that reflects the academic level of courses completed, such as Adelman s Academic Intensity measure (Adelman, 2006). Second priority would be to use the highest math course completed (i.e., AP calculus; Pre-calculus; Algebra 2; Geometry; and Algebra). If neither of these measures is available, we would accept a 14 Some reports include the results of multiple studies based on independent samples and, therefore, may contribute multiple measures to a meta-analysis. For example, the recent Talent Search evaluation (Constantine et al., 2006) separately reports the impact of the Talent Search program in three different states. Since these samples are non-overlapping and the authors do not present pooled impact estimates, we would treat the results from each state as a separate study. 16 The Campbell Collaboration

17 composite measure of completed coursework that reflects the number of units completed, such as a binary indicator of completion of the New Basics curriculum described in A Nation at Risk (National Commission on Excellence in Education, 1984). High school graduation is defined above to be the binary variable indicating that a student received a standard high school diploma within four years of starting high school. This definition should insure that a study reports at most one measure for this domain. College enrollment is defined above to be the binary variable indicating whether a student enrolled in any college (two-year or four-year) after the on-time date for high school graduation. When there are multiple measures of college enrollment taken at different points in time, we will focus on up to two measures: (1) the earliest date following the on-time date for graduation and (2) the most recent time period. Details of Study Coding Categories All retrieved studies will be coded using a data-coding instrument, included in this document as Appendix A. The first section of the coding instrument records bibliographic information and screens the studies for relevance. Relevance screening includes verifying that the intervention is a college access program, the study design is eligible, the study setting and timing are appropriate, and that the study reports one or more relevant outcome measures. 15 If a study fails any of the relevance criteria, a summary statement explaining the failure is written and no further information on the study is recorded. The second section of the data-coding instrument screens the study for quality, according to the criteria established above. This section records how participants are assigned to treatment and control groups, describes the counterfactual condition, notes the existence of any factor potentially confounded with treatment, assesses the quality of available outcome measures, calculates both overall and differential attrition for randomized controlled trials, and assesses the baseline equivalence of the treatment and control groups for quasiexperimental designs. Again, if a study fails any of the quality criteria, a summary statement explaining the failure is written and no further information on the study is recorded. If quality screening raises concerns but does not clearly exclude the study, a statement of the concern is recorded and the remainder of the instrument is completed. The final section of the data-coding instrument documents a range of moderating variables, including demographic details of the target population and the availability of program impacts for a variety of subgroups. To describe program implementation, we follow the taxonomy presented in Gandara (2001) in categorizing programs by their primary funding source and by the components of the programs. Programs are therefore categorized as belonging to one of the following categories based on the primary source of funding: (1) 15 A detailed description of acceptable outcome measures is given in Section a. Criteria for inclusion and exclusion of studies in the review and Section b. Criteria for determination of independent findings above. 17 The Campbell Collaboration

18 private nonprofit; (2) university based; (3) state government sponsored; (4) federal government sponsored; (5) community based; and (6) K-12 school systems (Gandara, 2001). 16 College access programs include one or more of the following components: (1) counseling; (2) academic enrichment; (3) parent involvement; (4) personal enrichment and social integration; (5) mentoring; and (6) scholarships (Gandara, 2001). See Attachment A for a complete list of all retained moderating variables. See the next section for a description of the moderator analysis to be included in the review. The data-coding instrument instructs reviewers to record any additional outcomes other than the key outcomes to be included in the review. After all studies have been coded, these additional outcomes will be revisited with particular attention to outcomes of secondary interest, such as measures related to completion of milestones in the college application process or success in the first year of college. Should there be a sufficient number of studies reporting a particular measure of interest, an additional meta-analysis will be conducted on this outcome. Statistical Procedures and Conventions Statistical software: Microsoft Excel will be used to code and store the data extracted from the studies and to transform extracted impact estimates reported in natural units into standardized mean differences for continuous measures and log odds ratio effect sizes for dichotomous measures. Comprehensive Meta-Analysis (Borenstein, Hedges, Higgins, & Rothstein, 2005) will be used to calculate pooled effect sizes for each of the four college readiness domains and for college enrollment. Effect size calculations: For the math achievement and language arts achievement domains, outcome measures are generally continuous. These study impact estimates will be reported in natural units (where possible) as well as converted to standardized mean differences. Both the standardized mean differences and the standard error associated with each standardized mean difference will be calculated as described by Lipsey & Wilson (2001, p. 49). The pooled effect size is given by the weighted average of these effect sizes, where the weights are given by the inverse of the squared effect size standard error (Lipsey & Wilson, pp ). Analysis of impacts on completed coursework, high school graduation, and college enrollment will be reported in both percentage point differences as well as log-odds ratios, since these are typically reported as dichotomous outcomes. 17 Study impacts reported in 16 Programs with different funding sources are responding to very different needs of their primary stake-holders (Gandara, 2001). When programs receive funding from multiple sources, we follow Gandara (2001) in classifying them based on which sector started the program and/or maintains primary responsibility for overseeing day to day operations. A list of primary funding sources a number of large-scale programs is available in Gandara (2001, pp ). 17 Note that these calculations assume that the outcomes are reported as binary measures. It is possible that some of the outcomes will be reported as multi-level categorical variables. If any such outcome measures are encountered, we will develop standards for converting such categorical measures into binary measures during the piloting the data-coding instrument. The standards adopted would be specified in the final report. 18 The Campbell Collaboration

19 percentages will be converted to log-odds ratios, and the variance of each log-odds ratio will be calculated as described by Wang and Bushman (1999, pp ). The pooled impact estimate is given by the weighted average of these log-odds ratios or by the weighted average of the difference in natural units, where the weights are given by the inverse of the variance (Wang & Bushman, 1999, pp ). If a study reports regression adjusted impact estimates, these impact estimates and their standard errors will be used in the meta-analysis. No adjustments will be made to account for missing data. Presentation of results: The review will report findings in both natural units and in effect sizes or log-odds ratios (as described above). However, we also will present a forest plot for each college readiness domain and college enrollment separately. Each plot will include an impact estimate and 95% confidence interval for all studies included in the review for that outcome measure. At the bottom of each plot, the pooled impact estimate and its 95% confidence interval will be displayed. Findings will be discussed in the narrative portion of the final report. Subgroup analysis: In cases where there are a sufficient number of studies, we will conduct moderator analyses to test the appropriateness of pooling impact estimates over the range of college access programs. In addition, we potentially would also conduct exploratory analyses of the moderating effects of factors such as the following: (1) characteristics of the target population; (2) program funding source; (3) program components; and (4) program setting. Sensitivity analysis: If there are a sufficient number of studies for multiple study designs, we will investigate the relationship between estimated program impacts and study design characteristics such as the following: (1) how the treatment and control groups were formed; (2) how the outcomes were measured; and (3) whether any concerns regarding quality were raised. Treatment of qualitative research: Reviewers will maintain a full reference list of studies, identified through the search process, that present qualitative analysis of college access programs. However, no attempt will be made to synthesize the findings from that research. 19 The Campbell Collaboration

20 REFERENCES ACT. (2004). ACT national data release. Iowa City, IA: Author. Adelman, C. (1999). Answers in the toolbox: Academic intensity, attendance patterns, and bachelor s degree attainment. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement. Adelman, C. (2006). The toolbox revisited: Paths to degree completion from high school through college. Washington, DC: US Department of Education. America Diploma Project (2004). Ready or not: creating a high school diploma that counts. Retrieved on 15 March 2010 from Avery, C., & Hoxby, C. M. (2004). Do and should financial aid packages affect students college choices? In C. M. Hoxby (Ed.), College choices: The economics of where to go, when to go, and how to pay for it (pp ). Chicago, IL: University of Chicago Press. Becker, G. S. (1993). Human capital: A theoretical and empirical analysis with special reference to education (3 rd ed.). Chicago, IL: University of Chicago Press. Bergin, D., Cooks, H., & Bergin, C. (2007). Effects of a college access program for youth underrepresented in higher education: A randomized experiment. Research in Higher Education, 48(6), Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. (2005). Comprehensive Metaanalysis. Version 2. Englewood, NJ: Biostat. Bound, J., & Turner, S. (2007). Cohort crowding: How resources affect collegiate attainment. Journal of Public Economics, 91(5-6), Bourdieu, P. (1986). The forms of capital. In J. G. Richardson (Ed.), Handbook of theory and research for the sociology of education (pp ). New York, NY: Greenwood Press. Bozick, R., & Lauff, E. (2007). Education Longitudinal Study of 2002 (ELS:2002): A first look at the initial postsecondary experiences of the high school sophomore class of 2002 (NCES ). Washington, DC: US Department of Education, Institute of Education Sciences, National Center for Education Statistics. Bragg, D. D., Eunyoung, K., & Barnett, E. A. (2006). Creating access and success: Academic pathways reaching underserved students. New Directions for Community Colleges 2006, (135), Cabrera, A. F., & La Nasa, S. M. (2001). On the path to college: Three critical tasks facing america's disadvantaged. Research in Higher Education, 42(2), Callan, P. M., Finney, J. E., Kirst, M. W., Usdan, M. D., & Venezia, A. (2006). Claiming common ground: State policymaking for improving college readiness and success. San Jose, CA: The Institute for Educational Leadership, The National Center for Public Policy and Higher Education, and The Stanford Institute for Higher Education Research. 20 The Campbell Collaboration

Improving College Access for Minority, Low-Income, and First-Generation Students

Improving College Access for Minority, Low-Income, and First-Generation Students Improving College Access for Minority, Low-Income, and First-Generation Students Monica Martinez Shayna Klopott Institute for Educational Leadership and the National Clearinghouse for Comprehensive School

More information

Participation and pass rates for college preparatory transition courses in Kentucky

Participation and pass rates for college preparatory transition courses in Kentucky U.S. Department of Education March 2014 Participation and pass rates for college preparatory transition courses in Kentucky Christine Mokher CNA Key findings This study of Kentucky students who take college

More information

College Transition Programs: Promoting Success Beyond High School

College Transition Programs: Promoting Success Beyond High School College Transition Programs: Promoting Success Beyond High School I s s u e P a p e r s T h e H i g h S c h o o l L e a d e r s h i p S u m m i t Parents have higher educational aspirations for their children

More information

Highest Math Course Completed in High School

Highest Math Course Completed in High School 1 Highest Math Course Completed in High School Summary Highest math course completed in high school is associated with successful completion of a postsecondary credential. Controlling for socioeconomic

More information

The Condition of College & Career Readiness l 2011

The Condition of College & Career Readiness l 2011 The Condition of College & Career Readiness l 2011 ACT is an independent, not-for-profit organization that provides assessment, research, information, and program management services in the broad areas

More information

Smoothing the Way to College: Impact of Early College High Schools

Smoothing the Way to College: Impact of Early College High Schools Smoothing the Way to College: Impact of Early College High Schools SERVE Center The University of North Carolina at Greensboro To increase the number of students graduating from high school who enroll

More information

74% 68% 47% 54% 33% Staying on Target. ACT Research and Policy. The Importance of Monitoring Student Progress toward College and Career Readiness

74% 68% 47% 54% 33% Staying on Target. ACT Research and Policy. The Importance of Monitoring Student Progress toward College and Career Readiness Staying on Target ACT Research and Policy The Importance of Monitoring Student Progress toward Research Brief 74% Percentage of ACT-tested 11 high school graduates took a core curriculum in high school

More information

Assessing the relationship between school counseling and college enrollment

Assessing the relationship between school counseling and college enrollment Assessing the relationship between school counseling and college enrollment Belasco, A.S. (2013). Creating college opportunity: School counselors and their influence on postsecondary enrollment. Research

More information

Readiness Matters The Impact of College Readiness on College Persistence and Degree Completion

Readiness Matters The Impact of College Readiness on College Persistence and Degree Completion Readiness Matters The Impact of College Readiness on College Persistence and Degree Completion Steve Kappler AVP Career & College Readiness ACT, Inc. steve.kappler@act.org NACAC Webinar Series February

More information

October 2008 Research Brief: What does it take to prepare students academically for college?

October 2008 Research Brief: What does it take to prepare students academically for college? October 2008 Research Brief: What does it take to prepare students academically for college? The research is clear on the connection between high school coursework and success in college. The more academically

More information

WWC Single Study Review A review of the design and summary of findings for an individual study

WWC Single Study Review A review of the design and summary of findings for an individual study What Works Clearinghouse WWC Single Study Review A review of the design and summary of findings for an individual study U.S. DEPARTMENT OF EDUCATION July 2015 WWC Review of the Report Interactive Online

More information

MCPS Graduates Earning College Degrees in STEM-Related Fields

MCPS Graduates Earning College Degrees in STEM-Related Fields Graduates Earning College Degrees in -Related Fields September 2012 Natalie Wolanin & Julie Wade OFFICE OF SHARED ACCOUNTABILITY Adrian B. Talley, Associate Superintendent 850 Hungerford Drive Rockville,

More information

On the path to college: What matters?

On the path to college: What matters? On the path to college: What matters? Topics Pathways For college For transfer For degree completion What matters most What can be done What to assess & when to do it Alberto F. Cabrera Professor & WISCAPE

More information

The Importance of Community College Honors Programs

The Importance of Community College Honors Programs 6 This chapter examines relationships between the presence of honors programs at community colleges and institutional, curricular, and student body characteristics. Furthermore, the author relates his

More information

Dual-Credit/Dual-Enrollment Coursework and Long-Term College Success in Texas

Dual-Credit/Dual-Enrollment Coursework and Long-Term College Success in Texas Issue Brief October 2014 Dual-Credit/Dual-Enrollment Coursework and Long-Term College Success in Texas Justine Radunzel, Julie Noble, and Sue Wheeler This study was a cooperative effort of the Texas-ACT

More information

Evaluation of the College Bound Program: Early Findings

Evaluation of the College Bound Program: Early Findings Working Paper Evaluation of the College Bound Program: Early Findings Vi-Nhuan Le, Louis T. Mariano, Susannah Faxon-Mills RAND Education WR-971-COBND February 2013 Prepared for TG and the College Bound

More information

The Campbell Collaboration www.campbellcollaboration.org. Study content coding Effect size coding (next session)

The Campbell Collaboration www.campbellcollaboration.org. Study content coding Effect size coding (next session) Systema(c Review Methods Workshop Study Coding Sandra Jo Wilson Vanderbilt University Editor, Campbell Collabora9on Educa9on Coordina9ng Group Workshop Overview Levels of Study Coding Study eligibility

More information

Virginia s College and Career Readiness Initiative

Virginia s College and Career Readiness Initiative Virginia s College and Career Readiness Initiative In 1995, Virginia began a broad educational reform program that resulted in revised, rigorous content standards, the Virginia Standards of Learning (SOL),

More information

Abstract Title Page Not included in page count.

Abstract Title Page Not included in page count. Abstract Title Page Not included in page count. Title: The Impact of The Stock Market Game on Financial Literacy and Mathematics Achievement: Results from a National Randomized Controlled Trial. Author(s):

More information

This REL report is in the public domain. While permission to reprint this publication is not necessary, it should be cited as:

This REL report is in the public domain. While permission to reprint this publication is not necessary, it should be cited as: REL 2015 053 The National Center for Education Evaluation and Regional Assistance (NCEE) conducts unbiased large-scale evaluations of education programs and practices supported by federal funds; provides

More information

Evaluating the Impact of Remedial Education in Florida Community Colleges: A Quasi-Experimental Regression Discontinuity Design

Evaluating the Impact of Remedial Education in Florida Community Colleges: A Quasi-Experimental Regression Discontinuity Design NATIONAL CENTER FOR POSTSECONDARY RESEARCH NCPR BRIEF BRIEF AUGUST 2009 Evaluating the Impact of Remedial Education in Florida Community Colleges: A Quasi-Experimental Regression Discontinuity Design Juan

More information

C2 Training: June 8 9, 2010

C2 Training: June 8 9, 2010 C2 Training: June 8 9, 2010 Data Evaluation: Initial screening and Coding Adapted from David B. Wilson and Mark W. Lipsey The Campbell Collaboration www.campbellcollaboration.org Overview Coding protocol:

More information

Higher Performing High Schools

Higher Performing High Schools COLLEGE READINESS A First Look at Higher Performing High Schools School Qualities that Educators Believe Contribute Most to College and Career Readiness 2012 by ACT, Inc. All rights reserved. A First Look

More information

TRIO Programs. National Studies Find. Effective at Increasing College Enrollment and Graduation

TRIO Programs. National Studies Find. Effective at Increasing College Enrollment and Graduation Since 1997, four national studies conducted by the U.S. Department of Education have been completed on the Federal TRIO programs. Data from these studies demonstrate that participation in a TRIO program

More information

Bridging the Gap: Academic Preparation and Postsecondary Success of First-Generation Students Edward C. Warburton, Rosio Bugarin, and Anne-Marie Nuñez

Bridging the Gap: Academic Preparation and Postsecondary Success of First-Generation Students Edward C. Warburton, Rosio Bugarin, and Anne-Marie Nuñez Postsecondary Education Bridging the Gap: Academic Preparation and Postsecondary Success of First-Generation Students Edward C. Warburton, Rosio Bugarin, and Anne-Marie Nuñez This article was originally

More information

Career & College Clubs SM

Career & College Clubs SM Career & College Clubs SM EXPLORE + ENGAGE = ELEVATE Promising Results: Evaluating Effectiveness of Career & College Clubs SM June 2013 Contact: Career & College Clubs c/o ALL Management Corporation 6601

More information

issue brief September 2013

issue brief September 2013 issue brief September 2013 DEVELOPING ALTERNATIVE PLACEMENT CRITERIA FOR ENGLISH COURSES AT CITY COLLEGE OF SAN FRANCISCO Sebastian Castrechini Recognizing the need to improve postsecondary access and

More information

Abstract Title Page Not included in page count.

Abstract Title Page Not included in page count. Abstract Title Page Not included in page count. Do College-Ready Students Benefit When High Schools and Colleges Collaborate? Experimental Evidence from Albuquerque, New Mexico Benjamin L. Castleman University

More information

Health Policy and Administration PhD Track in Health Services and Policy Research

Health Policy and Administration PhD Track in Health Services and Policy Research Health Policy and Administration PhD Track in Health Services and Policy INTRODUCTION The Health Policy and Administration (HPA) Division of the UIC School of Public Health offers a PhD track in Health

More information

SC T Ju ly 2 0 0 0. Effects of Differential Prediction in College Admissions for Traditional and Nontraditional-Aged Students

SC T Ju ly 2 0 0 0. Effects of Differential Prediction in College Admissions for Traditional and Nontraditional-Aged Students A C T Rese& rclk R e p o rt S eries 2 0 0 0-9 Effects of Differential Prediction in College Admissions for Traditional and Nontraditional-Aged Students Julie Noble SC T Ju ly 2 0 0 0 For additional copies

More information

SDP COLLEGE-GOING DIAGNOSTIC. Albuquerque Public Schools

SDP COLLEGE-GOING DIAGNOSTIC. Albuquerque Public Schools Public Schools May 2014 CURRENT SDP PARTNERS THE STRATEGIC DATA PROJECT (SDP) Since 2008, SDP has partnered with 75 school districts, charter school networks, state agencies, and nonprofit organizations

More information

Bowen, Chingos & McPherson, Crossing the Finish Line

Bowen, Chingos & McPherson, Crossing the Finish Line 1 Bowen, W. G., Chingos, M. M., and McPherson, M. S. (2009). Crossing the Finish Line: Completing College at America s Public Universities. Princeton, N.J.: Princeton University Press. The authors begin

More information

Policy Brief College Access and Retention of Career and Technical Education Graduates

Policy Brief College Access and Retention of Career and Technical Education Graduates Prepared by Paul E. Harrington and Neeta P. Fogg, Center for Labor Market Studies, Northeastern University, and Christine Shaw, Shawsheen Valley Regional Technical High School, in collaboration with the

More information

Title: College enrollment patterns for rural Indiana high school graduates

Title: College enrollment patterns for rural Indiana high school graduates Abstract Title Page Title: College enrollment patterns for rural Indiana high school graduates Authors and Affiliations: Mathew R. Burke, Elizabeth Davis, and Jennifer L. Stephan American Institutes for

More information

CHARTER SCHOOL PERFORMANCE IN PENNSYLVANIA. credo.stanford.edu

CHARTER SCHOOL PERFORMANCE IN PENNSYLVANIA. credo.stanford.edu CHARTER SCHOOL PERFORMANCE IN PENNSYLVANIA credo.stanford.edu April 2011 TABLE OF CONTENTS INTRODUCTION... 3 DISTRIBUTION OF CHARTER SCHOOL PERFORMANCE IN PENNSYLVANIA... 7 CHARTER SCHOOL IMPACT BY DELIVERY

More information

LOUISIANA SCHOOLS PROFILE RE-IMAGINING TEACHING AND LEARNING

LOUISIANA SCHOOLS PROFILE RE-IMAGINING TEACHING AND LEARNING LOUISIANA SCHOOLS PROFILE RE-IMAGINING TEACHING AND LEARNING NEW TECH NETWORK New Tech Network is a non-profit school development organization dedicated to ensuring all students develop the skills and

More information

Reading Between the Lines:

Reading Between the Lines: COLLEGE READINESS Reading Between the Lines: What the ACT Reveals About College Readiness in Reading Executive Summary Based on 2005 ACT-tested high school graduates, it appears that only about half of

More information

A C T R esearcli R e p o rt S eries 2 0 0 5. Using ACT Assessment Scores to Set Benchmarks for College Readiness. IJeff Allen.

A C T R esearcli R e p o rt S eries 2 0 0 5. Using ACT Assessment Scores to Set Benchmarks for College Readiness. IJeff Allen. A C T R esearcli R e p o rt S eries 2 0 0 5 Using ACT Assessment Scores to Set Benchmarks for College Readiness IJeff Allen Jim Sconing ACT August 2005 For additional copies write: ACT Research Report

More information

The Condition of College & Career Readiness 2013. National

The Condition of College & Career Readiness 2013. National The Condition of College & Career Readiness 2013 National 2013 by ACT, Inc. All rights reserved. The ACT college readiness assessment is a registered trademark of ACT, Inc., in the U.S.A. and other countries.

More information

In the past two decades, the federal government has dramatically

In the past two decades, the federal government has dramatically Degree Attainment of Undergraduate Student Borrowers in Four-Year Institutions: A Multilevel Analysis By Dai Li Dai Li is a doctoral candidate in the Center for the Study of Higher Education at Pennsylvania

More information

National Evaluation of Student Support Services: Examination of Student Outcomes After Six Years

National Evaluation of Student Support Services: Examination of Student Outcomes After Six Years U.S. DEPARTMENT OF EDUCATION National Evaluation of Student Support Services: Examination of Student Outcomes After Six Years Final Report National Evaluation of Student Support Services: Examination of

More information

Comparative Study of the Persistence and Academic Success of Florida Community College Student-Athletes and Non-Athlete Students: 2004 to 2007

Comparative Study of the Persistence and Academic Success of Florida Community College Student-Athletes and Non-Athlete Students: 2004 to 2007 Comparative Study of the Persistence and Academic Success of Florida Community College Student-Athletes and Non-Athlete Students: 2004 to 2007 David Horton Jr., Ph.D. AIR Dissertation Fellow (2008-2009)

More information

Guidelines for Preparation of Review Protocols

Guidelines for Preparation of Review Protocols Guidelines for Preparation of Review Protocols Type of document: Policy _x_ Guideline Procedure Version: 1.0, January 1 2001 Decision: Steering Group, Date? A Campbell Systematic Review is meant to review

More information

THE EDUCATIONAL PROGRESS OF BLACK STUDENTS

THE EDUCATIONAL PROGRESS OF BLACK STUDENTS NATIONAL CENTER FOR EDUCATION STATISTICS Findings from THE CONDITION OF EDUCATION 1994 NO. 2 THE EDUCATIONAL PROGRESS OF BLACK STUDENTS U.S. Department of Education Office of Educational Research and Improvement

More information

The Reality of College Readiness 2013. Georgia

The Reality of College Readiness 2013. Georgia The Reality of College Readiness 2013 Georgia Annually, ACT provides each state with The Condition of College & Career Readiness, a report that details the college readiness of students who took the ACT

More information

A New Measure of Educational Success in Texas. Tracking the Success of 8th Graders into and through College

A New Measure of Educational Success in Texas. Tracking the Success of 8th Graders into and through College A New Measure of Educational Success in Texas Tracking the Success of 8th Graders into and through College National Center for Management Systems (NCHEMS) Contents Introduction 1 The Challenge 2 Purpose

More information

Relating the ACT Indicator Understanding Complex Texts to College Course Grades

Relating the ACT Indicator Understanding Complex Texts to College Course Grades ACT Research & Policy Technical Brief 2016 Relating the ACT Indicator Understanding Complex Texts to College Course Grades Jeff Allen, PhD; Brad Bolender; Yu Fang, PhD; Dongmei Li, PhD; and Tony Thompson,

More information

Math Placement Acceleration Initiative at the City College of San Francisco Developed with San Francisco Unified School District

Math Placement Acceleration Initiative at the City College of San Francisco Developed with San Francisco Unified School District Youth Data Archive Issue Brief October 2012 Math Placement Acceleration Initiative at the City College of San Francisco Developed with San Francisco Unified School District Betsy Williams Background This

More information

GUIDELINES FOR REVIEWING QUANTITATIVE DESCRIPTIVE STUDIES

GUIDELINES FOR REVIEWING QUANTITATIVE DESCRIPTIVE STUDIES GUIDELINES FOR REVIEWING QUANTITATIVE DESCRIPTIVE STUDIES These guidelines are intended to promote quality and consistency in CLEAR reviews of selected studies that use statistical techniques and other

More information

A Guide for Reporting on Evaluations for the US Department of Education Mathematics and Science Partnerships (MSP)

A Guide for Reporting on Evaluations for the US Department of Education Mathematics and Science Partnerships (MSP) A Guide for Reporting on Evaluations for the US Department of Education Mathematics and Science Partnerships (MSP) A User-Friendly Guide for MSP Project Officials and Evaluators Contract # ED-CFO-10-A-0024

More information

Developing the STEM Education Pipeline

Developing the STEM Education Pipeline Developing the STEM Education Pipeline Developing the STEM Education Pipeline For almost 50 years, ACT has played a pivotal role in promoting student access into and success in science, technology, engineering,

More information

The Path to College. January 2010. Greater Texas Foundation

The Path to College. January 2010. Greater Texas Foundation A Summary of Helping Students Navigate the Path to College: What High Schools Can Do (U.S. Department of Education, Institute of Education Sciences, 2009) January 2010 Greater Texas Foundation Acknowledgements

More information

Co-Curricular Activities and Academic Performance -A Study of the Student Leadership Initiative Programs. Office of Institutional Research

Co-Curricular Activities and Academic Performance -A Study of the Student Leadership Initiative Programs. Office of Institutional Research Co-Curricular Activities and Academic Performance -A Study of the Student Leadership Initiative Programs Office of Institutional Research July 2014 Introduction The Leadership Initiative (LI) is a certificate

More information

Are high achieving college students slackers? Brent J. Evans

Are high achieving college students slackers? Brent J. Evans Motivation & Research Question: Are high achieving college students slackers? Brent J. Evans There is a growing body of evidence that suggests college students are not academically challenged by or engaged

More information

Barriers to College Attainment

Barriers to College Attainment AGENCY/PHOTOGRAPHER Barriers to College Attainment Lessons from Chicago Jenny Nagaoka, Melissa Roderick, and Vanessa Coca, The Consortium on Chicago School Research at The University of Chicago January

More information

Strategies for Promoting Gatekeeper Course Success Among Students Needing Remediation: Research Report for the Virginia Community College System

Strategies for Promoting Gatekeeper Course Success Among Students Needing Remediation: Research Report for the Virginia Community College System Strategies for Promoting Gatekeeper Course Success Among Students Needing Remediation: Research Report for the Virginia Community College System Josipa Roksa Davis Jenkins Shanna Smith Jaggars Matthew

More information

Majors Matter: Differential Performance on a Test of General College Outcomes. Jeffrey T. Steedle Council for Aid to Education

Majors Matter: Differential Performance on a Test of General College Outcomes. Jeffrey T. Steedle Council for Aid to Education MAJORS MATTER 1 Majors Matter: Differential Performance on a Test of General College Outcomes Jeffrey T. Steedle Council for Aid to Education Michael Bradley New York University Please send correspondence

More information

TEXAS SCHOOLS PROFILE RE-IMAGINING TEACHING AND LEARNING

TEXAS SCHOOLS PROFILE RE-IMAGINING TEACHING AND LEARNING TEXAS SCHOOLS PROFILE RE-IMAGINING TEACHING AND LEARNING NEW TECH NETWORK A LEARNING COMMUNITY New Tech Network is a non-profit school development organization dedicated to ensuring all students develop

More information

RELATIONSHIP BETWEEN THE PATTERN OF MATHEMATICS AND SCIENCE COURSES TAKEN AND TEST SCORES ON ITED FOR HIGH SCHOOL JUNIORS

RELATIONSHIP BETWEEN THE PATTERN OF MATHEMATICS AND SCIENCE COURSES TAKEN AND TEST SCORES ON ITED FOR HIGH SCHOOL JUNIORS RELATIONSHIP BETWEEN THE PATTERN OF AND COURSES TAKEN AND TEST SCORES ON FOR HIGH SCHOOL JUNIORS Xiaoping Wang, Administrative Consultant Abstract Three years of student-level data were used in this study

More information

National College Progression Rates For high schools participating in the National Student Clearinghouse StudentTracker service

National College Progression Rates For high schools participating in the National Student Clearinghouse StudentTracker service National College Progression Rates For high schools participating in the National Student Clearinghouse StudentTracker service October 15, 2013 This report provides high school- to- college transition

More information

A + dvancer College Readiness Online Remedial Math Efficacy: A Body of Evidence

A + dvancer College Readiness Online Remedial Math Efficacy: A Body of Evidence Running Head: A+dvancer COLLEGE READINESS ONLINE 1 A + dvancer College Readiness Online Remedial Math Efficacy: A Body of Evidence John Vassiliou Miami Dade College, FL Deborah Anderson Front Range Community

More information

Nebraska School Counseling State Evaluation

Nebraska School Counseling State Evaluation Nebraska School Counseling State Evaluation John Carey and Karen Harrington Center for School Counseling Outcome Research Spring 2010 RESEARCH S c h o o l o f E d u c a t i o n U n i v e r s i t y o f

More information

July 2009 Research Brief: Factors that support academic success

July 2009 Research Brief: Factors that support academic success July 2009 Research Brief: Factors that support academic success The single most important factor determining a student s academic success is the academic preparation a student receives and the rigor of

More information

College Choice Report

College Choice Report A Profile of 0 ACT-tested high school graduates College Choice Report Part Preferences and Prospects ACT is an independent, nonprofit organization that provides assessment, research, information, and program

More information

What Works Clearinghouse

What Works Clearinghouse U.S. DEPARTMENT OF EDUCATION What Works Clearinghouse June 2014 WWC Review of the Report Interactive Learning Online at Public Universities: Evidence from a Six-Campus Randomized Trial 1 The findings from

More information

From High School to the Future: Potholes on the Road to College

From High School to the Future: Potholes on the Road to College Research Summary c c s r CONSORTIUM ON CHICAGO SCHOOL RESEARCH AT THE UNIVERSITY OF CHICAGO March 2008 Executive Summary From High School to the Future: Potholes on the Road to College Melissa Roderick,

More information

Student Academic Preparation and Educational Partnerships

Student Academic Preparation and Educational Partnerships Student Academic Preparation and Educational Partnerships (SAPEP) Annual Report Highlights 2013-14 OV E RV I E W The University of California s Student Academic Preparation and Educational Partnerships

More information

February 2003 Report No. 03-17

February 2003 Report No. 03-17 February 2003 Report No. 03-17 Bright Futures Contributes to Improved College Preparation, Affordability, and Enrollment at a glance Since the Bright Futures program was created in 1997, Florida s high

More information

Program Attendance in 41 Youth Smoking Cessation Programs in the U.S.

Program Attendance in 41 Youth Smoking Cessation Programs in the U.S. Program Attendance in 41 Youth Smoking Cessation Programs in the U.S. Zhiqun Tang, Robert Orwin, PhD, Kristie Taylor, PhD, Charles Carusi, PhD, Susan J. Curry, PhD, Sherry L. Emery, PhD, Amy K. Sporer,

More information

A Brief Research Summary on Access to College Level Coursework for High School Students. Provided to the Oregon Education Investment Board August 2014

A Brief Research Summary on Access to College Level Coursework for High School Students. Provided to the Oregon Education Investment Board August 2014 A Brief Research Summary on Access to College Level Coursework for High School Students Provided to the Oregon Education Investment Board August 2014 Prepared by Hilda Rosselli, OEIB College and Career

More information

ACT National Curriculum Survey 2012. Policy Implications on Preparing for Higher Standards. improve yourself

ACT National Curriculum Survey 2012. Policy Implications on Preparing for Higher Standards. improve yourself ACT National Curriculum Survey 2012 Policy Implications on Preparing for Higher Standards improve yourself ACT is an independent, not-for-profit organization that provides assessment, research, information,

More information

College Participation Rates of Maine s Recent High School Graduates: Examining the Claims

College Participation Rates of Maine s Recent High School Graduates: Examining the Claims College Participation Rates of Maine s Recent High School Graduates: Examining the Claims David L. Silvernail James E. Sloan Amy F. Johnson Center for Education Policy, Applied Research and Evaluation

More information

What We Know About Developmental Education Outcomes

What We Know About Developmental Education Outcomes RESEARCH OVERVIEW / JANUARY 2014 What We Know About Developmental Education Outcomes What Is Developmental Education? Many recent high school graduates who enter community college are required to take

More information

Increasing student retention through an enhanced mentoring and tutoring program. Abstract

Increasing student retention through an enhanced mentoring and tutoring program. Abstract Increasing student retention through an enhanced mentoring and tutoring program Hua Li, Kai Jin Industrial and Mechanical Engineering Department hua.li@tamuk.edu, kai.jin@tamuk.edu Mohamed Abdelrahman

More information

Working to My Potential: Experiences of CPS Students in the International Baccalaureate Diploma Programme

Working to My Potential: Experiences of CPS Students in the International Baccalaureate Diploma Programme RESEARCH SUMMARY Working to My Potential: Experiences of CPS Students in the International Baccalaureate Diploma Programme Based on a research report prepared for the IB by Chicago Postsecondary Transition

More information

The Alignment of Common Core and ACT s College and Career Readiness System. June 2010

The Alignment of Common Core and ACT s College and Career Readiness System. June 2010 The Alignment of Common Core and ACT s College and Career Readiness System June 2010 ACT is an independent, not-for-profit organization that provides assessment, research, information, and program management

More information

RESEARCH REPORT 15-3

RESEARCH REPORT 15-3 RESEARCH REPORT 15-3 August, 2015 Post Program Earnings Differences Between the Associate in Applied Science and Applied Baccalaureate Degrees: Companion Report to Applied Baccalaureate Degree: Policy

More information

Traversing the Academic Pipeline

Traversing the Academic Pipeline July 2008 Dr. Eric J. Smith, Commissioner Dr. Willis N. Holcombe, Chancellor Traversing the Academic Pipeline Edition 2008-02 INTRODUCTION Previous research published by the Division of Community Colleges

More information

Power Calculation Using the Online Variance Almanac (Web VA): A User s Guide

Power Calculation Using the Online Variance Almanac (Web VA): A User s Guide Power Calculation Using the Online Variance Almanac (Web VA): A User s Guide Larry V. Hedges & E.C. Hedberg This research was supported by the National Science Foundation under Award Nos. 0129365 and 0815295.

More information

Special Report on the Transfer Admission Process National Association for College Admission Counseling April 2010

Special Report on the Transfer Admission Process National Association for College Admission Counseling April 2010 Special Report on the Transfer Admission Process National Association for College Admission Counseling April 2010 Each Spring, much media attention is focused on the college admission process for first-year

More information

College Choice Report 2015. National

College Choice Report 2015. National College Report National ACT is an independent, nonprofit organization that provides assessment, research, information, and program management services in the broad areas of education and workforce development.

More information

NATIONAL CENTER FOR EDUCATION STATISTICS U.S. Department of Education Office of Educational Research and Improvement NCES 2001 126

NATIONAL CENTER FOR EDUCATION STATISTICS U.S. Department of Education Office of Educational Research and Improvement NCES 2001 126 Access, Persistence, and Attainment Page 1 NATIONAL CENTER FOR EDUCATION STATISTICS U.S. Department of Education Office of Educational Research and Improvement NCES 2001 126 U.S. Department of Education

More information

Florida s Guide to Public High School Graduation

Florida s Guide to Public High School Graduation Florida s Guide to Public High School Graduation It s a Major Opportunity! For Students Entering Ninth Grade in 2008-2009 Florida Department of Education 2008 1 High School Graduation Programs Overview

More information

Standard applies to undergraduate elementary, secondary and special education programs. Standard and Indicator...page 2

Standard applies to undergraduate elementary, secondary and special education programs. Standard and Indicator...page 2 Standard 19: Rigor The program holds teacher candidates to the same or a higher level of expectations regarding coursework and grading standards as that to which students in the rest of the institution

More information

National Partnership for Educational Access RESEARCH BRIEF

National Partnership for Educational Access RESEARCH BRIEF National Partnership for Educational Access RESEARCH BRIEF SUMMER NUDGING: CAN PERSONALIZED TEXT MESSAGES AND PEER MENTOR OUTREACH INCREASE COLLEGE GOING AMONG LOW-INCOME HIGH SCHOOL GRADUATES? Benjamin

More information

COMPARISON OF SEO SCHOLARS AND

COMPARISON OF SEO SCHOLARS AND POLICY STUDIES ASSOCIATES, INC. POLICY STUDIES ASSOCIATES, INC. POLICY STUDIES ASSOCIATES, INC. COMPARISON OF SEO SCHOLARS AND NATIONAL TEST-TAKERS SAT SCORES, 2009-2013 Tamar Hellman Stephen Coleman April

More information

ANNUAL REPORT ON CURRICULUM, INSTRUCTION AND STUDENT ACHIEVEMENT

ANNUAL REPORT ON CURRICULUM, INSTRUCTION AND STUDENT ACHIEVEMENT The mission of the Greenbush-Middle River School is to provide a quality education which will prepare all students to meet success by developing self-directed thinkers and communicators who are responsible,

More information

THE EDUCATIONAL PROGRESS OF WOMEN

THE EDUCATIONAL PROGRESS OF WOMEN NATIONAL CENTER FOR EDUCATION STATISTICS Findings from THE CONDITION OF EDUCATION 1995 NO. 5 THE EDUCATIONAL PROGRESS OF WOMEN U.S. Department of Education Office of Educational Research and Improvement

More information

Low-SES Students and College Outcomes: The Role of AP Fee Reductions

Low-SES Students and College Outcomes: The Role of AP Fee Reductions RESEARCH REPORT 2011-9 Low-SES Students and College Outcomes: The Role of AP Fee Reductions By Jeffrey N. Wyatt and Krista D. Mattern Jeffrey N. Wyatt is an assistant research scientist at the College

More information

Organizing Your Approach to a Data Analysis

Organizing Your Approach to a Data Analysis Biost/Stat 578 B: Data Analysis Emerson, September 29, 2003 Handout #1 Organizing Your Approach to a Data Analysis The general theme should be to maximize thinking about the data analysis and to minimize

More information

Community College as a Pathway to Higher Education and Earnings

Community College as a Pathway to Higher Education and Earnings Community College as a Pathway to Higher Education and Earnings Thomas Brock MDRC Community College as a Pathway to Higher Education and Earnings According to the most recent data from the U.S. Department

More information

Academic Performance of IB Students Entering the University of California System from 2000 2002

Academic Performance of IB Students Entering the University of California System from 2000 2002 RESEARCH SUMMARY Academic Performance of IB Students Entering the University of California System from 2000 2002 IB Global Policy & Research Department August 2010 Abstract This report documents the college

More information

10 Ideas to Ensure College Readiness in the No Child Left Behind & Higher Education Acts

10 Ideas to Ensure College Readiness in the No Child Left Behind & Higher Education Acts New America Foundation Issue Brief 10 Ideas to Ensure College Readiness in the No Child Left Behind & Higher Education Acts MaryEllen McGuire, Federal Education Budget Project April 2009 It is a stark,

More information

California Lutheran University Information Literacy Curriculum Map Graduate Psychology Department

California Lutheran University Information Literacy Curriculum Map Graduate Psychology Department California Lutheran University Information Literacy Curriculum Map Graduate Psychology Department Student Learning Outcomes Learning Outcome 1 LO1 literate student determines the nature and extent of the

More information

Abstract Title Page Not included in page count.

Abstract Title Page Not included in page count. Abstract Title Page Not included in page count. Title: Measuring Student Success from a Developmental Mathematics Course at an Elite Public Institution Authors and Affiliations: Julian Hsu, University

More information

The College Application Gauntlet: An Systematic Analysis of College Application Step Completion

The College Application Gauntlet: An Systematic Analysis of College Application Step Completion The College Application Gauntlet: An Systematic Analysis of College Application Step Completion Daniel Klasik Stanford University Please address all correspondence to: Daniel Klasik Stanford University

More information

Participation in Postsecondary Remedial Education

Participation in Postsecondary Remedial Education 1 Participation in Postsecondary Remedial Education Summary Participation in postsecondary remedial education in the first year of college is a key measure demonstrating students readiness to do college-level

More information

BRIEF. Dual Enrollment Students in Florida and New York City: Postsecondary Outcomes. Dual Enrollment

BRIEF. Dual Enrollment Students in Florida and New York City: Postsecondary Outcomes. Dual Enrollment ISSN 1526-2049 C O M M U N I T Y C O L L E G E R E S E A R C H C E N T E R BRIEF NUMBER 37 FEBRUARY 2008 Students in Florida and New York City: Postsecondary Outcomes Melinda Mechur Karp, Juan Carlos Calcagno,

More information