Developmental Education in New Mexico

Developmental Education in New Mexico Report by the New Mexico Developmental Education Task Force 2009 Inquiries should be directed to: New Mexico Higher Education Department P-20 Division 2048 Galisteo Santa Fe, NM 87505-2100 505.476.8422

TASK FORCE MEMBERS Dr. Chandler Barrett (Task Force Co-Chair) P-20 Policy Analyst New Mexico Higher Education Department Best Practices Workgroup Dr. Sandra Tracy (Chair) Dean, Extended Learning San Juan College Ms. Mona Lee Armstrong Director, Adult Basic Education Clovis Community College Ms. Rose Auletta Program Director Transitional Studies UNM Valencia Ms. Liz Bahe Expanding the Circle Director Institute of American Indian Arts Mr. Thomas Bennett Tribal & Community Based Programs Dine College Mr. Mike Costello Registrar - Director of Admissions & Records Northern New Mexico College Dr. Joan Erben NMSU Faculty New Mexico State University Grants Dr. Pam Etre-Perez State Director, Adult Basic Education New Mexico Higher Education Department Dr. Ann Filmyr Dean of the Center for Arts & Cultural Studies Institute of American Indian Arts Dr. Reinaldo Garcia Dean of Instruction University of New Mexico Valencia Dr. Peter Gerity Vice President for Academic Affairs New Mexico Tech Janet Griffiths Faculty, Pre-Collegiate Mesalands Community College Ms. Elaine DeBrine Howell Associate Dean of the Center for Student Success New Mexico Tech Dr. Frank Renz (Task Force Co-Chair) Consultant Ms. Sara Grijalva Office of the Registrar New Mexico Tech Ms. Lucy Gurrola Professor, Math and Physical Science Department Dona Ana Community College Ms. Judy Hofer Director, Adult Basic Education The University of Mexico Taos Dr. James Jasmine Reading and English Instructor Eastern New Mexico University Roswell Ms. Shuli Lamden Faculty, Developmental Studies Department Santa Fe Community College Dr. Nick Macaluso English Instructor Eastern New Mexico University Roswell Dr. Ali Mageehon English Instructor New Mexico State University Alamogordo Dr. Matthew Martinez Director of Indian Education New Mexico Higher Education Department Dr. Kate Massengale Dean of Instruction The University of Mexico Los Alamos Dr. Bernie Pina Dean, Math and Physical Science Department Dona Ana Community College Dr. Becky Rowley Executive Vice President Clovis Community College Ms. Amy Schwebke Assistant Professor, English & Communications Dona Ana Community College Ms. Linda Vozar Sweet College Success Instructor and Department Chair Eastern New Mexico University

College Readiness Workgroup Ms. Tyler Weldon (Chair) Director of Planning & Research New Mexico Higher Education Department Ms. Linda Beattie Professor of Mathematics Western New Mexico University Mr. Mickey D. Best Dean, Arts & Humanities New Mexico Junior College Ms. Joan Brown Mathematics Instructor Eastern New Mexico University Ms. Bernadette Chavira-Merriman Department Chair, Developmental Studies Northern New Mexico College Dr. Helen Cogan Dean of Liberal Arts Eastern New Mexico University Roswell Dr. John Gratton Vice President for Instruction New Mexico Junior College Mr. Kirby Gchachu NSF Visions 21 Program Coordinator Southwestern Indian Polytechnic Institute Mr. R. Richard Kestner II Institutional Research New Mexico Military Institute Dr. Ron Liss Vice President of Academic and Student Affairs Santa Fe Community College Ms. Colleen Lynch Faculty, Developmental Studies Santa Fe Community College Ms. Valerie Montoya Vice President for Academic Programs Southwestern Indian Polytechnic Institute Mr. Bruce Petrie Vice President of Academic Affairs New Mexico State University - Carlsbad Mr. Jim Sayers Associate Professor, Transitional Studies The University of New Mexico - Gallup Dr. Anthony Sena Provost Northern New Mexico College Ms. Debra Teachman VP for Academic Affairs New Mexico State University - Alamogordo Ms. Lynette Wedig Mathematics Instructor New Mexico State University - Alamogordo Measurement & Evaluation Workgroup Dr. Mary Baumhover Director, Writing Center Western New Mexico University Ms. Pamela Brody Developmental Education Coordinator The University of New Mexico Taos Ms. Elisha Melendrez Garcia Academic Affairs Western New Mexico University Ms. Cherryl Kilness Dean of Business & Science Eastern New Mexico University Roswell Ms. Erin Lebacqz Assoc. Dean, School of Adult & General Education Central New Mexico Community College Mr. Roy Lujan Dean, Humanities & Fine Arts New Mexico Highlands University Ms. Lou Anne Lundgren Assoc. Dean, School of Adult & General Education Central New Mexico Community College Dr. Gilbert Rivera Vice President for Academic AffairsNew Mexico Highlands University Dr. Faye Vowell Provost, VP for Academic Affairs Western New Mexico University Ms. Janet Wallet-Ortiz Associate Professor of English Western New Mexico University

TABLE OF CONTENTS THE CURRENT STATE OF DEVELOPMENTAL EDUCATION... 1 HISTORY OF THE NEW MEXICO DEVELOPMENTAL EDUCATION TASK FORCE... 3 Best Practices... 3 College Readiness... 3 Measurement & Evaluation... 5 COLLEGE READINESS & DEVELOPMENTAL EDUCATION: DEFINITIONS... 5 WHO ARE OUR UNDERPREPARED STUDENTS?... 6 BEST PRACTICES IN DEVELOPMENTAL EDUCATION SURVEY RESULTS... 7 Organizational and Administrative Practices: Institutional Commitment... 8 Organizational and Administrative Practices: Integration... 9 Organizational and Administrative Practices: Approach... 9 Organizational and Administrative Practices: Acceleration... 11 Program Components: Assessment... 12 Program Components: Evaluation... 13 Program Components: Support... 13 Program Components: Curriculum & Instruction... 15 Program Components: Outreach... 16 Instructional Practices... 16 Instruction... 19 Adult Basic Education... 21 Professional Development... 21 CONCLUSIONS & RECOMMENDATIONS... 23 REFERENCES... 31

THE CURRENT STATE OF DEVELOPMENTAL EDUCATION Community colleges and undergraduate programs at many four-year institutions play a vital role in providing access to higher education degree programs for the residents of New Mexico. Students enrolling in their first year of college often need to take remedial education courses to recoup their core education skills for success in degree programs. New Mexico s Ready for College report shows that 50.1% of New Mexico s recent high school graduates took college remedial courses in numeracy and/or literacy during their first semester in college (Ready for College 2008: An annual report on New Mexico high school graduates who take remedial classes in New Mexico colleges and Universities, an NMHED/NMDFA: Office of Educational Accountability. Available at http://education.nmdfa.state.nm.us/cms/ku nde/rts/educationnmdfastatenmus/docs/236 164529-06-24-2009-12-11-09.pdf). This figure is high and yet does not include the many students who delay involvement in developmental education, and when viewed over a six-year period, 83% of high school graduates age 22 or younger enrolling in freshman year of college in New Mexico also enroll in at least one remedial course in numeracy and/or literacy while pursuing a degree (NMHED Planning and Research Division, 2009). Few students (about 30%) referred to developmental education actually complete their 50.1% of New Mexico s recent high school graduates took college remedial courses in numeracy and/or literacy during their first semester in college. Over 80% of recent high school graduates enrolling in New Mexico s colleges also enroll in at least one remedial course while pursuing a degree. developmental courses, and many students who are referred to developmental education never actually enroll in a course. From the Colonial period to the present, Ivy League universities and other colleges have made remedial coursework a regular part of the curriculum (cf. Breneman & Harlow, 1998; Ignash, 1997; Payne & Lyman, 1996). Yet, only recently have innovative best practices in postsecondary remediation become recognized as imperative to student success. The identification and diffusion of innovation in developmental education has been a primary goal of Achieving the Dream, Jobs for the Future, the Lumina Foundation, and more recently, the Bill & Melinda Gates Foundation. Programs across the county are beginning to show improvements through the implementation of best practices as accelerated remedial programming, centralized or highly coordinated remedial programs, learning communities, institution-wide commitment to remediation, and others. Remedial education at the postsecondary level has long been an undervalued and underfunded mission of higher education. According to the Carnegie Foundation for the Advancement of Teaching: At the institutional level, developmental programs are often treated as the poor cousin of transfer-level courses. Historically, the transfer function has 1

been the prestige mission for community colleges (Grubb and Associates, 1999, p. 5), and many campuses have not made the commitments or invested the resources required to make pre-collegiate education more than a revolving door. Nor, in fact, do they want to be seen as the place for remediation. In short, basic skills education on most campuses has been shunted off to the margins, staffed largely by part-timers, and underfunded. Expectations have been way too low. (The Carnegie Foundation for the Advancement of Teaching (2008) Basic Skills for Complex Lives, p. 6) Improvement of basic education skills is also crucial to workforce development. As Alexander Astin (2000) has said, effective remedial education would do more to alleviate our most serious social and economic problems than almost any other action we could take. Remediation, as undervalued as it has been, is not an appendage with little connection to the mission of the institution but represents a core function of the higher education community that it has performed for hundreds of years" (Merisotis & Phipps, 2000, p. 79). We are now realizing that developmental education represents those core education skills that build a capacity for success in all other disciplines and that improvement in our developmental programs is imperative if we are to increase student success. Because too many students are not learning the basic skills needed to succeed in college or work while they are in high school, New Mexico loses about $32 million a year. This Developmental education represents those core education skills that build a capacity for success in all other disciplines. 2 figure includes $9.8 million to provide remedial education to students who have recently completed high school. In addition, this figure factors in over $22 million that the economy loses because remedial reading students are more likely to drop out of college without a degree, thereby reducing their earning potential (Alliance for Excellent Education, Issue Brief, August 2006, Paying Double). Certainly, some amount of additional funding will likely be required to improve remedial education at the postsecondary level; however, any initial investment will largely be offset by savings in costs now being borne by New Mexicans. Cost effectiveness studies focused on the productivity of innovations in remedial education programs have shown that the costs of implementing research-based best practices reach a break even point and are often exceeded by the revenues generated from increased student retention in the program (Corash, K. & Baker, E.D., 2009; Levin & McEwan, 2001). Further, support services already in place might be engaged in more innovative ways to address the problem more effectively. Successful best practices for improving developmental education programs have recently emerged in many of our state s colleges, and with the formation of the New Mexico Developmental Education Task Force, the state has taken up the challenge of improving remedial education for its students. This report is available from the New Mexico Higher Education Department. P-20 Division.

HISTORY OF THE NEW MEXICO DEVELOPMENTAL EDUCATION TASK FORCE NMHED and New Mexico institutions of higher education formed a Developmental Education Task Force in March 2009 in an initial effort to improve programs throughout the state. Twentyseven of the twenty-eight colleges and universities with developmental education programs attended the initial meeting. The task force was formed for the following purposes: To encourage the development of innovations in developmental education and promoting the spread of best practices statewide; To establish a statewide learning network and local communities of practice for the dissemination of best practices in developmental education, and To ensure that developmental education is a major focus and an adequately funded activity of New Mexico s colleges. In order to realize these purposes, three workgroups were formed to address issues in the areas of best practices, college readiness, and measurement and evaluation in developmental education. Best Practices The Best Practices Workgroup has completed an environmental scan of best practices in developmental education nationwide and throughout the state. The group has designed and administered a survey on developmental education programs in the state and has identified current best practices and related challenges at New Mexico higher education institutions. Results from the survey are presented later in this document. "We need to recapture the spirit of innovation that has always moved America forward," President Barack Obama College Readiness The College Readiness Workgroup is focusing on the alignment of developmental education and entry-level postsecondary coursework competencies and on recommending common cutoff scores for college placement in New Mexico. The group has reviewed the alignment of secondary and postsecondary English and math competencies as well as placement assessment policies and cutoff scores in use across the state. Unlike many states, New Mexico does not have common placement cutoff scores for determining college readiness. As a consequence, a student can score into college-level courses at one college, while at another institution that same score would place the student in remedial coursework. Cutoff scores currently in use at New Mexico higher education institutions are widely divergent, even within the same institution among branch campuses. Achieve, Inc. research has shown that students are better able to prepare for college when a state has consistent and common placement scores, and in working with Achieve s Alignment Institute, New Mexico has been urged to adopt standard placement cutoff scores. The task force, in collaboration with ACT, Inc., the College Board, and higher education institutions throughout the state, is recommending that common placement cutoff scores be adopted at New Mexico s postsecondary institutions. Higher education institutions have a crucial interest in this issue. While there is agreement with the need to set scores that would place 3

students on a level playing field in terms of access to higher education degree programs, there are issues surrounding the setting of a state standard placement cutoff score for entry into degree program coursework, such as: Would setting a standard cut score result in more students being placed in remedial classes? Will differences in local culture result in significant variance in student access to degree programs work if a uniform cutoff score is set? Even if common cut scores are set, will outcomes be different for students without also aligning developmental education competencies and courses? In addition, the establishment of a standard placement cutoff score would help increase student success by: Setting clear college-level performance expectations that could be communicated to aspiring students; Increasing the accuracy with which students are placed in postsecondary courses appropriate to their skill level; Ensuring consistent placement standards from institution to institution, and Providing comparable and timely data on student outcomes that can be used in evaluating and improving current educational policies and practices. (Collins, M. L, 2008) Since 2005, the number of states requiring standardized cut scores for placement into developmental education has more than tripled (Collins, M. L, 2008). Further, of the 46 states that provided information in 2008: 27 states (up from 21 states in 2002) reported that a state-level policy was in place requiring community colleges to assess students needs for developmental education at the time of enrollment, 21 states (up from 11 states in 2002) specified one or more approved placement exams for colleges to use. COMPASS was the most frequently approved exam (14 states), followed by ACCUPLACER (11 states) and ASSET (7 states), and 19 states (up from 5 states in 2002) required colleges to use standardized cut scores or ranges on these exams for placement into developmental education. (Collins, M. L, 2008). NMHED and the New Mexico Developmental Education Task Force are recommending the establishment of a statewide standard placement cutoff score. In addition to the placement validation studies now completed by ACT and the College Board, the task force recommends a one year period to evaluate the implementation of standard scores for English and math suggested by the studies analyzing student record data submitted by colleges to determine actual student performance relative to the suggested cutoff scores. Finally, an overall goal of this initiative is to identify as many students as possible who are likely to succeed in degree program coursework and to ensure their success by providing the supports they need in order to succeed. 4

Measurement & Evaluation The Measurement and Evaluation Workgroup has reviewed existing developmental education measures from the state and national landscapes along with the NMHED Strategic Plan developmental education objectives, strategic activities and performance measures and is making recommendations for the effective measurement of developmental education outcomes. The group is looking at three groups of students: recent high school graduates, recent GED recipients, and other adults. Both full-time and part-time students will be included in any analysis, since part-time students make up a significant portion of students in remedial programs. The group suggests restricting analysis to first-time students. Further, students in the lowest levels of developmental education coursework would be considered as a separate cohort because of the low rates at which such students complete their entire developmental education sequence. The establishment of statewide performance measures and benchmarks will illuminate patterns of student progress and achievement. Crucial to this effort, of course, is the establishment of a P- 20 longitudinal database allowing use of longitudinal student unit record data. With such a system in place, New Mexico would have increased capacity to identify different student groups among first-time postsecondary student cohorts, to monitor student progress and achievement, and identify barriers to success. Further, such a system can be used to identify college practices and processes associated with successful outcomes and to inform the development of approaches that address barriers to achievement. Finally, through tracking student performance over time, higher education institutions can also measure their progress in promoting student success. COLLEGE READINESS & DEVELOPMENTAL EDUCATION: DEFINITIONS College readiness is defined here as the level of preparation a student needs in order to succeed without remediation in first-year, entry-level, degree program coursework at a postsecondary institution. A common placement cutoff score will serve as an indication of college readiness. The National Association for Developmental Education (NADE) has defined developmental education as follows: Developmental education is a field of practice and research within higher education with a theoretical foundation in developmental psychology and learning theory. It promotes the cognitive and affective growth of all postsecondary learners, at all levels of the learning continuum. Developmental education is sensitive and responsive to the individual differences and special needs among learners. Developmental education programs and services commonly address academic preparedness, diagnostic assessment and placement, development of general and discipline-specific learning strategies, and affective barriers to learning. Developmental education courses include remedial courses as well as student success and career readiness and other courses. A remedial course is defined as follows: Remedial course refers to courses with CIP codes of 32.0104 or 32.0108 that fall within the numeracy and computational skills, precollegiate mathematics skills, precollegiate 5

reading skills, precollegiate writing skills, or communication skills categories. [6.30.7.7 NMAC Rp, 6.30.7.7 NMAC, 6/30/08] For the purposes of this study data has been limited to remedial courses. WHO ARE OUR UNDERPREPARED STUDENTS? The cohort of students represented in this study are those who enrolled for the Fall 2002 semester in New Mexico institutions of higher education that operate developmental education programs, who were seeking a degree, and who were attending a college in New Mexico for the first time. Performance of the cohort was followed for a six-year period (through Spring 2008). Due to data collection restraints, the cohort does not include students from private or tribal higher education institutions in New Mexico. Of the 8,979 cohort of students enrolling, 75% (6,726 students) enrolled in at least one developmental education class during the six-year period. The younger students proved to be most underprepared. About 80% of all students in the cohort under the age of 18 enrolled in remedial classes, and 88% of those between the ages of 19 and 24 took remedial classes. As the following chart shows, the percentage of those needing remediation drops dramatically after the age of 25. 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 88% 80% 59% 37% 16-18 19-24 25-44 45+ About 54% of the cohort were female, and about 60% of those taking developmental education classes were female. Fifty-three percent (53%) of the students in the cohort initially enrolled as full-time students with 47% enrolled as part time students. Thirty-nine percent (39%) of the cohort students were low-income (Pell eligible). 36% enrolled only in remedial math, 13% enrolled only in remedial English, and 51% enrolled in both remedial math and English. Thirty-three percent (2,870 students) of the total student cohort attained a credential or transferred during the six-year period. Of these, a full 27% were students that had taken at least 6

one remedial class. By comparison, the graduation rate of New Mexico s universities averages about 35% (DFA submittals, October 2008). Of all college-ready students, 27% attained a credential during the six-year period, and of those students who enrolled in at least one remedial class, 35% attained a credential during the same period. Another 8% of the cohort had earned 30 credit hours or more and were still enrolled. About 28% of the students enrolled in only one remedial class, with 70% of the students attempting four or fewer DE classes. Twenty-six percent (26%) enrolled in five to nine classes, and 5% enrolled in ten to twenty-six remedial classes. The following graph shows the number of remedial classes enrolled in by students in the cohort study. Of those students involved in five or more remedial classes, only 2% (203 students) had attained an award after six years. BEST PRACTICES IN DEVELOPMENTAL EDUCATION SURVEY RESULTS Survey questions were developed by members of the Best Practices Workgroup and reflect research on best practices in remedial education. The following survey results have been obtained through the participation of 25 higher education institutions in New Mexico and include responses from 109 individuals. Key participants at each institution were asked to distribute the survey to others in the college with an influential role in the developmental education program. Respondents include faculty, staff and administrators at all levels of our higher education institutions. Summary responses, as percentages, for those who either agreed or strongly agreed, or yes in regards to yes/no responses, are presented within several categories below. When available, quotes from related research follow responses. 7

Organizational and Administrative Practices: Institutional Commitment 1. Developmental education is a clearly stated institutional priority in our strategic plan. (67%) Gabriner, et. al. (2007); Attewell, Levin, Domina, & Levey, (2006); Perin (2006); Roueche and Roueche (1993, 1999), and Roueche and Baker (1987) have argued that an institutionwide commitment to remedial education was a key factor in the success of community college remediation. An institution-wide commitment to remediation was reflected through public administrative support for remediation, appropriate allocation of resources for remediation, and institutional acceptance of remediation as a mainstream activity for the community college. In their study of Texas colleges and universities, Boylan and Saxon (1998) found that remedial programs integrated into the academic mainstream of the institution had higher pass rates in remedial courses and were more successful in retaining students than programs that were not thusly integrated. 2. Our developmental program/initiative has clearly defined its mission, goals, and measurable objectives. (64%) Clear goals and objectives. Of particular note were findings regarding the importance of establishing clear cut goals and objectives for remedial courses Gabriner, et. al. (2007); Roueche, (1968) and Roueche (1973). Later studies by Donovan (1974), Cross (1976), Kulik and Kulik (1991), and Boylan, Bonham, Claxton, and Bliss (1992) also found that remedial instruction based on carefully defined goals and objectives was associated with improved student performance. Apparently, understanding the goals of a particular course and knowing exactly what the instructor expects students to accomplish facilitated the learning of underprepared students. Furthermore, studies showed that the specification of course goals and objectives also facilitated the establishment of a clear course structure, another component of successful remediation. Program definition. Early studies of remediation also argued that successful programs should be guided by a clearly defined philosophy accompanied by clearly specified goals and objectives (Roueche & Snow, 1977). Later work by Casazza and Silverman (1996), Maxwell (1997), and Boylan and Saxon (1998) also reinforced this finding. The presence of an underlying program philosophy accompanied by program goals and objectives based on this philosophy appeared to characterize successful programs. This finding has, in fact, been incorporated into recent professional association guidelines for program certification. Certification guidelines established by the National Association for Developmental Education (Clark-Thayer, 1995) require that programs seeking certification specify their operational philosophy and describe the program goals and objectives based on this philosophy as part of the requirements for obtaining certification. 3. Developmental Education is adequately funded each year at our institution. (53%) Studies suggest that having adequate funds for the provision and rigorous evaluation of developmental education can promote better student outcomes. Making financial aid available to students while they are taking developmental education without reducing subsequent aid for credit-bearing courses and providing the aid early so that students 8

can concentrate on their studies and get a foothold on higher education, are two mechanisms that have been recommended as an effective use of aid dollars. A number of studies also suggest providing funds for child care, transportation, and other personal needs to facilitate regular college attendance (Gabriner, et. al, 2007; Goldrick-Rab, 2007). Organizational and Administrative Practices: Integration 4. The developmental education program/initiative employs a wide range of instructional approaches, e.g. small groups, computer-assisted instruction, individualized instruction, tutoring. (90%) 5. Developmental education faculty and staff provide access to a wide range of support services, e.g. career assessment, advising, child care, financial aid. (70%) The literature emphasizes the importance of colleges offering services to support developmental students in their learning. While each support can be provided individually, taking a holistic approach by implementing a variety of services to meet the diverse educational and personal needs of students seems to be most effective. In fact, experience suggests that student persistence increases with the number and extent of coordination of the services offered their availability, and their responsiveness to personal needs and schedules (Gabriner et al., 2007; Jenkins, 2006; Boylan, 2002; Roueche & Roueche, 1999; Muraskin, 1998). 6. Please indicate the extent to which developmental education student may participate in other college courses. Students can co-enroll in any course related to the majors and professions to which they aspire respecting prerequisite requirements. (86%) Students can co-enroll in any non-transferable degree-program course. (8%) Students cannot enroll in any courses other than the developmental education courses in which they have been placed. (6%) Organizational and Administrative Practices: Approach 7. Centralized vs. Mainstreamed (Decentralized/Collaborative) Programs Research on best practices in remedial education indicates that students in centralized and/or strongly collaborative programs are more likely to succeed in their remedial classes and are likely to be retained for longer periods of time than students in decentralized programs (Calcagno, 2007; Gabriner, et. al, 2007; Goldrick-Rab, 2007; Bailey & Alfonso, 2005; Boylan, 2002; Roueche & Roueche, 1999; Roueche & Kirk, 1974; Roueche & Snow, 1977; Donovan (1974), and Boylan, Bonham, Claxton, and Bliss (1992). Further, more recent research indicates that highly collaborative programs are equally as effective: 9

Responses: Highly Centralized - Little or no integration with academic programs 8% Centralized 32% Mainstreamed 40% Highly mainstreamed - Fully integrated in academic programs 20% Roueche and his colleagues have long advocated that remedial courses and services should be provided by a separate and centralized program as opposed to individual academic departments (Roueche & Kirk, 1974; Roueche & Snow, 1977). This finding was later validated by Donovan (1974), and Boylan, Bonham, Claxton, and Bliss (1992). Students participating in centralized remedial programs were found to be more likely to pass their remedial courses and more likely to be retained for longer periods of time than students participating in decentralized programs. Recent analysis of these findings, however, suggests that it is not a centralized program structure alone that contributes to success but the coordination and communication afforded by such a structure (Boylan, Bliss, & Bonham, 1997). Obviously, coordination of effort and communication among faculty and staff providing remediation occurs most easily in a centralized program. However, decentralized programs in which there is strong coordination of remedial education activities and abundant communication among those who teach remedial courses may be just as effective as centralized programs (Boylan, Bliss, & Bonham, 1997). 8. Our developmental education program/initiative is characterized by coordination of effort and communication among faculty and staff providing remediation. (71%) There has been little research on institutional effectiveness at community colleges. A key challenge is how to compare colleges that serve students with widely varying characteristics. Using transcript-level data on over 150,000 Florida community college students, we estimated the effect on the graduation, transfer, and persistence rates of minority students at each of the 28 Florida community colleges as a proxy for institutional effectiveness. We ranked the colleges based on these estimated effects and selected six colleges for field research three that had a high impact on minority student success and three that had a low impact. Our fieldwork focused on a set of seven elements of institutional policy, practice, and culture that we hypothesized are important for promoting student success. We found that the high-impact colleges were more likely than the low-impact colleges to coordinate their programs and services to support student success. We also found that minority students were generally more successful in colleges that had support services targeted specifically to their needs. The study suggests that a more important factor for institutional effectiveness than adopting particular policies or practices is how well a college manages and aligns its programs and services to support student success (Jenkins, 2007). Developmental education faculty can feel marginalized from the rest of the college. Thus, meeting with each other and with the college s total faculty can provide an opportunity 10

for them to share information, insights, and concerns. Colleges can also promote the sharing of syllabi to better align courses. Some colleges use email and listserves to promote the exchange of ideas and information and for discussing and debating strategies. Colleges that provide opportunities for faculty to meet in groups by discipline, for example to discuss content and pedagogy matters, and to report on conferences and seminars attended, can maximize the coordination of developmental education and other courses. College-wide meetings where developmental education is discussed can promote the mainstreaming of the program (Gabriner et al., 2007; Boylan, 2002). 9. Support services for developmental education students are comprehensive, individualized, and proactively offered. (71%) See question 5 above. 10. Students are kept aware of their progress as they move through their required sequence of courses. (79%) Organizational and Administrative Practices: Acceleration 11. Institutional processes facilitate student completion of necessary developmental coursework as early as possible in the educational sequence. (81%) Two alternative strategies for moving students through developmental education are prevalent. Both are aimed at fostering retention, and both are based on the premise that students should not take college-level English or math courses before passing a developmental course in that subject if one is necessary (Calcagno, 2007; Gabriner et al., 2007; Goldrick-Rab, 2007; Roueche & Roueche, 1999). The first strategy requires students to take developmental courses during their first semester and to complete all developmental courses before taking any credit-bearing academic or career courses. The rationale is that such sequencing moves students more quickly to a course load containing only degree-granting courses and gives them an educational foundation for achievement in higher level courses. The second strategy calls for students to take developmental and college-level courses (but not English or math) at the same time in the belief that mixing courses will maintain their interest and give them an immediate sense that they are moving toward a degree. Education provided through learning communities and linked courses, described below, reflect this strategy. It should be noted that there is also some evidence that delaying remediation taking developmental courses only after a semester of college-level work has been completed has a negative effect on student transfer and completion. 12. Entry-level, degree program courses provide opportunities for review and recoupment of basic skills necessary for success in the class. (55%) 13. Students have an opportunity to progress at their own pace within a modular approach, as one-credit modules. (11%) 11

Program Components: Assessment 14. Assessment is mandatory for all entering students. (94%) 15. Placement in courses is mandatory based on assessment. (95%) Accurate assessment and placement of students has been shown to be a major factor in the success of remedial/developmental programs (Calcagno, 2007; Gabriner, et. al, 2007; Perin, 2006; Grubb, 2001; Roueche & Roueche, 1999). Early research has also identified mandatory assessment and placement of students in remedial courses as a characteristic of successful remediation efforts (Roueche & Baker, 1987; Roueche & Roueche, 1993; Roueche & Snow, 1977). Later authors have continued to advocate for mandatory assessment and placement and have provided a variety of arguments and research studies to support their advocacy (Casazza & Silverman, 1996; Maxwell, 1997; Morante, 1987; Morante, 1989). However, the available evidence suggests that only mandatory assessment is clearly associated with student and program success in remedial courses (Boylan, Bonham, Claxton, & Bliss, 1992; Boylan, Bliss, & Bonham, 1997). The early identification of those students at risk of failure was, indeed, found to be associated with successful remediation (Adelman, 1999; Kulik, Kulik, & Schwalb, 1983). This seemed to support the argument for mandatory assessment and placement. Mandatory placement in remedial courses, however, appeared to have a statistically significant, negative impact on the retention of students in remedial programs (Boylan, Bonham, & Bliss, 1994). Boylan, Bliss, and Bonham (1997) have argued that this apparent inconsistency is really a function of a change in the types of students enrolled in remedial courses brought about by mandatory placement. When placement is voluntary a large number of the most poorly prepared students fail to volunteer for or otherwise avoid remedial courses. Although these students are likely to become attrition statistics, they are not counted as such by the remedial program if they are not participants. In essence, voluntary placement tends to prevent a large number of the weakest students from being included in the program s service population. The students participating in remedial courses under a voluntary placement system, therefore, tend to be more highly motivated or to recognize the need for developing their skills before pursuing curriculum courses. They are also more likely to be successful than less motivated and less realistic students. When placement is mandatory, a higher percentage of academically weaker and less motivated students are taking remedial courses. These students are among the least likely to be successful in remediation. This contributes to the negative relationship between mandatory placement and student retention when the results of voluntary placement are compared to the results of mandatory placement (Boylan, Bliss, & Bonham, 1997). This should not be construed as an argument against mandatory placement. As Cross (1976) points out, fewer than 10% of those needing remediation are likely to survive in college without it. Even though large numbers of the weakest students will become 12

victims of attrition under systems of mandatory placement, more will survive than if they had not received any remediation at all. 16. Students completing the required developmental education sequence may enroll in degreeprogram courses without retaking a placement exam. (93%) Program Components: Evaluation 17. Regular evaluations are conducted and data are used to improve programs, initiatives or services. (58%) Analysis of data from the National Study of Developmental Education suggested that the relationship between program evaluation and student success had a great deal to do with how program evaluation information was used. Apparently, programs performed better when evaluation included a combination of formative and summative evaluation and when formative evaluation data was used to refine and improve the program (Boylan, Bliss, & Bonham, 1997). This emphasis on the use of formative evaluation for the purpose of program improvement was also found to be associated with student success in a recent study of Texas community colleges (Boylan & Saxon, 1998). An early study, supported by later research, has been the importance of evaluation to the success of remediation efforts. Donovan s (1974) analysis of successful programs for at-risk students found that those that evaluated their efforts on a regular and systematic basis were more successful than those that did not. This finding was echoed in Roueche and Snow s (1977) study of successful remedial programs. Boylan, Bonham, Claxton, and Bliss (1992) later found that program evaluation was positively related to student grades in remedial courses and also associated with the long-term retention of remedial students. Levin and Calcagno (2008), in Remediation in the community college: An evaluator s perspective, provide a conceptual framework for the evaluation of remedial education programs and identify many of the difficulties associated with such evaluations. 18. Data are regularly collected to measure the success of students in each remedial course and in the first related entry-level degree program course in the sequence. (57%) Program Components: Support 19. Tutoring is provided to developmental students in basic skills subjects. (94%) The impact of tutoring on remedial students has been widely debated in the literature. Early studies of remediation suggested that tutoring was an important component of successful programs for underprepared students (Roueche & Snow, 1977). Maxwell (1997), however, has argued that research findings on the impact of tutoring on underprepared students have been mixed with no conclusive results being found. Research by MacDonald (1994), Casazza and Silverman (1996), and Boylan, Bliss, and Bonham (1997) helps to clarify this inconsistency. Apparently the effectiveness of tutoring is strongly influenced by the quality and the amount of training received by 13

tutors. This is particularly true when the subjects of tutoring are underprepared students. Boylan, Bonham, Claxton, and Bliss (1992) found that there was no difference in the performance of students participating in remedial programs whether they received tutoring or not, unless the tutoring program included a strong tutor training component. As MacDonald (1994) pointed out, tutors will be ineffective unless they are able to consistently and usefully apply strategies appropriate to each student s situation. This can only be accomplished through training. 20. Developmental education students have easy access to computer assisted instruction. (75%) Several studies have found that integrating classroom and laboratory instruction was associated with student success in remedial courses (Gabriner et al., 2007; Boylan, 2002; Boylan, Bliss, and Bonham, 1997). When classrooms and laboratories were integrated, instructors and laboratory personnel worked together to insure course objectives were directly supported by laboratory activities. Boylan and Saxon (1998) found that the integration of classroom and laboratory instruction in this manner was also related to student success on a state-mandated achievement test in Texas. 21. Learning communities are organized for developmental students. (46%) The use of learning communities in remedial courses has also been found to improve the performance of students participating in remediation (Gabriner et al., 2007; Bailey & Alfonso, 2005; Bloom & Sommo, 2005; Taylor, Morre, MacGregor, & Limblad, cited in Bailey & Alfonso, 2005; Boylan, 2002). Learning communities have combined courses and groups of students organized as cohorts. Typically, these cohorts of students took courses linked together by a common theme, and instructors of these courses functioned as a team to insure that the content of each course was related to and supportive of the other courses (Adams & Huneycutt, 1999). The use of paired courses has offered another example of the learning community concept. A reading course, for instance, might be paired with a social science course and students would enroll as a cohort in both courses. The instructors of these two courses would then collaborate to insure that concepts taught in reading related directly to what was being learned in sociology courses (Adams & Huneycutt, 1999). Tinto (1997) found that underprepared students participating in remedial courses organized around the principles of learning communities had better attitudes toward learning and had higher course completion rates than students in traditional remedial courses. In later research Tinto (1998) found that the use of learning community concepts to teach remedial courses resulted in improved retention for participating students. Commander, Stratton, Callahan, and Smith (1996) found that participating in paired courses improved student performance and resulted in higher levels of reported student satisfaction. 14

22. Students who are overly challenged by developmental education curricular activities are screened to identify individual learning styles and appropriate accommodations are made in developmental education classes. (35%) The body of research suggesting that remedial students learn in ways not accommodated by traditional instruction has been growing (Gabriner et al., 2007; Goldrick-Rab, 2007 Levin & Calcagno, 2007; Boylan, 2002; Grubb, 2001). Canfield (1976), for instance, found that students enrolled in community college remedial courses were much more likely to be either iconic (visual) or hands on learners than other students. Using a modified version of the Kolb Learning Styles Inventory, McCarthy (1982) found that weaker college students tended to be more visually oriented or more inclined to learn through direct experience than other learners. Lamire (1998) cited half a dozen studies of community college students indicating that a dominant learning style among them was visual followed by what he referred to as haptic or learning by doing. Apparently the use of a variety of instructional methods, particularly those using visual or hands on approaches to learning were more likely to appeal to the learning styles of students typically enrolled in remedial courses. 23. Advising support is substantial, accessible, and integrated into academic courses. (65%) The literature emphasizes the importance of colleges offering services to support developmental students in their learning. While each support can be provided individually, taking a holistic approach by implementing a variety of services to meet the diverse educational and personal needs of students seems to be most effective. In fact, experience suggests that student persistence increases with the number and extent of coordination of the services offered their availability and their responsiveness to personal needs and schedules (Boylan, 2002; Jenkins, 2006; Gabriner et al., 2007; Muraskin, 1998; Roueche & Roueche, 1999) Program Components: Curriculum & Instruction 24. Developmental education course competencies are aligned throughout the sequence, including entry-level, degree program courses. (68%) Colleges would do well to consider their policies from a holistic perspective. Doing so would help ensure that all aspects of their programs and services are well aligned to support student success (see Jenkins, 2006). A holistic perspective would also help to ensure that all of the diverse needs of students are adequately met (see, for example, Gabriner et al., 2007) Developmental students often lack the ability to comprehend and to organize multiple concepts simultaneously. Experience suggests that students develop this ability through very structured learning experiences in the classroom. Structured learning involves dividing the curriculum into manageable units and teaching those units in a step-bystep sequence while providing tutoring or other academic support when students have difficulty mastering particular topics. (Boylan, 2002; Gabriner et al., 2007) 25. The content of developmental courses is contextualized by career-technical expectations. (33%) 15

The contextual teaching and learning approach is another promising approach; it involves creating meaningful contexts in which students can learn. A contextual approach to teaching and learning can be carried out in a number of ways: by teaching students basic skills in the context of instruction in technical subject matter; by relating subject matter to real world applications; or by allowing students to solve problems through simulations or even in actual settings in the workplace or elsewhere outside the classroom. Contextual teaching and learning allows students to build on the knowledge they already have by enabling them to learn in contexts that make sense to their lives outside of the classroom. This approach also puts more of the responsibility for learning on students, with the teacher helping to facilitate the learning process rather acting as the purveyor of knowledge. (Gabriner et al., 2007; Goldrick-Rab, 2007) Program Components: Outreach 26. Successful strategies are shared with area feeder schools to improve preparation of secondary school students. (22%) 27. Career counseling is integrated with developmental education courses. (40%) Early research also found that successful remedial education programs had a strong counseling component (Gabriner, et. al., 2007; Goldrick-Rab, 2007; Bailey & Alfonso, 2005; Roueche & Mink, 1976; Roueche & Snow, 1977). This relationship between an emphasis on personal counseling for students and successful remediation was supported in later research by Keimig (1983), Kulik, Kulik, and Schwalb (1983), Boylan, Bonham, Claxton, and Bliss (1992), the Higher Education Extension Service (1992), and Casazza and Silverman, (1996). This latter research indicated, however, that counseling in and of itself was not sufficient to impact upon student success. In order for counseling to be successful with remedial students it had to: (a) Be integrated into the overall structure of the remedial program (Kiemig, 1983); (b) Be based on the goals and objectives of the program (Casazza & Silverman, 1996); (c) Be undertaken early in the semester (Kulik, Kulik, & Schwalb, 1983); (d) Be based on sound principles of student development theory (Higher Education Extension Service, 1992), and (e) Be carried out by counselors specifically trained to work with developmental students (Boylan, Bliss, & Bonham, 1997). Instructional Practices 28. Indicate the number of dedicated developmental education faculty who full-time and parttime. Responses indicate that full-time faculty make up about 30% of those instructors teaching developmental education courses. Full-time faculty who usually teach credit-bearing courses may also teach developmental courses. Ensuring that these instructors are adept at using the teaching 16

strategies appropriate to developmental students can increase their effectiveness. Frequently the faculty who teach developmental courses are adjuncts. It is important that adjunct instructors be integrated into the college community as fully as possible. This can be difficult given their part-time status. Ensuring that adjuncts are available to their students outside of class can also be a challenge. Part-time status can offer greater flexibility in scheduling classes and meetings and can result in a better match between instructor, course, and class (Gabriner, et. al., 2007). 29. The following instructional methodologies are a part of our developmental education courses: Variety of teaching methods. The use of a variety of different teaching methods was also recommended in the early studies of remedial instruction (Gabriner, et. al., 2007; Goldrick-Rab, 2007; Levin & Calcagno, 2007; Grubb, 2001; Roueche, 1968; Roueche & Wheeler, 1973). Students in remedial courses have been lectured to in the past without much effect. If traditional teaching methods had worked for these students, they would 17

not be taking remedial courses. Consequently, Roueche and his colleagues argued for the use of a wide variety of teaching techniques featuring class discussions, group projects, and various types of mediated learning. Again, these early findings have also been validated through later research. Cross (1976), Kulik and Kulik (1991), and Casazza and Silverman (1996) all found that students in remedial courses were likely to be more successful when a variety of instructional methods were used. Mastery learning. Roueche and his colleagues also emphasized mastery learning as a component of effective remedial instruction (Roueche, 1968; Roueche & Wheeler, 1973). They were particularly influenced by the work of Bloom (1968) and Carroll (1963) in this regard. All of the approaches to mastery learning utilized small units of instruction and frequent testing and required students to be able to master the material in one unit before progressing to the next unit. This emphasis on mastery was beneficial to students in remedial courses because it provided regular reinforcement of concepts through testing. An emphasis on mastery required students to develop the prerequisite knowledge for success in a given course and to demonstrate this knowledge through testing. Although mastery learning is not nearly as popular today as it was in the 1960s and 1970s, the evidence has suggested that it is still a highly effective instructional technique for remedial courses. Research by Cross (1976), Kulik and Kulik (1991) has also strongly supported the use of mastery learning for remedial courses. Students exposed to mastery learning techniques in remedial courses were more likely to pass these courses, obtain higher grades, and be retained than students whose remedial courses were taught using more traditional techniques (Boylan, Bonham, Claxton, & Bliss 1992). A recent study of remedial courses in Texas community colleges also found that students taught using mastery learning techniques were more likely to pass a statewide achievement test in the remedial subject area than students taking remedial courses which did not feature mastery learning (Boylan & Saxon, 1998). Critical Thinking. Remedial students not only have been less likely than others to understand the expectations and rewards of college, they also have been less likely to understand the types of thinking required for success in college courses. The emphasis of critical thinking throughout the remedial curriculum has proven successful in improving the performance of underprepared students (Gabriner, et. al, 2007; Pascarella & Terenzini in Bailey & Alfonso, 2005). The work of Chaffee (1992) and his colleagues teaching critical thinking at La Guardia Community College has been particularly impressive in its impact on underprepared students. Participation in courses, programs, and activities designed to enhance critical thinking has improved students performance in reading and writing (Chaffee, 1992; St. Clair, 1994-95), improved students attitudes toward learning (Harris & Eleser, 1997), and contributed to higher grade point averages and retention (Chaffee, 1998). 18

Instruction 30. Technology is used as a supplement for instruction in developmental courses. (85%) Computer-assisted models offer a promising accompaniment to traditional means of teaching content in remedial classes, allowing students to learn at their own pace, reinforce an instructor s efforts, monitor students progress, and providing diagnostic feedback (Garbriner, et. al., 2007; Boylan 2002). In an analysis of computer-based instruction at 123 colleges and universities, Kulik & Kulik (1986) found that the use of the computer as a tutor designed to supplement regular instruction had several positive effects. These included: (a) More student learning in less time, (b) Slightly higher grades on post-tests, and (c) Improved student attitudes toward learning. In a later review of research on the use of computers with underprepared students, Kulik & Kulik (1991) found that computer-based instruction has raised student achievement in numerous settings. (p. 32) Roueche & Roueche (1999) found that the use of computers for students to do writing assignments and as a tutor in mathematics contributed to the success of remedial courses. Using data from the National Study of Developmental Education, Bonham (1992), however, found that the effectiveness of computer-based instruction declined when it was used as the primary delivery technique in remedial courses. Computer-based instruction appeared to be most successful when it was used as a supplement to regular classroom activities in remedial courses. Where computer-based instruction was used in this, students were more likely to complete remedial courses and to earn higher grades (Bonham, 1992). These findings were later verified in work by Maxwell (1997) and a recent study of remedial programs in Texas (Boylan & Saxon, 1998) Research in the past decade has identified several other factors that contribute to successful remedial courses and programs. Boylan, Bliss, and Bonham (1997), for instance, found that integrating classroom and laboratory instruction was associated with student success in remedial courses. When classrooms and laboratories were integrated, instructors and laboratory personnel worked together to insure course objectives were directly supported by laboratory activities. Boylan and Saxon (1998) found that the integration of classroom and laboratory instruction in this manner was also related to student success on a state-mandated achievement test in Texas. 31. Learning strategies, i.e. college success skills, are taught in a separate course from developmental English or math. (92%) Some developmental education programs offer separate courses or tutorials, sometimes called strategic learning courses. Such courses cover a variety of learning-to-learn strategies, including guidance on taking notes, group- and self-study, test taking, time management, and successful education and personal habits. An even broader conception of learning-to-learn skills is sometimes provided in what are called college 19

success courses. These courses include content on career exploration and planning, introduction to the culture and expectations of college, and, in some cases, life skills tutorials on topics such as personal finance. Some research suggests that students who take such courses have better college outcomes (Gabriner, et al., 2007; Zeidenberg, Jenkins, & Calcagno, 2007; Pascarella & Terenzini, cited in Bailey & Alfonso, 2005). Community college remedial programs have recently begun providing organized college orientation seminars for their students. Although the freshmen seminar was initially developed for university students, this concept has since been successfully implemented at many community colleges (Upcraft, & Gardner, 1989). As demonstrated in this research, because community college students were likely to be the first generation of their family to attend college, they tended to be unfamiliar with the expectations and rewards of academe. It should come as no surprise, therefore, that they frequently failed to meet these expectations or to be rewarded in academe. College orientation courses, therefore, were useful tools for helping students learn what was expected of them and assisting them in adjusting to the college environment. Recent research (Gardner, 1998) has shown that underprepared students participating in ongoing orientation courses were much more likely to be retained in the community college than students who did not participate in these courses. 32. Learning strategies, i.e. college success skills, are embedded in developmental courses. (68%) An emphasis on strategic learning also has contributed to the effectiveness of remedial courses. In the early 1980s, Claire Weinstein argued that underprepared students do not know how to acquire and process information and must, therefore, be taught to monitor their comprehension and think strategically about learning (Weinstein, 1982). In short, remedial students had to learn to recognize when they were not comprehending material and then be able to apply alternative strategies to improve their comprehension. Weinstein and her colleagues expanded these concepts and developed a comprehensive model for teaching underprepared students to think strategically (Weinstein & Rogers, 1985; Weinstein, 1988). When this model for strategic thinking processes was integrated into the remedial curriculum, students became more effective learners, obtained higher grades, and were retained over longer periods of time (Weinstein, Dierking, Husman, Roska, & Powdrill, 1998). 33. A high degree of structure is provided in developmental education instruction. (74%) Another principle emphasized in the early research was that of structure. Roueche (1973) found that students taking remedial courses required a high degree of structure for their learning experiences. Cross (1976) later argued remedial students tended to lack the organizational schema necessary to comprehend many academic concepts. The provision of highly structured learning experiences helped students compensate for this shortcoming by modeling appropriate methods of organizing information. Based on their research on the interaction between student aptitude and instructional methods Cronbach and Snow (1977) also argued that structured learning environments provided the most benefit to the weakest students. Their position was further supported by the research of Kulik and Kulik (1991) and Boylan, Bonham, Claxton, and Bliss (1992). 20

Adult Basic Education 34. Potential developmental education students are informed of ABE/GED college prep services. (46%) 35. Curricula/competencies are aligned between ABE and DevEd. (38%) 36. Students may enroll in both ABE and DevEd coursework during the same semester. (48%) 37. There is a designated adviser serving developmental education students. (36%) 38. There is a designated adviser serving ABE/GED students. (44%) Professional Development 39. Professional development for developmental educators is consistently supported and ongoing. (51%) Many authors have described the importance of training for those who work with underprepared students (Gabriner et al., 2007; Jenkins, 2006; Boylan, 2002; Casazza & Silverman, 1996; Maxwell, 1997; Roueche, 1973). Recent research has validated the need for faculty and staff working with remedial programs to be specifically trained in the techniques, models, and methods appropriate to helping underprepared learners. Boylan, Bonham, Claxton, and Bliss (1992) found that students were more likely to pass remedial courses, earn higher grades, and be retained longer if remedial programs placed a strong emphasis on professional development for faculty and staff. Later analysis of data from this study indicated that the training of staff contributed to increased effectiveness of individual program components such as instruction, counseling, and tutoring as well as to overall program effectiveness (Boylan, Bliss, & Bonham, 1997). The importance of professional training of those working with underprepared students has also been emphasized in the work of Casazza & Silverman (1996) and Maxwell (1997). 40. Developmental educators are currently involved in their professional associations. (44%) 41. Instructional strategies are regularly and systematically shared among developmental instructors. (65%) 42. Tutors working with developmental students participate in related professional development activities. (41%) Research has shown that the effectiveness of tutoring is strongly influenced by the quality and the amount of training received by tutors (MacDonald, 1994); Casazza and Silverman, 1996; and Boylan, Bliss, and Bonham, 1997) This is particularly true when the subjects of tutoring are underprepared students. Boylan, Bonham, Claxton, and Bliss (1992) found that there was no difference in the performance of students participating in remedial programs whether they received tutoring or not, unless the tutoring program included a strong tutor training component. As MacDonald (1994) pointed out, tutors will be ineffective unless they are able to 21

consistently and usefully apply strategies appropriate to each student s situation. This can only be accomplished through training. 43. Professional development is clearly connected to faculty reward structures. (22%) 44. Developmental education instructors are trained in the assessment of learning styles. (51%) One additional and significant research reference comes from the 2007 Community College Survey of Student Engagement (CCSSE) report. Findings from the report, based on five years of data from the initial 27 ATD colleges, show that first-term completers of any remedial course(s) are more likely to persist and succeed than any other student group, including those who did not need developmental education. This speaks to the critical importance of students in remedial classes experiencing some initial success. 22

CONCLUSIONS & RECOMMENDATIONS From these survey results and related research, a few broad conclusions and recommendations are presented as a starting point for thinking about how New Mexico might improve remedial education programs and services to the many students arriving underprepared at our postsecondary institutions. First, the above review of the literature indicates that no set of chosen best practices is effective for every student or even every college. Second, analysis of these survey results has revealed wide discrepancies in responses across the state and even within individual institutions. This indicates a need statewide for better understanding of our developmental/remedial education programs. Results taken in aggregate, however, remain a good indication of where New Mexico might focus its resources for improving the college readiness of its many underprepared students. Conclusions based on a survey of the field are as follows: 1. The New Mexico colleges stated commitment to developmental education as a priority is the case in about 2/3 of our colleges. It is included in the strategic plans of these colleges and it appears that many of these same colleges have clearly defined mission, goals and measureable objectives for their developmental education programs. What is not clear is how this intention is translated into practice. (Questions 1 and 2). 2. In a little over half of the colleges, there is the belief that the program is adequately funded. Where best practices exist, they often operate on the margins along side, but not a significant influence on historically offered (and sometimes unexamined) services and strategies. If these adequately funded colleges are also showing good outcomes, this needs to be explored. (Question 3) 3. The developmental education programs in New Mexico seem to universally employ a number of instructional approaches. A wide variety of modes are used by about half. Among the instructional methodologies used most regularly are learning strategies offered as either a separate or an embedded course, computer assisted instruction and active learning. In terms of computer assisted instruction, it is not known how integrated the classrooms and labs were and if they were embedded in their course schedule. Among instructional methodologies used least regularly are fast start/acceleration and self-paced, modularized approaches and contextualized learning... approaches that emerging research indicates is working. What right combination of approaches is most effective and how structured cannot be determined by the survey results. (Questions 4, 13, 20, 21, 25, 29, 30, 31, 32, and 33) 4. Most of the colleges (70%) indicate that a wide range of support services are available to developmental education students. About the same percentage indicate that these support services are comprehensive, individualized and proactively offered. Tutoring is the support service most predominantly utilized. Substantial advising support is provided by about 2/3 of the respondents but career counseling by only about 1/4th. What is not clear is what is happening with the other 30% of the respondents in terms of 23

what is provided in student support. What research suggests is a holistic approach and what is not clear is what a successful holistic approach might be and what support services if any may be mandated to specific student cohorts (such as those in the lowest levels). (Questions 5, 9, 23, 27 and 37) 5. Only 6% of the respondents indicate that developmental students cannot enroll in any other courses but developmental. Further, 55% indicate that there are opportunities for review and recoupment of basic skills in their entry level college courses. (Questions 6 and 12) 6. Most of the colleges developmental education programs (60%) are mainstreamed to some degree. What is more important is that a high degree of coordination of effort and communication among faculty and staff providing remediation occurs. 71% of the colleges indicate that there is this coordination of effort and communication among faculty and staff providing the remediation. (Questions 7 and 8) 7. 81% of the respondents indicate that institutional processes facilitate student completion of necessary developmental coursework as early as possible in the developmental sequence. What these processes are and how early in the sequence the practice is occurring cannot be determined from the survey responses. Likewise, the same percentage indicate that students are kept aware of their progress as they move through their required course sequence, but how they are kept aware (such as through required advising or frequent diagnostic testing) cannot be determined from the survey responses. (Questions 10 and 11) 8. Assessment and placement in courses being mandatory appears to be a common practice at most of the colleges reporting. While these policies seem common, practices behind the policies may very well differ from college to college and these would be of interest to determine. In addition, there would be interest in exploring what other assessments can supplement these results to allow diagnostic information for targeted and individualized enhancement and instructional modes that may match students specific needs and allow for advancements without full course enrollment. In a related question, retaking the placement exam is not generally required of students completing their developmental sequence before enrolling in their degree program. (Questions 14, 15 and 16) 9. Only a little more than half of the respondents regularly collect data to measure the success of students in each developmental education course and in the first related entry-level degree program course in the sequence. Further, the same percentage conduct regular program/service evaluations for improvement based on data. It can thus be assumed that many colleges do not know the extent to which developmental students are successful or not successful and without this data cannot reasonably evaluate the effectiveness of their programs. Given the low success rate of developmental students (especially those in the lowest level) of ever achieving their educational goals, this would seem to be an imperative. (Questions 17 and 18) 24

10. Only 35% of the respondents indicate there is attention paid and accommodations made to developmental students having learning style differences. Further, it is not known from the survey if developmental education faculty are working with learning disabilities experts to provide appropriate support and intervention for learning disabled students. This may indicate a need for professional development in these areas. (Question 22) 11. 2/3 of the respondents indicated that developmental education course competencies are aligned throughout the sequence, including entry-level, degree courses. Experience suggests (ATD Initiative) that developmental education students in particular develop success through very structured and sequential learning experiences. Evidently some colleges have made this type of alignment a priority we need to assess further what the results of these efforts were. Once assessed, those that have accomplished a successful alignment resulting in more positive student outcomes need to demonstrate to others what and how it was done. Again, once these models have been documented, the issue would be ripe for a discussion of statewide alignment of competencies and courses. (Question 24) 12. Only about 1/4 of the respondents share successful strategies with area feeder schools to improve preparation of secondary students. This starts with data such as that obtained through early testing. Data have shown that developmental education needs are greatest among the recent high school graduates the colleges enroll. Given the persistence and achievement gaps between college-ready students and those who must complete developmental coursework (especially those in the lowest levels) as a prerequisite to college-level study, reducing the number and array of pre-college courses student take is an important goal. (Question 26) 13. Responses indicate that full-time faculty make up about 30% (on average) of those instructors teaching developmental education courses. Given the high percentage of part-time faculty in these programs and the relative lack of success of students in these programs, it is important that these adjuncts are given opportunities to participate in departmental meetings, related professional development and in discussions evaluating the effectiveness of the programs and services being provided. It is also necessary that full-timers teaching developmental education have the opportunity for adequate professional development. (Question 28, 39, 40, 41, 42, 43, and 44) 14. ABE/GED alignment/services appear not to be a high priority for most. This is an emerging issue and could be addressed by the joint efforts of the existing ABE Transitions Task Force and Developmental Education Task Force. (Questions 34, 35, 36 and 38) Specific recommendations derived from the above study and other evidence-based research are as follows: 25

Organizational and Administrative Practices: Developmental education is a clearly stated institutional priority with institution-wide commitment to it embraced and understood by campus stakeholders toward this end stakeholders need to be armed with useful and continuous information about the success/non-success of developmental students. A clearly articulated mission based on a shared operational philosophy drives the developmental program and clearly describes the program goals and objectives based on this philosophy. As a precursor to setting a coherent course of action for improving developmental education, institutions should collect, analyze and disseminate data relative to student achievement and success at critical junctures, determine critical domain for improvement, and set clear goals and incremental benchmarks by which progress can be measured. Adequate funds are provided for the provision and rigorous evaluation of developmental education program. Institutional policies and practices facilitate student completion of necessary developmental coursework as early as possible in the educational sequence with strategies to include evidence-based course placement accuracy, early advisement so that taking developmental education courses aren t delayed. A comprehensive system of support services exists, and is holistic, characterized by a high degree of coordination and integration among academic and student support services with these support services targeted specifically and individually toward their needs. Whether centralized or mainstreamed, a high degree of coordination of effort and communication among faculty and staff providing remediation occurs. Bridge the chasms between and among adult basic education, high school and college. Program Components: Orientation, assessment, placement and other supports such as advising may need to be made mandatory for all new students and for at-risk students in particular. Developmental/at-risk students don t do optional, that is, given options, many students will opt out of some or all of the many services available to them. Regular evaluations are conducted and data (to include student success data suggested below) are used to improve programs, initiatives or services. Data are regularly collected to measure the success of students in each remedial course and in the first related entry-level degree program course in the sequence, and achievement gaps are regularly identified among target groups. 26

Advising and counseling support is substantial, accessible and integrated into academic courses and programs (such as made mandatory until enrolling in college-level courses or for the first 30 credit hours taken). Build community and strong connections among students, faculty and staff wherever possible. Developmental education course competencies are aligned throughout the developmental education course sequence and into entry-level degree program courses. Draw lessons from demonstrated best practices; grow successful practices; and encourage innovation and experimentation. Financial aid is available to students while taking developmental courses without reducing subsequent aid for credit-bearing courses, and students are made aware of and provided assistance for procuring all aid available opportunities for aid. Instructional Practices: Sound principles of learning theory in general and adult learning theory in particular as well as culturally responsive teaching theory are applied in the design and delivery of courses. Curricula and practices that have been proven to be effective within specific disciplines are demonstrated and implementation strategies are employed. Accelerated developmental education programming can reduce the time a student is involved with developmental education classes, increasing the time devoted to pursuit of a postsecondary credential, and reducing the cost of education to the state, while maximizing the effectiveness of student financial aid. Modular programs for remediation, supplemented by online or computer-assisted tutorial support can support, students in becoming college-ready within their own schedules and at their own pace. In-class tutors and supplemental instruction within both remedial and degree program courses can increase the success of students in need of recouping or maintaining an effective academic skill level. Student success courses on entry that cover college readiness skills, information necessary to succeed in college, time management and other skills either paired with developmental education in learning communities or as individual courses, have been shown to be effective in enhancing student success. The developmental education program addresses holistic development of the student to include social and emotional development as well as to their cognitive growth and learning styles. 27

A high degree of structure is provided in developmental education courses again, students don t do optional so the more that is built into their courses or their schedule, the better. Developmental education faculty employ a variety of instructional methodologies and techniques to accommodate student diversity and levels of preparedness and routinely share these strategies with others. Programs align entry/exit skills among levels of developmental education offered and link end of sequence course content to entry-level college course performance requirements. Faculty and advisors closely monitor, report back student progress and maintain effective early alert processes. Programs provide comprehensive academic support mechanisms. Professional Development: Administrators support and encourage faculty development (to include part-timers) in basic skills, and the improvement of teaching and learning is connected to the institutional mission. The faculty play a primary role in needs assessment, planning, and implementation of professional development programs and activities in support of basic skills programs. Professional development programs are structured and appropriately supported to sustain them as ongoing goals related to institutional goals for the improvement of teaching and learning as well as student development and achievement goals. Staff development opportunities are flexible, varied, and responsive to developmental needs of individual faculty, diverse student populations, and coordinated programs/services. Where possible, faculty development is clearly connected to intrinsic and extrinsic faculty reward structures. It is unreasonable to expect that any combination of the effective practices described above will create large changes in success rates in a short period of time. However, the literature does show that sustained efforts over extended periods of time do transform institutions and student success rates do improve over time. It is also unreasonable to expect that budgets can support many of the effective practices described above especially in tough economic times. Nor is there any magic combination of these effective strategies that have proven to work given the institutional cultural differences of our institutions. Therefore, a more prudent approach may be recommended as suggested by Schwartz and Jenkins (2007). They suggest that a more productive approach would be through establishment of continuous improvement processes for developmental education programs: 28

Research suggests that colleges will be more successful in improving outcomes for developmental students not by adopting the latest best practice, but by adopting a continuous improvement process that involves regularly monitoring the progress of students, trying different approaches to help students overcome identified barriers to success, evaluating the success of such interventions, and making further adjustments based on these results. Next Steps and Areas for Further Inquiry. In order to see what is actually working/not working and to get more clarity/detail related to survey responses, it is suggested that on-campus interviews might be conducted by a panel of external experts on best practices in remedial courses and programs. A protocol for this campus interview proposal may be as follows: Clarify from the responses given in the survey areas of further inquiry and specific interview questions; Identify a team of administrators/faculty/staff/students at selected colleges to participate in the interview process; Identify and describe remedial programs within participating colleges; Convene focus group and individual interviews around the strategies and practices that increase access and success of underprepared populations. Report to colleges on their current capacity to serve the target population within a research-based, best practices context. Interview questions that might be considered: 1. What are the most significant things you ve learned from the work your campus has been doing in DE? 2. What programs/courses have been the most successful ones? Why? What practices do you utilize? What outcomes are you most proud of? (would require analysis/ outcomes documentation ahead of time) 3. What propelled you to take on any particular initiatives/interventions? (models they might have replicated, literature read, conferences/workshop ideas, etc.) 4. If you have successful practices and your most successful practices are not yet college wide, how, if at all, does the college plan to bring these ideas to scale? If it doesn t, what ideas do you have about doing so? 5. What have been your greatest challenges regarding efforts your college has made to strengthen student outcomes in developmental education (or in your area of developmental education)? What learning have you taken away from those experiences? 29

6. Are there combinations of courses/services that have worked particularly well? What have been the outcomes if measured? Why do you think they have been successful? 7. What practices/policies/interventions are you now considering or working on? Why are you moving in that direction? 8. What/who are your most often-turned to sources of information about effective developmental education policies and practices? Why do you use those sources? (want to know if general references or specific program sources) 9. What would you most like to get out of this investigative project for your institution? (Are there specific things you want to learn from other NM colleges or other colleges regarding developmental practices and policies? This report conclusions, recommendations and next steps will be discussed and refined at the 2009 Governor s Higher Education Summit Adult Basic and Developmental Education Pre-Conference. 30

REFERENCES Adams, S., & Huneycut, K. (1999, April). Learning communities at Sandhills Community College. Paper presented at the North Carolina Community College System Conference on Alternatives to Remediation, Boone, NC. Adelman, C. (1999, June). Answers in the tool box: Academic intensity, attendance patterns, and bachelor s degree attainment. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement. Attewell, P., Lavin, D., Domina, T., & Levey, T. (2006). New evidence on college remediation. Journal of Higher Education, 77(5), 886-924. Bailey, T. R. (in press). Bridging the high school-college divide. New York: Columbia University, Teachers College, Community College Research Center. Bailey, T. R., & Alfonso, M. (2005, January). Paths to persistence? An analysis of research on program effectiveness at community colleges. New Agenda Series. Indianapolis, IN: Lumina Foundation. Bailey, T. R., Leinbach, T., & Jenkins, D. (2005, January). Community college low-income and minority student completion study: Descriptive statistics from the 1992 high school cohort. New York: Columbia University, Teachers College, Community College Research Center. Bettinger, E., & Long, B. (2005). Remediation at the community college: Student participation and outcomes. New Directions for Community Colleges, 129(1), 17-26. Bettinger, E., & Long, B. (in press). Institutional responses to reduce inequities in college outcomes: Remedial and developmental courses in higher education. In S. Dickert-Conlin & R. Rubenstein (Eds.), Economic inequality and higher education: Access, persistence and success. New York Russell Sage. Bloom, D., & Sommo, C. (2005). Building learning communities: Early results from the Opening Doors Demonstration at Kingsborough Community College. New York: MDRC. Bloom, B. (1968). Learning for mastery. In Evaluation comment. Los Angeles, CA: Center for the Study of Evaluation in Instructional Programs, University of California. Bonham, B. (1992, November). Research findings on developmental instruction. Paper presented at the First National Conference on Research in Developmental Education. Charlotte, NC. Boylan, H. R. (2002). What works: A guide to research-based best practices in developmental education. Boone, NC: Appalachian State University, Continuous Quality Improvement Network and National Center for Developmental Education. 31

Boylan, H., Bliss, L., & Bonham, B. (1997). Program components and their relationship to student success. Journal of Developmental Education, 20(3), 2-8. Boylan, H., Abraham, A., Allen, R., Anderson, J., Bonham, B., Bliss, L., Morante, E., Ramirez, G., & Vadillo, M. (1996). An evaluation of the Texas Academic Skills Program. Austin, TX: Texas Higher Education Coordinating Board. Boylan, H., & Bonham, B. (1992). The impact of developmental education programs. Research in Developmental Education, 9(5), 1-4. Boylan, H., Bonham, B., & Bliss, L. (1994). Characteristic components of developmental programs. Research in Developmental Education, 11(1), 1-4. Boylan, H., Bonham, B., Claxton, C., & Bliss, L. (1992, November). The state of the art in developmental education: Report of a national study. Paper presented at the First National Conference on Research in Developmental Education, Charlotte, NC. Boylan, H., & Saxon, D. (1998). An evaluation of developmental education in Texas public colleges and universities. Austin, TX: Texas Higher Education Coordinating Board. Bruner, J. (1966). Towards a theory of instruction. New York, NY: Norton & Co. Calcagno, J. C. (2007). Evaluating the impact of developmental education in community colleges: A quasi-experimental regression-discontinuity design. PhD. Dissertation, Columbia University, Graduate School of Arts and Sciences. Canfield, A. (1976). The Canfield learning styles inventory: Technical Manual. Ann Arbor, MI: Humanics Media. Carroll, J. (1963). A model for school learning. Teachers College Record, 64. 723-733. Casazza, M., & Silverman, S. (1996). Learning assistance and developmental education. San Francisco, CA: Jossey-Bass. Chaffee, J. (1992). Critical thinking skills: The cornerstone of developmental education. Journal of Developmental Education, 15(3), 2-8, 39. Chaffee, J. (1998, January). Critical thinking: The cornerstone of remedial education. Paper presented at the Conference on Replacing Remediation in Higher Education, Stanford University, Palo Alto, CA. Clark-Thayer, S. (1995). NADE self-study guides. Clearwater, FL: H & H Publishing. 32

Claxton, C. (1992, July). The adult learner. Presentation at the Kellogg Institute for the Training and Certification of Developmental Educators, Boone, NC: National Center for Developmental Education. Cohen, A., & Brawer, F. (1989). The American community college. San Francisco, CA: Jossey- Bass. Commander, N., Stratton, C., Callahan, C., & Smith, B. (1996). A learning assistance model for expanding academic support. Journal of Developmental Education, 20(2), 8-16. Cronbach, L., & Snow, R. (1977). Aptitudes and instructional methods. New York, NY: Irving Publishing Company. Cross, K. (1976). Accent on learning. San Francisco, CA: Jossey-Bass. Donovan, R. (1974). Alternatives to the revolving door: FIPSE National Project II. Bronx, NY: Bronx Community College. Dwinnel (Eds.), Developmental education: Preparing successful college students (pp. 85-96). Columbia, SC: National Research Center for the First Year Experience and Students in Transition. Gabriner, R. S., et al. (2007, February). Basic skills as a foundation for student success in California community colleges. Part 1: Review of literature and effective practices. Sacramento: The Research and Planning Group of the California Community Colleges, The Center for Student Success. Gardner, J. (1998, November). The changing role of developmental educators in creating and maintaining student success. Keynote address delivered at the College Reading and Learning Association Conference, Salt Lake City, UT. Goldrick-Rab. S. (2007, February). Promoting academic momentum at community colleges: Challenges and opportunities. CCRC Working Paper No. 5. New York: Columbia University, Teachers College, Community College Research Center. Grubb, W. N. (2001, February). From black box to Pandora s box: Evaluating remedial/developmental education. CCRC Brief No. 11. New York: Columbia University, Teachers College, Community College Research Center. Harris, J., & Eleser, C. (1997). Developing critical thinking: Melding two imperatives. Journal of Developmental Education, 21(1), 12-19. Higher Education Extension Service. (1992). The academic performance of college students: A handbook on research, exemplary programming, policies and practicies. New York, NY: Higher Education Extension Service, Teachers College, Columbia University. 33

Jenkins, D., & Boswell, K. (2002). State policies on community college remedial education: Findings from a national survey. (Technical Report No. CC-0201.) Denver: Education Commission of the States. Jenkins, D. (2007) Institutional effectiveness and student success: A study of high- and lowimpact community colleges. Community College Journal of Research and Practice, Volume 31, Issue 12 December 2007, pages 945 962 (18). Keimig, R. (1983). Raising academic standards: A guide to learning improvement (ASHE/ERIC Research Report # 1). Washington, DC: Association for the Study of Higher Education. Kulik, C-L., Kulik, J., & Schwalb, B. (1983). College programs for high risk and disadvantaged students: A meta-analysis of findings. Review of Educational Research, 53, 397-414. Kulik, C-L., & Kulik, J. (1986). Effectiveness of computer-based education in colleges. AEDS Journal, 19, 81-108. Kulik, C-L., & Kulik, J. (1991). Developmental instruction: An analysis of the research. Boone, NC: National Center for Developmental Education, Appalachian State University. Lamire, D. (1998). Three learning styles models: Research and recommendations for developmental education. The Learning Assistance Review, 3(2), 26-40. Levin, H. M., & Calcagno, J. C. (2008, January). Remediation in the community college: An evaluator s perspective. Community College Review, Vol. 35, No. 3, 181-207. MacDonald, R. (1994). The master tutor: A guidebook for more effective tutoring. Williamsville, NY: The Cambridge Stratford Study Skills Institute. Martin, D., & Arendale, D. (1994). Supplemental instruction: Increasing achievement and retention. New directions in teaching and learning. San Francisco, CA: Jossey-Bass. Martin, D., & Arendale, D. (1998, January). Mainstreaming of developmental education: Supplemental instruction and video based supplemental instruction. Paper at the Conference on Replacing Remediation in Higher Education, Stanford University, Palo Alto, CA. Maxwell, M. (1997). Improving student learning skills. Clearwater, FL: H & H Publishing Co. McCarthy, B. (1982, July). Learning styles and developmental education. Paper presented at the Kellogg Institute for the Training and Certification of Developmental Educators, Appalachian State University, Boone, NC. 34

Morante, E. (1987). A primer on placement testing. In D. Bray & M. Belcher (Eds.), Issues in student assessment (pp. 55-63). San Francisco, CA: Jossey-Bass. Morante, E. (1989). Selecting tests and placing students. Journal of Developmental Education, 13(2), 2-4,6. Muraskin., L. (1998). Best practice in student support services: A study of five exemplary sites. Washington, DC: U.S. Department of Education, Planning and Evaluation Service. National Center for Education Statistics. (1996). Remedial education at higher education institutions, Fall, 1995. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement. O Hear, M., & MacDonald, R. (1995). A critical review of research in developmental education, Part I. Journal of Developmental Education, 19(2), 2-6. Perin, D. (2006). Can community colleges protect both access and standards? The problem of remediation. Teachers College Record, 108(3), 339-373. Ramirez, G. (1997). Supplemental instruction: The long-term effect. Journal of Developmental Education, 21(1), 2-10. Roueche, J. (1968). Salvage, redirection, or custody? Washington, DC: American Association of Junior Colleges. Roueche, J. (1973). A modest proposal: Students can learn. San Francisco, CA: Jossey-Bass. Roueche, J., & Baker, G. (1983). Access and Excellence. Washington, DC: Community College Press. Roueche, J., & Wheeler, C. (1973, summer). Instructional procedures for the disadvantaged. Improving College and University Teaching, 21, 222-225. Roueche, J., & Kirk, R. (1974). Catching up: Remedial education. San Francisco, CA: Jossey- Bass. Roueche, J., & Mink, O. (1976). Helping the unmotivated student: Toward personhood development. Community College Review, 3(4), 40-50. Roueche, J., & Snow, J. (1977). Overcoming learning problems. San Francisco, CA: Jossey-Bass. Roueche, J., & Roueche, S. (1993). Between a rock and a hard place: The at risk student in the open door college. Washington, DC: American Association of Community Colleges, The Community College Press. 35

Roueche, J., & Roueche, S. (1999). High stakes, high performance: Making remedial education work. Washington, DC: American Association of Community Colleges, The Community College Press. Saxon, D.P., et al. (1998). Annotated research bibliographies in developmental education. Boone, NC: National Center for Developmental Education. Schwartz, W. & Jenkins, D. (2007). Promising practices for community college developmental education. New York: Columbia University, Community College Research Center. Spann, M.G., & Drewes, S. (1998). The annotated bibliography of major journals in developmental education 1991-1998, Volume 2. Boone, NC: National Center for Developmental Education. Spann, M.G., & Durchman, L.K. (1991). The annotated bibliography of major journals in developmental education. Boone, NC: National Center for Developmental Education. St. Clair, L. (1994/95). Teaching students to think: Using library research and writing assignments to develop critical thinking. Journal of College Reading and Learning, 26(2), 65-74. Stahl, N., Simpson, M., Hayes, C. (1992). Ten recommendations from research for teaching high risk college students. Journal of Developmental Education, 16(1), 2-4,6,8,10. Tinto, V. (1997). Classrooms as communities: Exploring the educational character of student persistence. Journal of Higher Education, 68(6), 599-623. Tinto, V. (1998). Learning communities and the reconstruction of remediation in higher education. Paper presented at the Conference on Replacing Remediation in Higher Education, Stanford University, Palo Alto, CA. Upcraft, M. L., & Gardner, J. N. (1989). The freshman year experience. San Francisco: Jossey- Bass. Weinstein, C. (1982). Learning strategies: The meta-curriculum. Journal of Developmental Education, 5(2), 6-7,10. Weinstein, C., & Rogers, B. (1985). Comprehension monitoring: The neglected learning strategy. Journal of Developmental Education, 9(1), 6-9, 28-29. Weinstein, C. (1988). Executive control process in learning: Why knowing about how to learn is not enough. Journal of College Reading & Learning, 21, 48-56. Weinstein, C., Dierking, D., Husman, J., Roska, L., & Powdrill, L. (1998). The impact of a course on strategic learning on the long-term retention of college students. In J. Higbee & P. 36

Zeidenberg, M., Jenkins, D., & Calcagno, J. C. (2007, June). Do student success courses actually help community college students succeed? CCRC Brief No. 36. New York: Columbia University, Teachers College, Community College Research Center. 37