Achieving the Dream. Washington State Community & Technical Colleges Phase II, Baseline Assessment Summary Report

Transcription

1 Achieving the Dream Washington State Community & Technical Colleges Phase II, Baseline Assessment Summary Report Bob Watrus & Deena Heg December 2012

2 Achieving the Dream Washington State Community & Technical Colleges Phase II, Baseline Assessment Summary Report Introduction An independent third party evaluation of Washington State s Achieving the Dream (AtD) Initiative funded by College Spark Washington is being conducted to document and evaluate the impact of the initiative on the 10 community and technical colleges that are participating during , provide timely feedback to colleges to help inform their efforts moving forward, and to document lessons learned and their implications for policy and practice. Questions guiding the evaluation are: Is AtD promoting institutional change? Are colleges creating a culture of evidence to help guide and shape efforts to improve student success? Are strategic interventions helping targeted students achieve key student outcomes/momentum points and progress on the pathway to obtaining certificates or degrees? Particular attention will be paid to issues of scale, intensity, and sustainability. Are colleges closing achievement gaps? During the first year of the evaluation, an evaluation framework was developed in collaboration with the State Board for Community and Technical Colleges (SBCTC), College Spark Washington, and representatives from the colleges, and baseline assessments were conducted regarding where colleges are in terms of institutional change, student success, and achievement gaps; where they are headed; and how they plan to get there. This included a review of colleges AtD applications, readiness assessments completed by the colleges when they joined the initiative, and planning year work plans; site visits and structured interviews and focus groups with faculty, staff, administrators, and students; and a review of colleges implementation plans and other AtD reports and documents. From all of this, profiles of each of the colleges were developed. The profiles include: An assessment of college starting points in terms of AtD s principles of institutional improvement: committed leadership; use of evidence to improve policies, programs, and services; broad engagement; and systemic institutional improvement. A summary of colleges AtD strategic interventions and anticipated results. Baseline Student Achievement Initiative data, provided by the State Board for Community and Technical Colleges. Opportunities and challenges. Over the next three years, progress against these profiles will be assessed, using Student Achievement Initiative data at the institutional and intervention level, annual AtD reports and other documents, and site visits and structured interviews. This report summarizes findings from the baseline assessment. 1

3 Washington s AtD Colleges Ten Washington community and technical colleges joined AtD in These colleges will receive grant funding from College Spark and supports from AtD through They represent a wide range of institutions: community, technical, and tribal colleges; urban and rural; large and small; transfer and work-force focused. For example, technical colleges have traditionally focused on helping students get jobs by providing them needed skills more than on completing certificates or degrees. Some community colleges focus primarily on academic programs, while others have a mix of academic and workforce programs. As a tribal college, Northwest Indian College is very rooted in community, place, family, and relationships. These different institutional cultures and characteristics all have implications for how colleges approach AtD, institutional change, increasing student success, and closing achievement gaps. In conducting the evaluation, it will be important to take this into account. The 10 colleges are: Bellingham Technical College, Clover Park Technical College, Edmonds Community College, Everett Community College, Grays Harbor College, Lower Columbia College, Northwest Indian College, Skagit Valley College, Spokane Falls Community College, and Whatcom Community College. 2

4 AtD principles of institutional improvement Committed Leadership Senior college leaders actively support efforts to improve student success, not just to increase enrollments, and are committed to achieving equity in student outcomes across racial, ethnic, and income groups. Administrators, board members, and faculty and staff leaders demonstrate a willingness to make changes in policies, programs, and resource allocation to improve student success. Several institution-level developments have helped to lay the groundwork for this round of AtD in Washington State: All AtD colleges are participating in the Governance Institute for Student Success (GISS), which focuses on best practices in effective governance that promote student success and college completion. GISS has provided presidents and trustees the same conceptual framework, language, and best practices knowledge that is part of AtD, helping them to be well-equipped to support and lead their colleges in the transformational work they are engaged in through their AtD participation. Current accreditation requirements from the Northwest Commission on Colleges and Universities (NWCCU) that are focused on core themes, strategic directions, and key performance outcomes and indicators. Many of the participating colleges have Title III/Strengthening Institutions federal grants. This program is aimed specifically at increasing college capacity to serve low-income students by improving academic quality, institutional management, and fiscal stability. The development of the Student Achievement Initiative database, which has sharpened the focus on both student outcomes and progression, as well as the related research studies conducted by or commissioned by the State Board for Community and Technical Colleges. The SAI measures provide the colleges with common performance measures that can be used as a lens through which to assess all student success initiatives. Many of these colleges have had presidential changes since Some are new to their colleges; several have previous ties to and/or familiarity with their campuses. These new presidents have fully embraced and are committed to AtD as a way to move forward on student success. For the most part, the presidential transitions do not appear to have had any negative impact on the AtD work, partly because the college community was already engaged in AtD and partly because the presidents came in supporting such activity. These 10 presidents are data-savvy and believe in the use of data to improve student outcomes. Many come from academic or workforce backgrounds where the use of outcomes data is common and valued. Some are personally involved in specific initiatives designed to improve student achievement for example, projects that are tailored to increase success for certain groups that have struggled to succeed, such as rural students or at-risk youth. Most see AtD as providing a broad strategic framework or umbrella to help focus and advance institutional change efforts already underway. Some presidents are more directly and intensively involved in AtD than others for example, attending many core team meetings and being directly engaged in strategy development in this planning year while others have designated strong leadership teams to carry out the AtD work. Several of these leadership teams include directors of offices of institutional effectiveness or strategic planning or some other high-level planning functions, which report directly to their presidents and are engaged in aligning and tracking outcomes for strategic plans, accreditation, major grants such as Title III, and AtD. The leadership teams typically also include vice presidents of instruction and 3

5 student services, institutional research (IR) directors, and a mix of others who are engaged in broad institutional improvement initiatives. AtD asks colleges, as part of their readiness assessment in the leadership area, to look at how their visions, mission, strategic plans, and other similar guiding components address student success, achievement gaps, and other key components of AtD. Virtually all of the vision and mission statements for these colleges now include a student success component. As their strategic plans have been updated during the five-year cycle, they have added a new emphasis on student success in addition to more traditional aspects such as access and equity. All 10 colleges have stated that they plan to align their strategic plans with accreditation core themes and measures and with AtD. Many have already done so. Several colleges have specifically addressed multicultural issues and equity across all groups. For example, several have or plan to have multicultural, equity, or diversity offices, centers, coordinators, or work groups. For many colleges, the increased focus on student success requires considerable work on changing institutional culture, since access has been so highly valued as an equity strategy for so long in the community/technical college world. The culture shift to a focus on student success, as measured by completion, can be a particular challenge for colleges and programs with a workforce focus, which have traditionally defined success as students getting jobs, regardless of whether they complete their programs. Leaders in these colleges find themselves working to gently persuade instructors on the importance of having students complete credentials before leaving to take a job so that they have better access to promotions and mobility, and at the same time, on the value of having instructors work towards their own postsecondary credentials, as many have been primarily practitioners. These are significant cultural shifts within colleges that take some time and care to achieve. The leadership teams appear to be energized and eager to support changes in policies and procedures designed to improve success. Again, they often cite the broad umbrella of AtD as a way to justify acting on changes. In some cases, strong leaders had already taken some major steps towards what they saw as needed reforms, including merging high level positions to remove organizational silos, revamping approaches to student services, reviewing policies and procedures, and making changes to practices identified as problems such as late registration. Some colleges have also already changed their resource allocations to support institutional research (as detailed in the next section) and strategic interventions (e.g., pre-college math reforms). Use of Evidence to Improve Policies, Programs & Services The college establishes processes for using data about student progression and outcomes to identify achievement gaps among student groups, formulates strategies for addressing the gaps identified and improving student success overall, and evaluates the effectiveness of those strategies. At the beginning of this AtD round, most of the colleges reported using various kinds of data in annual program reviews, accreditation reporting, strategic plan reporting, grant reporting, and the like. The kind and specificity of data varied considerably from project to project and college to college, although many said that the NWCCU accreditation requirements were leading them to develop more targeted and measurable indicators. A few colleges routinely went well beyond minimum reporting requirements in their use of data, developing or adapting tools for analysis and reporting, and looking at retention, completions, course progression, and similar measures. Several were already engaged in longitudinal cohort tracking for various purposes. For most colleges, data were reported in different programs, departments, and divisions, with little institution-wide data sharing, analysis, or evaluation. Data use in reports to Boards of Trustees was similarly variable, and most colleges noted that this needs to be increased. There is still some concern from faculty at various colleges that data may be used to punish rather than to improve and thus the need to continue to make the case for the value of using evidence to improve student outcomes. 4

6 Most colleges were well able to produce the quantitative data requested by AtD for use during the planning year. A couple of the colleges struggled with this and needed much longer time periods and some outside help to get it done. The majority of the colleges conducted in-depth quantitative analyses that looked at retention, course completion, persistence, and progression, disaggregated by race/ethnicity, gender, age, family status, full and part time, and a variety of other characteristics. These findings were presented at AtD kickoff meetings and were often followed by small group discussions about what the findings meant for student success and the beginnings of generating ideas about how to address achievement gaps shown by the data. All of the colleges collected qualitative data as well. Prior to ATD, many colleges used the Community College Survey of Student Engagement (CCSSE), and several also conducted their own student satisfaction inventories tailored to their particular interests, needs, and student bodies. A few colleges have also undertaken telephone interviews with students who left college to understand the barriers they faced as well as the kinds of support that might have helped them stay, and some have done interviews with successful students to understand what aspects of college life worked best for promoting success. Focus groups were frequently conducted as part of the AtD planning year. Some already had the capacity to do this. Some arranged to get focus group training and then train others so that they could continue to conduct such groups as needed. Others brought in outside experts to conduct the groups. A couple of colleges had also recently engaged in major strategic planning efforts that included multiple, often large community events, and were able to take advantage of these by generating discussion about student outcomes data and possible strategies for improvement. Most colleges continued those discussions and took deeper looks into the data at AtD core and data team meetings through the planning year, and used the findings to guide strategy and intervention development. Their individual implementation plans discuss their data in detail and explain how their findings led to the development of specific strategies and interventions. The colleges have also become accustomed to the use of Student Achievement Initiative momentum points. As part of a system goal to increase educational attainment in Washington State, the State Board for Community and Technical Colleges uses a research-based performance funding system called the Student Achievement Initiative, which provides financial incentives to colleges for student gains in basic skills, pre-college, earning 15 and 30 credits, completing college level math, and completing degrees, certificates, or apprenticeship training. These points are tracked for all colleges in the state system, and will be used to track AtD performance, in addition to other measures that individual colleges choose to add. The SBCTC is providing baseline data for AtD and support to the AtD colleges as they develop methods for tracking students who participate in interventions. There is also frequent collaboration among institutional research (IR) professionals throughout the community and technical college system, both formal and informal, and this has resulted in the sharing of many tools and skills. Many colleges have increased IR capacity within the last year or two---some by a little but others by quite a lot. Some have added permanent IR staff, at a variety of levels---director, research analyst, and/or program assistant. Several colleges have worked with a consultant to create the tools they need to conduct AtD and similar data analyses, and others have purchased some of what they need. Some of these newly acquired tools include operational data stores, the development of SQL queries to answer specific questions about student outcomes, data intranet sites that give access to outcomes data to faculty and staff, new software for managing student data and for data reporting, and the like. A few colleges have provided release time to faculty to participate in AtD data analyses and discussions. For a couple of colleges, an IR function was only added within the last few years and they are still building the infrastructure of the office. There is also turnover in IR and IT staff. Some of the positions have taken months to fill. Other colleges have made significant progress in stabilizing their IR staff and improving the quality of college data. IT capacity varies considerably from college to college. A few colleges say that they have sufficient IT capacity to support their IR work. Others are working on improving the intersection between IR and IT and believe they are 5

7 moving in the right direction. And a few report that lack of IT resources is a real barrier, with no clear fix in sight; for help they must go either to the SBCTC or their IR colleagues in other schools. A college may be able to develop an operational data store but they do not have a database administrator to maintain it. At a couple of schools, they continue to work on improving data integrity, since inconsistent coding in the past had yielded data of questionable validity and resulted in faculty mistrust of the data. The colleges are accustomed to evaluating individual projects, since many grants require an evaluation component. Most say that previous evaluation efforts have been piecemeal in response to specific project requirements, and that they would like to develop a broader, more comprehensive and coordinated approach to evaluating interventions in a way that allows them to develop, test, and prioritize effective work. Several colleges are using some of their AtD funding to strengthen their capacity to use evidence (see next section on Strategic Interventions). Some colleges presented well-developed evaluation plans for their AtD interventions, while others have struggled with this. A couple had not yet fully developed their evaluation plans even as the interventions were being launched. The consolidation of strategic planning, accreditation, grant reporting, AtD, and many other research functions into the same office while considered to be a systemic improvement has increased the demands on institutional research offices; thus, even as the colleges have increased IR capacity, they have also increased the demands on these offices. While some colleges say they have enough IR capacity, many report being stretched thin. College leaders are aware of this, and there is much discussion about the need to become more strategic in their approach to research and data, setting priorities and crafting directions for research, and reducing the demand on IR to respond to individual requests or to conduct administrative functions that others could do. The development of tools that faculty and staff can use themselves and providing them the training to use those tools can help if the tools and training are disseminated throughout colleges in an effective way. There is considerable interest at these colleges in improving college-wide communication and outreach about data, research findings, AtD intervention results, and other student success efforts. Broad Engagement Faculty, student services staff, and administrators share responsibility for student success, and collaborate on assessing the effectiveness of programs and services and improving them. Other stakeholders with influence on student success (K-12 systems, community groups, employers, etc.) are included in discussions about student performance, desired outcomes, and potential improvement strategies. The college also gains invaluable insight about ways to improve student success from students themselves through surveys, focus groups, and/or advisory councils. All of the colleges have traditionally had faculty and staff involvement on a variety of committees, councils, and other participatory groups that work on annual program reviews, strategic planning, accreditation, curriculum and instruction, and the like. Looking at data on student outcomes, identifying achievement gaps, and trying to formulate campuswide responses are relatively new activities for most colleges. All of them conducted all-campus AtD kickoff events where data on student success were presented and discussed. As part of the AtD planning year, they all have engaged faculty, staff and administrators in planning and developing their AtD implementation plans, on core teams, data teams, and strategy teams. Some have also made a point of involving faculty and staff who work on related student success efforts. This not only serves to expand the number of faculty and staff involved in AtD, but also recognizes the good work already underway at the colleges, builds on that good work, and helps to address the issue of AtD just being another initiative rather than a way to create deeper institutional change. 6

8 Some colleges report that alignment and collaboration among faculty, student services, and administrators is a particular strength for them. Others say that this is an area they continue to need to work on, and a couple of colleges are still struggling with how to best improve working relations between instruction and student services. All of them report that it is challenging to find practical ways to involve part time and adjunct faculty. Some of the ways in which colleges are working to increase engagement and cross-function collaboration include: Large core and data teams with a mix of faculty, staff, and administrators Core, data, and/or strategy teams officially co-led by faculty and student services Active recruitment of faculty and staff who are known to be innovators and enthusiasts to participate on various AtD teams Use of AtD or institutional funds to pay part time and adjunct faculty to participate in AtD, to attend the DREAM Institute, and/or to attend professional development events on campus AtD team member presentations at faculty meetings, department meetings, or brown bag sessions All faculty and staff participating in student orientation Faculty incentives to conduct research projects on increasing student success All of the colleges also have successful previous experience in working with external stakeholders for specific purposes, such as professional-technical advisory committees, resource development, and strategic planning. Again, the topics of student outcomes, what the data show about student achievement, and having student success as an explicit goal in addition to access, are recent additions to some of the community conversations. The colleges included K-12, community groups, local employers, and in some cases four-year institutions in their AtD kickoffs. Some, as mentioned earlier, were able to include these issues in large community engagement events that were held in the last few years for strategic planning purposes. A couple of the colleges have made a particular effort to conduct outreach to local churches, nonprofits that represent communities of color, and other community groups that have a particular interest in helping to close achievement gaps, and have enlisted these partners in helping to lead larger conversations in the community to help move change at their colleges. With respect to student engagement, the colleges conduct regular student surveys such as CCSSE as well as homegrown surveys that investigate college-specific issues of interest. AtD has provided a more focused use for these surveys with its emphasis on understanding and addressing barriers to and supports for student success. Some colleges have done student focus groups in the past, and students also participate in a variety of committees, councils, and student government positions. Many have learning communities, service learning, and other activities designed to strengthen student engagement. Most say that they would like to get better at involving students in AtD and in broader efforts to increase student success beyond the use of surveys and focus groups. Some colleges have recruited student representatives to participate on AtD teams and on institution-level student success committees. Several colleges have developed AtD interventions that address engagement specifically. Some of these are highlighted in the following section that summarizes intervention approaches. Continuing and broadening engagement as AtD moves forward is an area where colleges say they would welcome continued counsel and advice. 7

9 Systemic Institutional Improvement The college establishes planning processes that rely on data to set goals for student success and then uses the data to measure goal attainment. The college regularly evaluates its academic programs and services to determine how well they promote student success and how they can be improved. Decisions about budget allocations are based on evidence of program effectiveness and are linked to plans to increase student success. Faculty and staff are afforded professional development opportunities that reinforce efforts that help to close achievement gaps and improve overall student success. Many colleges have started to integrate AtD with other efforts to create a broad, institutional wide student success agenda. This includes: Linking AtD to other institutional improvement efforts (e.g., strategic planning, accreditation, etc.) Expanding the scope of existing entities to focus on student success (e.g., creating student success committees) Incorporating other existing student success programs/projects (e.g., developmental math and English, etc.), as part of AtD The colleges reported that they are in the process of aligning and integrating their strategic plans, accreditation approaches, and AtD. These activities include adding student success as a key goal, and developing indicators to measure success that can be used in reporting for all three functions. Some have made their strategic plan goals and accreditation themes identical. Others have consolidated these functions and teams through an Office of Institutional Effectiveness or some similar structure. Most colleges say they are interested in having a broad student success committee to oversee all activities related to student success, and a few have already done this. Some are considering having their AtD core teams take on this function. Many are still in the process of determining their indicators for accreditation and strategic plan goals, and how they will collect and use the data on these indicators. Measures often mentioned included retention, course completion and progression, certificate and degree completion, and the like. All of the colleges have already been conducting a wide variety of programs, initiatives, and projects that are meant to increase the likelihood that their students will succeed, prior to AtD. Such programs include I-BEST (integrating basic skills and job skills training), I-TRANS (moving students through math or English faster, with instructional and advising support and the earning of college credits), TRIO, significant redesigns of precollege math and English curricula and instruction, early alert systems, learning centers, Opportunity Grants and other programs that include case management/coaching, mentors, Title III grants, and other similar efforts to help low income students and students of color succeed. Some of the colleges have designed one or more of their AtD interventions to build off this existing good work when it has appeared to be particularly effective. A few have formally incorporated other existing success work as part of creating a larger base for the umbrella of AtD, such as the precollege math or English redesigns. Most colleges said repeatedly that they are anxious to use AtD to help them develop systemic, institution-wide approaches and measures for assessing the student success-related work they do so they can allocate scarce resources where they will be most effective. At this point, the ways in which colleges assess effectiveness can vary widely. Measures used to assess programs, initiatives, grants, and so on differ considerably, as do institutional responses to evaluation results. Some activities are evaluated; others may not be. And evaluations are often driven, not by the institution, but by the funder. The measures used can be so disparate as to make comparison and prioritization on a broad institutional level not possible. 8

10 Most colleges do not yet have strong mechanisms for following up on evaluation results and considering them in a broad campus-wide context. But the interest in doing so is very high, and one striking finding from the baseline assessment site visits and interviews was the presence in almost every college of some high-level strategic thinkers who are centrally and deeply involved in AtD as well as other student success work. These leaders are very conscious of the value in developing ways to look at all of their student success-related work in an integrated fashion, assess the work using common measures, and think strategically about what practices are most effective and how to institutionalize those. Most of them believe that the establishment of institution-wide measures of success is an important step in being able to justify changing budget allocations using a data-based approach that is clearly understood by all. Professional development activities that provide support for closing achievement gaps and increasing student success are also variable, not only across colleges but from department to department within them. Most colleges cited professional development as an area of relative weakness within the context of AtD. Of course this does not reflect lack of activity or interest. Partly, the traditional definition of academic freedom results in approaches to professional development that vary based on individual, departmental, contractual, and college policy interests, preferences, clauses and beliefs. At some colleges, it is a matter of policy or contract that attendance at campus professional development events is not required, and sometimes only deans and department heads will go. Also, professional development is often first on the chopping block in times of high resource constraint, again not because it s viewed as unimportant but because it is often an optional item compared to other needs. Some colleges have had all-campus professional development events that are related to their efforts to improve student outcomes. Examples include all-staff events on a framework for understanding poverty. Others are planning to provide training to faculty and staff on using data to understand and improve student outcomes. And several have incorporated professional development into AtD interventions, as described in the following section. It is an area, though, where most colleges would welcome improvement opportunities. 9

11 AtD Strategic Interventions Washington s 10 AtD colleges are implementing about 30 strategic interventions aimed at increasing student success and closing achievement gaps, based on work done during the planning year (e.g., gathering and analyzing data on student outcomes; engaging faculty, staff, and students in the process; setting priorities, goals, and outcomes; and developing strategies). Most of the interventions fall into the following categories: first year experience, pre-college/ developmental education, changes in instructional practices, institutional research capacity, and coaching support. As part of their AtD implementation plans, colleges have developed or are continuing to develop plans for evaluating their strategic interventions. In addition, Student Achievement Initiative data will be analyzed at the intervention level (as well as the institutional level) to assess progress over time. Selected examples of AtD interventions are described below to illustrate the range of approaches that are being taken. First Year Experience Almost all colleges are implementing strategic interventions that address students first year experience. In general, the aim is to better prepare students for their college experience and as a result increase retention during the first quarter and first year. Elements of colleges first year experience interventions include: New student orientation Advising (entry and academic) College success courses Early alert systems The colleges approaches vary considerably. Some focus on one or two elements; others contain several elements. Some colleges have made certain components mandatory but not others. Several colleges are piloting their efforts before making them mandatory. Some target all new students; others, specific populations (e.g., those testing at precollege levels). There are interventions that focus primarily on the first quarter, while others include elements that extend throughout the first year. For example, Grays Harbor s mandatory extended orientation is required for all first time students. It consists of four days of mandatory and optional workshops the week before classes start. Workshops cover topics such as financial aid, e-learning, expectations of college, computer literacy, resources, and peer support. In addition, three day math and/or English review workshops are also offered (but not required) that aim to prepare students to retest for possible advancement into a higher pre-college course. Also included are campus tours and meetings with advisors. Focus groups held with students, faculty, and staff who participated in the first mandatory extended orientation conducted as part of this baseline assessment in October 2012 found mostly positive reactions. Students reported they learned about college resources and expectations. They said that it helped them with the transition to college. One student reported being terrified about going back to school and said that orientation really helped her contend with this. Another reported it made going back to school much less intimidating. Students also said that the orientation activities helped them to make connections to other students as well as faculty and staff. Almost all 10

12 faculty and staff participated in orientation and also had positive reactions to it. The students, faculty, and staff were energized by and enthusiastic about the orientation experience. Skagit Valley s new student orientation and registration intervention includes not only orientation before the start of the quarter, but follow up advising during the first year. This includes one-on-one assistance with registration and course selection, so that students leave with that quarter s schedule in place; sessions with assigned advisors to guide students in their educational planning, resulting in creation of an educational plan and a date for follow up contact; and continuing support and assistance, including follow-up phone calls/ s from advisors during the first three quarters and quarterly meetings to review educational and class plans. Northwest Indian College s first quarter holistic advising intervention also provides students support that extends beyond the start of school. It creates advising teams of core faculty, student services staff, and campus connectors (people on campus selected for being good at reaching out and communicating and trained to provide help and linkages) to provide all first quarter students ongoing, direct contact for making connections, addressing needs, and helping them to develop college and campus navigation skills. This is in addition to new student orientation that is required of all new students and provides information on college processes and resources. Whatcom s first year student success intervention incorporates multiple elements, including new student orientation and a college success course. It scales up new student orientation (and makes it mandatory by the fourth year), expands its college success course (and develops and delivers related faculty training); and develops more comprehensive first year support systems and programs. It also includes taking a broader look at related college policies, procedures, and practices that particularly impact the first year, as well as best practices. Pre-College/Developmental Education Many colleges are implementing strategic interventions that reform pre-college or developmental education, with the goal of increasing student completion rates in pre-college and college level math and English. Elements of colleges pre-college/developmental education interventions include: Examination of existing institutional policies, practices and procedures (e.g., testing and placement) Curriculum and instruction changes (e.g., fewer pre-college levels and modularized, self-paced curriculum) Tutoring and other student supports Faculty professional development Lower Columbia s pre-college intervention is comprehensive and focuses on reducing the number of students having to take pre-college courses, increasing the success rate of those who do enroll in pre-college courses, reducing the amount of time it takes to move from pre-college to college level courses, and increasing the success rate of students who move on to the college level. Its pre-college math reform incorporates new diagnostic testing, a move from four to three levels of pre-college math; a self-paced, modularized curriculum; and a math achievement center that offers one-on-one help. There is also is a math boot camp, an intensive workshop that provides a brush up on math skills, after which students can retest, making it possible to jump pre-college levels. Its pre-college English reform features a bucket course in which all students testing into pre-college are initially placed, where students can make accelerated progress. Clover Park s pre-college math intervention focuses on redesigning the curriculum into modules, with each module focused on a core concept required for success in college math. Students will complete only the modules they need, accelerating their movement through pre-college math. Pilot classes will also include comprehensive assessment at the start, pre and post module tests, and instructors, tutors, and online tutorials. 11

13 Whatcom s pre-college math and English success intervention includes examination of placements exams and scores as well as alternative methods of placement, curriculum revisions, embedded study skills or lessons around student attributes, new pathways to college level math and English courses, and faculty professional development. Changes in Instructional Practices Some colleges are implementing changes in instructional practices, with the aim of increasing transition rates from pre-college to college level coursework and increasing completion rates in high enrollment, low completion or gatekeeper courses. Elements include: Instructional changes (e.g., supplemental instruction, reading apprenticeship, and contextualized learning opportunities) Student supports (e.g., peer mentoring) Faculty professional development Bellingham s supplemental instruction intervention targets gatekeeper, pre-college, and degree programs with high enrollment, low completion rates for pre-college transitioning students, particularly those in science, math, and allied health. It will be mandatory for students who have retaken a course or have below a C GPA or on tests. It is meant to help pre-college students transition more successfully into degree programs that are high demand and early attrition. In addition, Bellingham s reading apprenticeship intervention provides professional development to faculty in basic skills, developmental education, and some certificate and degree programs with low success and high attrition in terms of pre-college transitioning students. Reading apprenticeship is a reading comprehension approach that uses social, personal, cognitive and knowledge-building elements to teach students how to better analyze and think about what they read and its meaning. Edmond s high enrollment, low completion courses intervention targets specific math, science, social science, and humanities courses with the aim of improving placement and success in these courses. It includes a review of placement scores, prerequisites, and course objectives to ensure proper placement and preparation for future courses; faculty engagement through professional development; more contextualized learning opportunities and supplemental instruction; and peer to peer mentoring for students. Whatcom s gatekeeper course success intervention focuses initially on general psychology and on professional development for faculty teaching this course. General psychology has some of the highest enrollments at the college, but a lower completion rate than the department s target. Faculty will gather quantitative and qualitative data; identify factors impeding student success; and review promising practices. Based on this work, two strategies for improving success rates will be designed and implemented. Plans for scaling up include sharing best practices with other faculty teaching context heavy, survey, academic courses. Institutional Research Capacity Some colleges are implementing strategic interventions to increase their institutional research capacity. Elements include: Additional IR and/or IT staff Data tools and products 12

14 Faculty and staff engagement Everett s institutional research intervention aims to strengthen the college s culture of evidence and broaden faculty and staff engagement in the use of data to improve student success. It includes disseminating data to faculty and staff to help familiarize them with using data to understand problems and generate solutions; providing micro-grant funding using a research framework to help structure evidence-based thinking; help with designing evaluations for the funded interventions; having the AtD priority group assess which interventions merit follow-up or scaling up; and using a standardized report format and disseminating reports campus wide, to provide faculty and staff with finished work that can be used and to create an institutional memory about what works that can continue to fuel new ideas. They plan to do pre and post surveys to measure changes in faculty and staff use of data. Lower Columbia s institutional research intervention includes hiring a full time IR data manager to produce ongoing data reports and respond to data requests; increasing IT capacity for data and data accessibility; developing a data coach or data liaison in each department to create a more grassroots culture of evidence; and offering a Data 101 course for faculty and staff. Edmonds campus understanding and support for student equity and success intervention includes improving IR (and IT) capacity by automating collection and dissemination of data (e.g., converting legacy system reforms into Microsoft SQL reports accessible online and providing web access to data). Coaching & Advising A couple of colleges are implementing strategic interventions that provide coaching support to students, with the goal of increasing first year retention rates. One college is providing stand-alone coaching; another is embedding it in the classroom. Another college is aiming to create an entirely new academic advising system with a significant professional development component. Grays Harbor s coaching for success intervention provides students who test more than one level below college level in two or more areas a coach/advisor who provides them various kinds of support: help developing educational plans, help understanding how to navigate the college, referrals to support services, intrusive advising, and life skills training. The coach also works with students on the challenges of poverty and staying in school; and tracks student academic success. Clover Park s intervention embeds coaching in a pre-college English course, as a pilot. Coaches will increase student awareness of campus services and provide students with study skills, test taking techniques, financial aid and literacy information, and ongoing advising and educational planning. The coaching will continue until students complete their first college level English course. If the pilot is successful, coaching will be scaled up by expanding it to additional sections of pre-college English and/or professional technical programs. Targeted would be programs with historically low completion rates. Modules would be added to address areas that have been barriers to completion (e.g., personal development and career success). Spokane Falls academic advising intervention proposes the creation of a new academic advising system that includes mission, outcomes, program goals, standardized practices, assessment, and a significant professional development component, based on current best practices in the field. Its target groups are pre-college and undeclared students, and its goals are to increase fall to fall retention and degree completion for advised students and to decrease the gap between student perceptions of the importance of academic advising and their satisfaction with it. 13

15 Emerging Issues Key issues that emerged from the baseline assessment include: Institutional change AtD colleges have approached institutional change in a variety of ways, reflecting their particular cultures, values, and personalities top down, bottom up, or some combination of the two. In terms of top down, many colleges have active, engaged presidents and others in key leadership positions committed to student success and a culture of evidence. Some of these leaders see AtD as helping to advance changes already in the works. In terms of bottom up, many colleges have faculty and staff who are actively engaged, driving changes in instruction and student services. Faculty members at several colleges are spearheading innovative reforms in pre-college math and English, with a great deal of passion, commitment, and energy. Said one math faculty member at an AtD college, We saw students weren t being successful, so we revised it. And student services staff are likewise leading efforts to redesign college success courses, new student orientation, and advising. Regardless of the starting point, it is important to take a conscious, strategic approach to build, encourage, and increase buy in and engagement at all levels top, bottom, and middle. Where there have been gaps or initial reluctance to participate, whether it has been at a relatively high administrative level or at the faculty or staff level, the few colleges where this has surfaced appear to be finding creative workarounds. Colleges are encouraged to persist at trying to find ways to engage those who have yet to get on board to be part of some aspect of AtD and/or other student success efforts. Achievement gaps and equity AtD is about increasing student success and closing achievement gaps, and attention needs to be paid to both. This includes looking at the impact of strategic interventions not only on student success at a broad level, but also specifically for the groups who experience significant gaps in achieving success at college. In analyzing student outcome data during the planning year, most AtD colleges identified some achievement gaps, particularly among Black/African American, Hispanic/Latino, Native American, and low income students. Many of the colleges strategic interventions focus on increasing student success across income and racial/ ethnic lines, and at the same time, intentionally target issues that disproportionately affect students of color and low income students (e.g., pre-college math and English, and high enrollment, low completion courses). Will this be enough to close achievement gaps? Or will more targeted interventions be required? Answering these questions will require intervention evaluations that pay particular attention to their impact on achievement gaps. National AtD research suggests the importance of commitment and attention to achievement gaps and equity, along with student success, and to using data to make sure that such gaps are closing or to make course corrections if they are not. Changes in instructional/classroom practices Most AtD colleges are implementing changes in their instructional and classroom practices, unlike earlier rounds of Achieving the Dream elsewhere. For many colleges, this takes the form of pre-college/developmental education reform, with a wide range of changes being implemented (e.g., testing and placement; brush up sessions; fewer pre-college levels; modularized, self-paced curriculum; cohorts; and tutoring and other student supports). Some colleges are also targeting high enrollment, low completion or gatekeeper courses, with changes including supplemental instruction, Reading Apprenticeship, and contextualized learning opportunities. At least one college is embedding college success skills in the classroom. All of this is in line with national AtD research that highlights the importance of changes in instructional and classroom practices (in part, as a way to achieve scale and sustainability); strengthening pathways into and through programs of study; faculty driven reforms; and department wide change. This last point speaks to the 14

16 importance of professional development at the departmental and/or college level, not just the individual level. Some colleges have built professional development into their interventions. Culture of evidence AtD colleges have already made significant progress in building data capacity and they are doing this in different ways (e.g., additional IR and IT staff, faculty release time, consultants, and data coaches/liaisons at the departmental level). They have also developed new products and tools (e.g. operational data stores). Some have started to shift away from one time reports in response to specific request to ongoing, strategic use of data. While these improvements in capacity here are encouraging, it is important to note that colleges continue to report constraints in certain areas, including limited IR-IT collaboration (in part because of limited IT capacity); difficulties in developing and maintaining useful tools such as operational data stores; and the increased demands on IR as colleges consolidate their data and research functions. Broad engagement National AtD research highlights the importance of broad engagement in adopting and sustaining change. All 10 colleges have actively engaged faculty, staff, administrators, and students in their AtD planning, development, and implementation efforts. This has ranged from all-college forums or summits exploring student outcome data and their implications, new student orientation involving all faculty and staff, and strategy teams developing and implementing strategic interventions. Specific challenges include: - Maintaining the engagement of faculty and staff already involved in AtD, as well as broadening and deepening engagement. - Creating effective communications strategies and tools, as one way to keep those already involved engaged and up to date, and broaden engagement. - Engaging colleges extended sites, an issue for a number of colleges (e.g., Northwest Indian College, Skagit Valley, and Grays Harbor). The colleges may want to think about how to use their core and data teams over time to take on some of these challenges. Broad, systemic approach to student success Several AtD colleges are taking steps toward a broad, systemic approach to student success, rather than seeing AtD as yet another initiative. Ways to promote this broad, systemic approach include: - Creating student success committees. Grays Harbor College, for example, has restructured its strategic planning committee, which is now the student success committee charged with helping to focus the college s planning on student success. It combines the college s strategic planning, accreditation, and AtD committees. - Using common outcome measures for student success. Using common outcome measures for AtD, strategic plans, and accreditation makes it possible to assess student success in a systemic way and evaluate the effectiveness of efforts to improve student success. - Leveraging, building on, and/or integrating AtD with other student success efforts. For example, Whatcom s and Lower Columbia s AtD implementation plans incorporate and build on existing efforts at reforming pre-college/developmental education; and Edmonds plan builds on existing college success courses for specific populations/programs in designing a college success course as part of its first year 15

17 experience intervention. This approach also serves to broaden faculty and staff engagement by recognizing and building on prior good work. National AtD research highlights the importance of systemic change to improve student success and use of common performance measures to enable colleges to assess the effectiveness of their student success work and to be able to prioritize them for the purposes of scaling up and of making the best decisions for budget allocations. Relationship building and sense of community Students participating in focus groups conducted during site visits to the AtD colleges as part of the baseline assessment spoke to the importance of building relationships with other students (and especially the value of student mentors) faculty, and staff; and having a sense of community to student success. A number of strategic interventions being implemented have elements of relationship and community building (e.g., Northwest Indian College s campus connectors, Grays Harbor s pre-college math cohorts, Edmonds peer mentoring; and Clover Park s coaching support). This may be an area for further exploration in terms of different approaches, outcomes, and implications. Student financial hardship Another issue students raised in focus groups was financial hardship and its impact on student success. This was especially true of colleges serving low income communities and populations. Approaches colleges are taking or could take to address this issue include: - Providing students in depth information on available resources (e.g., federal financial aid, Opportunity Grants, Basic Food Employment and Training, Worker Retraining, etc.). Financial aid is often the focus of workshops conducted as part of new student orientation. For example, Grays Harbor students who participated in the college s new mandatory extended orientation said the financial aid sessions on the Free Application for Federal Student Aid (FAFSA) and financial aid and scholarship programs were particularly helpful. - Providing students help applying for financial aid, including scholarships. For example, Northwest Indian College is embedding scholarship writing and submission activities into spring English courses. This in response to the finding that one of the college s major leakage points is after the first year due to students financial needs. - Reviewing college policies and practices for their impact on low income students and identifying ways to address those having a negative impact (e.g., finding alternatives to students having to spend hundreds of dollars on textbooks rarely used in class; and providing access to computers, printers, and the Internet, etc.). - Providing referrals to campus services and supports, which is often part of coaching and advising. - Partnering with community based organizations that can provide needed services and supports. Shared learning The fact that multiple AtD colleges are implementing strategic interventions in the areas of first year experience, pre-college/developmental education, changes in instructional practices, institutional research capacity, and coaching support and are taking different approaches to these issues presents a great opportunity to share across colleges some of the lessons being learned about what works and doesn t work, why, and what the implications are moving forward in each of the areas. College Spark Washington and the State Board for Community and Technical colleges can help promote this shared learning through the regular convening of the AtD colleges, perhaps forming learning groups in selected areas. 16