SCALING COMMUNITY COLLEGE INTERVENTIONS

Transcription

1 CUTTING EDGE SERIES SCALING COMMUNITY COLLEGE INTERVENTIONS No. 2 Ctting Edge Series: Scaling Commnity College Interventions 1

2 Dear Colleages: Achieving the Dream is pleased to partner with Pblic Agenda to offer yo this important pblication, Scaling Commnity College Interventions. Second in a three-part series, this practical gide addresses specific, common challenges commnity colleges face. The origin of this series is the recent interim report by MDRC and the Commnity College Research Center called Trning the Tide: Five Years of Achieving the Dream in Commnity Colleges. The interim report identified areas of great progress as well as aspects of Achieving the Dream s work that need deeper focs. The report conclded with recommendations for next steps, specifically paying more attention to scaling promising initiatives to reach more stdents. Achieving the Dream designed this series with or fonding partner, Pblic Agenda, to address those challenges and ensre that every Achieving the Dream instittion has the tools necessary to move the needle on stdent sccess and completion. This particlar gide provides an overview of the most common barriers to scaling, principles to help position commnity colleges for sccessfl scaling, and a checklist of critical qestions to gide each step of the way. Overall, this new gide shold help commnity college leaders better anticipate and address the roadblocks to sccessfl scaling. On behalf of the entire team at Achieving the Dream, I d like to extend my appreciation to Pblic Agenda for their diligent and thoghtfl work on this timely series so far, and my best wishes to each instittion in prsit of greater stdent sccess otcomes. Sincerely, William E. Treheart President & CEO Achieving the Dream Ctting Edge Series: Scaling Commnity College Interventions 2

3 How to Use This Gide This is the second gide of the Ctting Edge Series and is designed to help colleges apply strategies that will allow sccessfl instittional change and stdent achievement initiatives to reach more stdents. Section 1 of this gide lays ot the most common obstacles to sccessfl scaling and serves as a sobering reminder of the complexity of the scaling challenge. Section 2 offers a set of principles for colleges to follow in their instittional change and stdent sccess innovation processes that, when applied, will increase the chances of reaching scale. In Section 3 we provide two examples of scaling at Achieving the Dream colleges to illstrate the principles in action. The Critical Qestion Checklists we present in Section 4 are intended to serve as tools to prompt discssion of key factors that inflence scaling at each stage in the Achieving the Dream 5-Step Process for Increasing Stdent Sccess throgh Instittional Change. By considering these qestions, we hope that colleges may better anticipate and address the roadblocks to sccessfl scaling. Finally, for those who are interested in finding more information on varios aspects of the scaling isse, two appendices are inclded in this gide: a scaling tool created by MDC based on the SCALERS model developed at Dke University s Fqa School of Bsiness, and a list of scaling resorces. These resorces offer valable insights from an array of sectors facing the scaling challenge. Members of the work grop conslted in the development of this gide as well as other sorces of expert inpt into this tool may be fond on page MDC was the managing partner of Achieving the Dream from 2004 to 2010 and contines to direct the Developmental Edcation Initiative (DEI), which works with several Rond I and Rond II Achieving the Dream colleges to bild demonstrated reslts in developmental edcation innovations.

4 Introdction This gide is based on findings from Pblic Agenda s exploration of the most promising practices for scaling stdent sccess innovations at commnity colleges sing its combined research and stakeholder engagement model. The crrent stdy was prompted by an interim report by MDRC and the Commnity College Research Center (CCRC) titled Trning the Tide: An Examination of Rond 1 Achieving the Dream Colleges Progress After Five Years in the Initiative. As part of its comprehensive analysis of the early experience of the first 26 commnity colleges that have participated in Achieving the Dream since 2004 as Rond 1 colleges, the report concldes that the efforts of colleges that scceeded the most in making progress toward improved stdent achievement shared several key featres, inclding broad-based engagement of college stakeholders; strong instittional research capacity; and, the featre in focs here, scaling stdent sccess and instittional change interventions. 2 According to the metrics sed by MDRC evalators, stdent improvement strategies were defined as large-scale if they reached more than 25 percent of their intended target poplations, medim-scale if they reached between 10 percent and 25 percent, and small-scale if they reached fewer than 10 percent of their target poplations. By these measres, the majority of strategies implemented dring the early experience of Rond 1 colleges were small in scale (52% of strategies), and roghly one-third of the strategies were considered large-scale (31% of strategies). What Does Scaling Mean? While benchmarks sed in the MDRC interim report offer a valable qantitative perspective of what it means for a program, service or policy to be scaled, colleges may find it sefl to consider complementary definitions or indicators of scale. For instance, scale is achieved when The program, service or policy has an impact on the majority of the defined poplation and there are measreable improvements or expected otcomes that can be docmented. The practice or policy has become bsiness as sal or has been instittionalized for the college. A college s processes are modified to spport the program or service (e.g., when the college s recritment/enrollment, schedling and resorce allocation decisions are impacted for sstainability). Instittional resorces and policies are aligned in spport of the program, service or policy. Ctting Edge Series: Scaling Commnity College Interventions 2 Throghot this gide we se the term intervention as shorthand for strategies, programs, services and policies that are implemented to enhance stdent sccess and bring abot instittional change. 4

5 MDRC s interim findings also showed that the intensity of a sccessfl strategy has an impact on the likelihood of scaling, with high-intensity strategies more likely to reslt in bigger gains bt less likely to reach large nmbers of the target stdent poplation. MDRC defined high-intensity strategies as those that reached stdents for 10 or more hors per semester, medim-intensity as those that reached stdents for between 5 and 10 hors, and low-intensity as those that reached stdents for 5 or fewer hors. Becase higher-intensity strategies are more likely to meaningflly affect stdents experiences, and ths are more likely to improve their otcomes in observable ways, finding ways to scale these interventions is a special challenge for colleges. Defining High-Intensity Interventions In addition to defining intensity by the nmber of contact hors per stdent, the level of intensity of an intervention can be characterized by its: Level of ambition Degree of toch Degree of faclty involvement Level of necessary coordination across elements or systems at the college (e.g., registrar, stdent services, system office) To help Achieving the Dream colleges work throgh the challenge of moving an intervention from small to large scale, Achieving the Dream asked Pblic Agenda to gather the best thinking abot and promising practices for scaling interventions. To this end, Pblic Agenda condcted a mltimethod stdy, consisting of a broad literatre review, an online discssion and an in-person work grop convening of diverse stakeholders, practitioners and experts in instittional transformation and higher edcation reform. This gide offers recommendations and insights drawn from these sorces and has been reviewed by work grop participants and Achieving the Dream for content, accracy and applicability to higher edcation broadly and Achieving the Dream commnity colleges specifically. Throghot this gide we refer to the challenge of scale in the context of interventions that begin as pilots. This approach is consistent with the Achieving the Dream model of instittional transformation, which begins with college leaders committing to instittional change; emphasizes a data-informed intervention design and implementation process; promotes meaningfl stakeholder engagement to refine and bild commitment to the change process; spports an ongoing evalation and improvement process for the implemented strategies; and then moves to scale those strategies that prove to be sccessfl while establishing a cltre of continos improvement. We focs here on sitations in which pilots are scaled; however, there are instances where colleges have sccessflly gone straight to scale, meaning the intervention was rolled ot widely from the get-go. Commnity colleges may find it sefl to forgo the pilot stage if there is already broad-based consenss that the intervention is necessary, well designed and doable for both implementers and spport personnel. Going straight to scale may be beneficial if the stakes are small in other words, the benefits of the intervention far otweigh the risks posed to stdents, faclty and staff or if a college faces the risk of losing momentm arond an initiative by starting too small. Ctting Edge Series: Scaling Commnity College Interventions 5

6 SECTION 1: Common Obstacles to Scaling Thogh or main prpose in this gide is to provide principles, practices and tools that can be sed to inform scaling efforts, we think it sefl to begin by acknowledging the most common and serios obstacles to achieving scale. Consideration and amelioration of these isses at the beginning and throghot the instittional change process can help to spport a college s efforts to scale in the long term. We encorage instittional change agents to think careflly and together abot the ways these challenges manifest themselves in local contexts. Lack of leadership and governance spport: If the problem being addressed or the intervention being piloted does not align with the broader instittional goals, mission or cltre, it is nlikely that the governing bodies or leadership will agree to scale and ltimately sstain an intervention. Lack of effective leadership may manifest as few commnications abot the intervention among stakeholders, or a loss of momentm de to personnel trnover. Lack of financial and hman resorces: Secring adeqate resorces to scale a sccessfl intervention is an ongoing challenge as instittions strggle to do more with less, and as faclty and staff face heavier workloads and more breacratic responsibilities. Lack of nderlying Instittional Research/ Information Technology (IR/IT) capacity: Commnity colleges IR and IT departments historically have been involved primarily with instittional effectiveness measres and accontability measres for their states and accrediting bodies; they are not departments designed for evalative research on initiatives. Withot adeqate capacity to evalate and present the reslts of ongoing assessment, it is impossible to tell whether an intervention shold or can be scaled. Intervention crowding: The sheer volme of experiments, initiatives and interventions implemented at the college may be crippling for stakeholders and implementers. Faclty and staff may sffer from initiative fatige or share the perception of a change d jor. These experiences can complicate college reform efforts to either drop nsccessfl programs or scale those that are working. Lack of faclty and staff spport: As a pilot expands and the instittion reqires more implementers on board, it becomes necessary to bring additional faclty and staff into the process to contribte in a significant way. Withot growing nmbers of faclty and staff advocates or champions, the energy and ability to carry a pilot to scale will ltimately sffer. Intervention fidelity and adaptation: For complex interventions, scaling a pilot so that it adheres to the pilot design may improve the chances of sccessfl scaling bt can be difficlt nder resorce constraints. In settings where an intervention is to be scaled across a mlti-camps or mlti-college system, variation in local context and cltre between colleges and campses can add an additional level of complexity when deciding whether and how to scale. High-intensity interventions: Thogh highintensity strategies are the most likely to reslt in large gains for participants, they are the least likely to reach large nmbers of the target stdent poplation. For these interventions that reqire more faclty or staff time and instittional fnds, secring adeqate resorces for scaling is an acte challenge. Ctting Edge Series: Scaling Commnity College Interventions 6

7 SECTION 2: Principles & Practices of Scaling Interventions Thogh the challenges to scaling sccessfl interventions may be met at varios points along the path toward instittional change and improved stdent otcomes, there are a nmber of principles that, if followed, can bolster colleges scaling efforts and conteract the derailers. 1. Design pilots with scale in mind Too often, colleges start with a pilot withot thinking throgh what it will mean to bring it to scale down the line. Rather than thinking abot the pilot as a discrete initiative in and of itself, think abot the pilot as a phase in the longer-term change process or as a strategy toward scale. With this perspective, a nmber of strategies can be implemented early on in the life of an initiative to increase its chances for scalability later on. Use a program logic model to articlate the intervention s theory of change that is, how the actions of an intervention will bring abot the desired goal of instittional change or stdent achievement. Use this logic model to gide evalation planning and as a means to reglarly gage progress toward the goal. Develop a formative evalation plan that incldes scalability as an intended otcome and draws a clear line of sight from the theory of change embodied in the intervention to the resorces and commitments that will be reqired to sstain and scale the intervention in sbseqent phases. Use formative evalation as a tochstone dring implementation to gide necessary revisions or improvements to the program model. Dring the design stage, start with a clear definition of the problem at scale, and frame the problem in terms of stdent learning. By defining problems as specifically as possible, the college can break down overwhelming problems into ones that are addressable while maintaining an nderstanding of the scope of the challenge ahead. For example, althogh a pilot may ltimately aim to tackle low developmental math completion rates, it is important to frame its prpose as accomplishing goals sch as integrating more collaborative learning techniqes in the classroom or establishing a learning commnity for stdents enrolled in both developmental math and developmental English. What is a logic model? A logic model is a graphical representation of the resorces, actions and activities involved in an intervention and of how these elements are expected to achieve the stated objectives and otcomes over time. Evalating Stdent Sccess Interventions, Principles and Practices of Stdent Sccess, by Rigoberto J. Rincones-Gomez, provides a practical gide for Achieving the Dream colleges to follow when developing an intervention logic model as a fondation for evalation. The gide can be accessed throgh the Achieving the Dream website: docs/gides/atd-eval_interventions.pdf Ctting Edge Series: Scaling Commnity College Interventions 7

8 2. Establish systems for gaging readiness Colleges often find it difficlt to articlate how they determine the right timing and conditions for scaling, sggesting that they know when they know, or that it s a gt feeling. The right timing and conditions will depend on the intervention and the instittional context; however, identifying indicators of and a process for assessing readiness will help to grond the decision to move to scale. Recommended strategies inclde: Condct feasibility/cost stdies Connect evalation of a pilot to prposefl planning for scale throgh feasibility stdies, and commnicate the reslts broadly in order to bild recognition that the college has done its de diligence in anticipating what scaling will involve. Perform systematic debgging Adeqate preparation for scaling will involve an adit of the varios systems at the college that will be involved directly and peripherally in the intervention. Doing systematic checks dring the decision-making process to move to scale will help to identify potential roadblocks along the way. Get the fll pictre and foster connections Have the steering committee or team spearheading the initiative present to interdisciplinary grops in order to get mltiple perspectives and identify potential derailers for design development at the pilot and, most critically, scaling stages. Cast a wide net Look beyond higher edcation for tools and models that are helpfl to gide readiness assessment and timing the move to scale. Examples of efforts to apply interdisciplinary models to the commnity-college context inclde the following: In Scaling Social Entreprenerial Impact, Pal Bloom and Aaron Chatterji 3 identify seven organizational capabilities that spport sccessfl scaling of a social enterprise, represented by the acronym SCALERS: staffing, commnicating, alliance-bilding, earnings generation, replicating and stimlating market forces. As part of its Developmental Edcation Initiative (DEI), MDC has applied SCALERS at commnity colleges to help them identify scaling needs (Appendix 1). Clearly articlating scaling needs can help a college determine whether it is capable of spporting those needs, what is needed to bild p the resorces and when it anticipates that those needs can be filled so scaling can occr. As part of its Postsecondary Sccess Initiative, the Bill and Melinda Gates Fondation is developing a Framework for Scaling, Sstainability and Systems Change Applied to Higher Edcation. This framework offers a nine-step approach to selecting scaling strategies and draws on lessons from beyond higher edcation, inclding the Technology (Innovation) Adoption Lifecycle, 4 the concepts of the Chasm 5 and Disrptive Innovations, 6 Malcolm Gladwell s Tipping Point, 7 SCALERS and two systems-change frameworks. 8 3 Bloom and Chatterji Rogers Moore Christensen Gladwell Cobrn; Grieff, Proscio, Wilkins Ctting Edge Series: Scaling Commnity College Interventions 8

9 3. Engage stakeholders effectively Engaging stakeholders early and often gives instittional transformation and improved stdent achievement efforts a fighting chance at sccess and is pivotal when planning for, implementing and sstaining the scaling of innovations. What is more, broad-based engagement is needed to anchor instittional commitment and shore p interventions against disrptive staff changes or attrition. Strategies for effective engagement of adjnct and fll-time faclty in instittional change efforts are offered in the first volme of the Ctting Edge Series; here we apply the lessons of meaningfl and effective engagement to the scaling process. Pt faclty and other key implementers front and center Often, the individals most involved in implementation will be faclty; however, depending on the intervention, other key players might inclde advisors, ttors, financial aid officers and even external players like employers. Where faclty are the main implementers, engagement and cltivation of faclty champions is essential to drive scaling. Growing the nmber of intervention advocates and implementers throgh hiring and engagement will bild the necessary hman resorces for scale. Thogh exective leadership is essential, faclty and staff shold be at the fore of efforts to scale and reallocate resorces to make scaling possible. Engage the naysayers Those who are resisting change and scale shold be welcomed to taskforces and design processes. The critical voice can psh efforts to be more data-informed and based in research and can help to identify the knowledge gaps or loopholes to be filled before an intervention is ready for scaling. Engage the owners of the problem The owner (area or department at the college) of the problem shold be a part of the soltion selection, as the soltion will be part of how that owner operates going forward. The owner of a problem can be identified by asking, Where did the problem that we are addressing come from or Where does it reside? Effective engagement can help pinpoint, and also expand, the ownership of a problem in ways that will spport scalability down the line and avoid the assignment of blame. Engage stdents Captring the stdent voice is a critical component of nderstanding how an intervention is received among those most affected and of anticipating whether or not a small-scale intervention will be accepted if scaled. Stdent engagement can happen throgh sch means as srveys, online assessments, qalitative assessments, invitations to participate in committees and facltystdent dialoges. Engage beyond the sal sspects Looking beyond the most obvios stakeholders can be critical to forming alliances on the camps or otside of it which can provide resorces (hman or financial) to make scaling more feasible. For example, the Opening Doors Initiative at Chaffey College, led by Ricardo Diaz, ses gradate stdents from local conseling programs to increase the college s stdent advising capacity. Gradate stdents from local programs provide qality spport, receive clinical hors and are affordable. They do not eliminate fll-time advisors becase they are specifically focsed on providing services to stdents involved in Opening Doors, bt they do redce the workload for fll-time advisors. Ctting Edge Series: Scaling Commnity College Interventions 9

10 Throghot the pilot process, project leaders shold maintain transparency and clear commnication. Development of good answers to key qestions of problem definition, prioritization and soltion design will lead to improved accontability of design decisions and, down the line, will lead to better identification of those soltions that are and are not working. 4. Align resorces with the intervention yo are scaling Scaling a sccessfl intervention reqires that resorces be shifted from the things that are not working to the things yo want to see happen. Efforts to align resorces for scaling will have the greatest chance of sccess if: The intervention is integrated with the college s mission When the intervention is linked to the vision, mission and core vales of the instittion it has a fighting chance of sstainability. College leadership, faclty and staff are more likely to be on board with a new initiative if it is seen as one that clearly aligns with the college s overall goals and other sccessfl programs already in place. The intervention is linked with other programs or initiatives that are nderway and that are part of a larger instittional approach to stdent sccess By aligning programs, instittions can leverage financial, hman and intellectal resorces that are needed to carry an intervention to scale. Alignment can also serve to bffer the initiative against the perception of a change d jor, which can impede the broad-based engagement and spport necessary for scaling. Each intervention shold be seen as a component of a larger, more complex stdent sccess strategy for the instittion. Yo take the time to trim back and let go of nsccessfl practices to make room for the more sccessfl ones that yo are emphasizing Jst as alignment with other reforms reinforces the new practices yo are scaling, removing those practices that are not sccessfl and not aligned, sch as an ineffective se of technology or a failing ttoring program, frees p attention and resorces to help the new intervention achieve liftoff. The intervention is spported by fnctioning nderlying systems Colleges mst demonstrate a willingness and ability to address nderlying flaws in fondational systems. As interventions at the pilot stage ncover systemic problems in one or more nderlying systems for example, malfnctioning data collection in IR/IT, insfficient financial aid staffing or gaps in stdent services instittions mst address these isses prior to scaling. 5. Bild broad-based instittional research (IR) and data capacity Developing a cltre of evidence at the commnity college to inform stdent sccess and instittional change efforts reqires strong instittional research capacity that is not only localized in a department bt distribted across the college commnity. A robst set of promising principles and strategies for bilding IR and IT capacity in Achieving the Dream colleges will be presented in the third installment of the Ctting Edge Series. Here we offer those strategies that show promise in spporting scaling sccessfl innovations. Devote time early on in the scaling process to developing a strategy, tools and training for evalation of the scaled intervention. By articlating the criteria for sccess and how the colleges will measre sccessfl implementation and achievement of intended otcomes from the get-go, colleges can Ctting Edge Series: Scaling Commnity College Interventions 10

11 anticipate data needs, prepare resorces and ensre that systems are in place to reglarly monitor and track the progress of the scaled intervention. If the evalation strategy is developed too late or after the intervention has already been implemented, opportnities for assessment might be missed and potential derailers might go ndetected. Working with the Achieving the Dream Coach and Data Coach to develop an evalation plan can help colleges determine how to execte meaningfl and informative evalation. In accordance with Achieving the Dream s core principles of instittional improvement, 9 bild p IR s programmatic assessment capability so that it goes beyond data collection and analysis for accreditation or state accontability. Increasing IR capacity to achieve this may involve adding evalation expertise to IR departments. Improve IR personnel s ability to translate and commnicate data. Improved IR capacity to translate and commnicate abot data might occr throgh setting new or modified hiring criteria or providing professional development opportnities to IR personnel (for example, workshops or refresher corses sch as those provided by the Association for Instittional Research). Contextalize the nmbers, placing them in the context of the college commnity and the broader goals. Strengthening Instittional Research and Information Technology Capacity throgh Achieving the Dream, by Rhonda Glover, provides a practical gide for Achieving the Dream colleges to review and bild p their IR and IT fnctions. Increase analytic capacity to disaggregate data. Segmentation of data allows for close analysis of how an intervention has an impact on different grops within the target poplation. Scaling does not always mean more of the same thing; it can also mean cstomizing and personalizing. Identifying impacts and achievement gaps among stdent grops is a central practice of Achieving the Dream colleges. The gide can be accessed throgh the Achieving the Dream website: docs/gides/atd_ir_it.pdf Leverage adjnct and fll-time faclty research expertise by involving faclty in data collection, analysis and interpretation. Increase the ability of college leaders to nderstand and commnicate abot data to the wider college commnity. 9 Achieving the Dream is based on the premise that to improve stdent sccess on a sbstantial scale, colleges need to fndamentally change the way they operate. Achieving the Dream colleges that effectively promote stdent sccess adhere to for principles: 1) Committed Leadership, 2) Use of Evidence To Improve Programs & Services, 3) Broad Engagement, and 4) Take Action Aimed At Systemic Instittional Improvement. Ctting Edge Series: Scaling Commnity College Interventions 11

12 SECTION 3: Case Stdies Scaling Case Management at Soth Texas College The se of qalitative data from focs grops and essays paired with historical stdent otcome data and srvey data led to the transformation of the advising process at Soth Texas College (STC). The college piloted the case management approach to stdent advisement with First Time in College (FTIC) stdents in the fall of The case management approach to advising incldes for mandatory contacts dring key periods of the semester. 1. Initial contact after mandatory orientation for registration 2. 4th week follow-p (originally at 6 weeks bt adjsted based on the need to engage stdents earlier in the semester) 3. Priority registration 4. Final contact one week before finals Preliminary reslts of the case management approach to stdent advising reflected a positive impact on retention, completion and other key stdent sccess indicators. As a reslt, the Comprehensive Advising Taskforce was charged with developing a plan to scale the case management approach to stdent advising to maximize the nmber of STC stdents impacted by the intervention. As a reslt, STC developed a Comprehensive Advising Model for Stdent Advising, which consists of a five-prong approach to stdent advising that leverages the case management approach to reach different segments of the stdent poplation. 1. FTIC (First Time in College) case management 2. Stdent sccess specialists (previosly retention specialists) 3. Faclty advising 4. Probation/sspension conseling 5. Beacon mentoring The FTIC case management program was sed as a model to scale the nmber of stdents benefiting from case management. Faclty and staff nderwent a semester long training to become certified Faclty Advisers or Beacon Mentors. The advisers, conselors and stdent sccess specialists also participated in the training. The Dean of Stdent Spport Services developed a plan with the spport of the academic deans and program chairs to transition stdents from the academic adviser to a faclty adviser throgh the stdent sccess specialists. Over 450 faclty participated and were certified as faclty advisers, in addition to over 120 bachelor-level employees to serve as Beacon Mentors, who are assigned to a gatekeeper corse. Each of these advising strategies reqires that the case manager provide the for mandatory contacts otlined above. STC has sccessflly leveraged the stdent otcome data from the case management approach to stdent advising to scale the intervention by secring fnding for additional staffing throgh grants. Department of Edcation, College Cost Redction & Access Act MDC, Developmental Edcation Initiative Ctting Edge Series: Scaling Commnity College Interventions 12

13 Lower Rio Grande Valley Workforce Board, Nrsing & Allied Health Stdent Retention & Sccess Initiative Department of Hosing & Urban Development, Hispanic Serving Instittions Assisting Commnities Texas Department of Agricltre, Parallel Pathways to Sccess The ability to secre external fnding gave the college the opportnity to scale p by spporting stdents reqiring additional spport. All new grant-spported staff are reqired to complete the advising training and to implement at minimm the for mandatory contacts with stdents. Patrick Henry Commnity College s Cooperative Learning SCALE-Up Model Early in the Achieving the Dream process, Patrick Henry Commnity College (PHCC) discovered the impact that the cooperative learning initiative was having on both stdent retention and overall sccess measres. However, this sccess was limited to a core grop of flltime faclty who, combined with a handfl of adjnct faclty, were tilizing this strategy to its maximm potential. Thanks to the Developmental Edcation Initiative Grant (DEI), PHCC has been able to scale p this initiative and take greater advantage of its ability to positively impact stdent sccess. Most important, nearly all fll-time faclty at PHCC have now completed the Fndamentals in Cooperative Learning workshop, which instrcts them in ways of infsing cooperative learning into their corses. Moreover, 75% of adjnct faclty members have also completed the training, which is now offered twice per semester to both new and veteran instrctors. Additionally, PHCC has created an active learning institte called the Sothern Center for Active Learning Excellence (SCALE), which trains not only the local instrctors bt also other commnity college faclty members. This academic year, , the institte was expanded to inclde not only a spring session bt a midwinter session as well. PHCC has also scaled its cooperative learning initiative as a methodology by which the stdent learning otcomes for its critical thinking initiative are met. The topic of critical thinking is not only a Virginia Commnity College System Core Competency, it also serves as the topic of PHCC s Qality Enhancement Plan (QEP) for SACS reaccreditation. Accordingly, the central topic for the faclty and staff development sessions for the academic year has been the tilization of cooperative learning to enhance the critical thinking skills of the stdent. The trainings have been extremely sccessfl and have been expanded so that they are now offered as part of the SCALE institte. Accordingly, PHCC has sccessflly scaled p its cooperative learning model as a primary mechanism by which the learning otcomes of other initiatives are met. The se of cooperative learning is a major factor in the sccess of the college s efforts to accelerate developmental edcation math and English corses. The college is providing fast-track developmental math corses and a second format that allows developmental stdents to take college level corses along with their developmental corsework. Both methods reqire the se of cooperative learning. Accordingly, becase of the fnding efforts of both Achieving the Dream and DEI, PHCC is able to improve stdent sccess. Ctting Edge Series: Scaling Commnity College Interventions 13

14 SECTION 4: Critical Qestion Checklists The following lists of critical qestions are designed to assist colleges in their efforts to follow the Principles of Scale otlined above throgh Achieving the Dream s 5-Step Process for Increasing Stdent Sccess throgh Instittional Change. 10 These qestions can be sed as discssion starters at different levels and departments within the college, and they can be sed as a means to gage intervention and instittional readiness for scaling a particlar innovation. They can also serve as an informal checklist of practices to follow throgh the process of instittional change and reform that will increase the college s chances of scaling sccessflly at a later date. Step 1: Commit The college s senior leadership, with spport from the board of trstees and faclty leaders, commits to making changes in policy and resorce allocation necessary to improve stdent otcomes, commnicates the vision widely within the instittion, and organizes teams to oversee the process. What is the fll scale of the problem we have chosen to tackle in or pilot? What proportion of the stdents impacted by the problem will be served by the pilot? Is this a systemic problem or is it localized in one department/office? How does the chosen problem align with other problems/isses at the college? Are there connections or overlaps of problems in different departments or offices? Are there existing initiatives or programs that are already addressing the problem on camps? Are there existing initiatives or programs that are committed to or are crrently addressing the problem in the broader commnity (e.g., K-12, P-20)? Who recognizes this problem? Are there existing initiatives or programs that are committed to or are crrently addressing the problem in the broader commnity (e.g., K-12, P-20)? How committed are we to solving the problem at hand? What is demonstrating that we are committed to solving this problem? What are the key components to the pilot design, and how are these components going to lead to or intended otcome? Have we developed a logic model to gide or program planning and exection? 10 Achieving the Dream: Commnity Colleges Cont Ctting Edge Series: Scaling Commnity College Interventions 14

15 Step 2. Use Data to Prioritize Actions The college ses longitdinal stdent cohort data and other evidence to identify gaps in stdent achievement. A key premise of Achieving the Dream is that once faclty and staff see that stdents overall are not achieving at desired levels and that certain grops of stdents are not doing as well as others, they will be motivated to try new approaches to improve stdent sccess. To ensre that they se their resorces to greatest effect, colleges are encoraged to prioritize the stdent achievement isses they plan to address. Does data inform or knowledge and decisions to move forward with the pilot? How do we allocate resorces within instittional research departments? What percentage of or resorces goes to data collection for accreditation? What percentage of or resorces goes to initiative evalation and assessment? What are or instittional research staffing strengths and gaps? Who are the adjnct and fll-time faclty with research expertise who can be conslted abot and integrated into data collection, analysis and interpretation activities to increase or IR department s capacity? Does IR have the analytic capacity to disaggregate data? In addition to being data experts, are the IR and IT personnel effective data translators? Is the IR department involved in the development of an evalation strategy, tools and training for the pilot? What is the evalation plan for the pilot, and will we have the capacity to evalate the program if it is scaled? How do we define vale? How will we track changes and program evoltion? How will we collect contextal information that maps onto the data? Is the IR department involved in the development of an evalation strategy, tools and training for the pilot? Are there professional development opportnities for IR and IT personnel to become stronger evalators and data translators? Are there professional development opportnities for faclty and staff to become stronger data sers, analysts, evalators and translators? Do college leaders nderstand the data? What are the mechanisms for commnication and learning between IR and college leadership? Can college leaders commnicate abot the data to the wider college commnity? Is data commnicated in a way that places the nmbers in the context of the college commnity and its broader goals? For mlti-camps colleges, is IR capacity centralized at one camps, or is it distribted across them? If the latter, how will we standardize program monitoring and evalation practices? Are information and data systems integrated in or mlti-camps or mlti-system college context? Ctting Edge Series: Scaling Commnity College Interventions 15

16 Step 3. Engage Stakeholders The college engages faclty, staff and other internal and external stakeholders in developing a limited set of focsed strategies for remedying priority problems with stdent achievement, based on a diagnosis of the cases and an evalation of the effectiveness of previos attempts by the instittion and others to address similar problems. Have we condcted broad-based engagement with frontline players and secondary players? Is broad-based engagement a part of or college cltre? What are the barriers and challenges to meaningfl engagement at or instittion? Are there existing venes or opportnities in which to bild in engagement? How do college leaders commnicate with other stakeholder grops? Who is responsible for identifying and reaching ot to varios grops of stakeholders? When do we engage varios grops of stakeholders in the life of instittional interventions? Arond which types of initiatives or isses are different grops of stakeholders most likely to engage? How do we know this? Which stakeholders shold be involved in defining and prioritizing the problem to address in or pilot? Are there professional development opportnities for faclty and staff to become stronger data sers, analysts, evalators and translators? Which stakeholders shold be involved in developing soltions and prioritizing actions in or pilot? What actions can we take to expand the nmber of faclty, staff and stdent champions of or intervention? To what extent do stakeholders interact with instittional data? Do we have the right relationships between IR/IT, staff and faclty to se data to improve or chances of scaling sccessflly? Internally, do the people involved believe that scaling is possible? Ctting Edge Series: Scaling Commnity College Interventions 16

17 Step 4. Implement, Evalate, Improve The college implements the strategies for increasing stdent sccess, making sre to evalate the otcomes and sing the reslts to make frther improvements. What resorces does the intervention reqire at the pilot level and at the scaled level? Have we condcted rigoros cost analyses for both the pilot and the scaled intervention? Will we offer any incentives dring the pilot stage? Have we considered how incentive strctres might change if the pilot scaled? Do we nderstand which resorces are most important at varios steps in the process from pilot to scale? Are there resorce constraints that we anticipate and that we shold keep on or radar as we plan for scale (e.g., temporary fnding sorces, pcoming elections or other clearly emergent political/fiscal volatility)? Shold we train additional faclty or staff now in order to plan for ftre growth of the program or initiative? What policies or practices might interfere with stdent ptake of the program? How can we proactively prevent these factors from derailing the program? What is the implementation spport plan at the pilot level? Can and will this spport plan be contined at scale? How will implementers provide feedback, commnicate experiences and express programrelated needs or concerns? Will spport extend to all of the departments and services that are involved in the initiative? Will spport be provided only to the primary implementers? Do we have an evalation plan in place for or pilot, and are we following it? What is or system for identifying early warning signs that changes need to be made to the intervention? Who is condcting monitoring and evalation activities? Dring the pilot, have we identified gaps or problem areas that can point s to where we need debgging of the nderlying factors and systems (e.g., financial aid, stdent services, IR/IT) After mltiple ronds of checking and assessment, do we know that the intervention works and that it is making a difference that we vale? How and with whom have we shared the evalation reslts? To move the program to scale, have we considered or made modifications to the pilot design? How do these modifications impact fidelity to the pilot model? Are these impacts showstoppers? Are there components of the pilot design that need to be adopted and not modified in order to protect the integrity of the intervention design? Can the SCALERS model be sed to gide or assessment of readiness and also prepare s for scaling? Based on the application of this model, is scaling feasible? Ctting Edge Series: Scaling Commnity College Interventions 17

18 Step 5. Establish a cltre of continos improvement The college takes steps to instittionalize processes for improving the impact of programs and services on stdent otcomes. Attention is given to how resorces are allocated in order to bring new initiatives to scale and sstain proven strategies. Processes for program review, planning and bdgeting are driven by evidence of what works best for stdents. Does the intervention remain a top priority for the college? How will we rotinely revisit this qestion? Is there alignment among college stakeholders? Are college leaders, trstees, faclty, staff and stdents on the same page abot program effectiveness and feasibility at scale? How will we rotinely revisit this qestion? Does the pilot, as it is scaled, still align with other programs at the college? Do we have instittionalized systems for obtaining program feedback, sharing lessons and identifying spport needs? Are the nderlying systems pon which an intervention depends reglarly evalated in terms of capacity, resorce needs, challenges and sccesses? Do we have the capacity to deal with problems as they come p? How do the costs of programs compare to or planning projections? Are there new opportnities for alignment with external sorces of fnding? Do we have a system to monitor and docment evidence of program implementation and impacts? Ctting Edge Series: Scaling Commnity College Interventions 18

19 APPENDIX 1: Applying SCALERS at the Commnity College In Scaling Social Entreprenerial Impact, Pal Bloom and Aaron Chatterji 11 identify seven organizational capabilities that spport sccessfl scaling of a social enterprise, represented by the acronym SCALERS. With some modifications, MDC developed this application for commnity colleges. Applying SCALERS at the Commnity College In Scaling Social Entreprenerial Impact, Pal Bloom and Aaron Chatterji 1 identify seven organizational capabilities that spport sccessfl scaling of a social enterprise, represented by the acronym SCALERS. With some modification, these concepts are a constrct of what is reqired for scale p in the commnity college setting. SCALERS as Defined by Bloom & Chatterji SCALERS at the Commnity College Qestions to Consider Staffing The effectiveness of the organization at filling labor Staffing The effectiveness of the college at marshalling Does the strategy reqire labor-intensive & skilled services? needs with people who have the reqisite skills for resorces at their disposal to meet labor needs, What HR capacity is necessary to recrit, train, the needed positions, whether they be paid staff or inclding faclty, staff, & stdent employee positions, retain & sstain the reqisite expertise? volnteers leadership & data collection & analysis Commnicating The effectiveness with which the organization is able to persade key stakeholders that its change strategy is worth adopting and/or spporting Alliance-Bilding The effectiveness with which the organization has forged partnerships, coalitions, joint ventres, and other linkages to bring abot desired social changes Lobbying The effectiveness with which the organization is able to advocate for government actions that may work in its favor Earnings Generation The effectiveness with which the organization generates a stream of revene that exceeds its expenses Replicating The effectiveness with which the organization can reprodce the programs and initiatives that it has originated Stimlating Market Forces The effectiveness with which the organization can create incentives that encorage people or instittions to prse private interests while also serving the pblic good Commnicating The effectiveness with which the college is able to articlate clear goals & persade faclty, staff, & stdents to adopt & spport the strategy Alliance-Bilding The effectiveness with which the college is able to engage the necessary parties, forming alliances that spport the strategy Demonstrating Impact The effectiveness with which the college is able to demonstrate to instittional, state, and federal decision-makers that strategies have sbstantial benefits, relative to costs Resorces The effectiveness with which the college manages & secres resorces to sstain the strategy s infrastrctre--staffing, space, technology, etc. Replicating Impact The effectiveness with which the college develops instittional expertise & commitment to spport qality implementation of an expanded strategy Sstaining Engagement The effectiveness with which the college can create incentives that encorage college leadership, faclty, staff & stdents to participate in & vale the strategy What kinds of commnication are reqired to ensre necessary participation in the strategy? What spporting alliances exist within yor instittion? What additional alliances cold yo seek ot to increase the likelihood of sccessfl scaling p? What data do yo need to demonstrate the impact of yor strategy? How does the crrent state policy inflence instittional work? Beyond personnel costs, what resorces are necessary to sstain the strategy? How do state and Federal fnding inflence intervention spport & delivery? What professional development system is necessary to ensre continos improvement for faclty & staff implementing the strategy? What incentives appeal to college leadership? Faclty? Staff? Stdents? What incentives can yo create to encorage adoption of, spport for & participation in interventions from these grops? 1 Bloom, P.N & A.K. Chatterji. (2009) Scaling social entreprenerial impact. California Management Review, 51(3) Bloom and Chatterji

20 APPENDIX 2: Resorce List Achieving the Dream: Commnity Colleges Cont (2009). Achieving the Dream Field Gide for Improving Stdent Sccess. MDC, Inc.and Lipman Hearne. Available at: campsstrategies/resorcesforcollege/ defalt.tp#fieldgide. Bloom, P., Chatterji A. (2009, Spring). Scaling Social Entreprenerial Impact. California Management Review 51(3). Bloom, P. and Skloot, E., eds. (2010). Scaling Social Impact: New Thinking. Palgrave Macmillan. Bradach, Jeffrey. (2010, Smmer). Scaling Impact: How to get 100X the reslts with 2X the organization. Stanford Social Innovation Review. Bryk, A.S., Gomez, L.M., Grnow, A. (2010). Getting Ideas Into Action: Bilding Networked Improvement Commnities in Edcation. Carnegie Fondation for the Advancement of Teaching, Stanford, CA, essay, retrieved from fondation.org/spotlight/webinarbryk-gomez-bilding-networked improvement-commnities-in-edcation Cobrn, C.E. (2003, Agst/September). Rethinking Scale: Moving Beyond Nmbers to Deep and Lasting Change. Edcational Researcher 32(6). Fixsen, D.L., Blase, K.A., Horner, R., & Sgai, G. (2009, Febrary). Readiness for Change. Scaling Up Brief #3. Chapel Hill: The University of North Carolina, Frank Porter Graham Child Development Institte, State Implementation and Scaling Up of Evidence Based Practices (SISEP). Available at docs/sisep_brief_3_readiness_2009.pdf Report Fixsen, D.L., Blase, K.A., Horner, R., & Sgai, G. (2009, Febrary). Intensive Technical Assistance. Scaling Up Brief #2. Chapel Hill: The University of North Carolina, Frank Porter Graham Child Development Institte, State Implementation and Scaling Up of Evidence Based Practices (SISEP). Available at ed/~sisep/docs/sisep_brief_2_intensive_ TA_2009.pdf Fixsen, D.L., Blase, K.A., Horner, R., & Sgai, G. (2009, Febrary). Scaling-Up Evidence-Based Practices in Edcation. Scaling Up Brief #1. Chapel Hill: The University of North Carolina, Frank Porter Graham Child Development Institte, State Implementation and Scaling Up of Evidence Based Practices (SISEP). Available at ed/~sisep/docs/sisep_brief_1 ScalingUp_2009.pdf Christensen, Clayton. The Innovator s Dilemma: The Revoltionary Book that Will Change the Way Yo Do Bsiness. Harper Bsiness,