Evaluating Collective Impact: The Strive Partnership s Evaluation and Response

Transcription

1 Evaluating Collective Impact: The Strive Partnership s Evaluation and Response Greg Landsman, Leslie Maloney, and Geoff Zimmerman Strive Cradle to Career Network Convening Milwaukee, WI September 27-28, 2012

2 2

3 CRADLE TO CAREER CIVIC INFRASTRUCTURE Kindergarten Readiness 4 th Grade Reading 8 th Grade Math HS Graduation & ACT Postsecondary Enrollment, Retention & Completion Strive Partnership 2012 Coordinating Action to Support Student Achievement: Early Childhood Leaders District Superintendents College and University Presidents Community, Corporate, and Private Funders Business and Parent Leaders Service Providers Five Overarching Goals: 1. Prepared for school 2. Supported inside and outside of school 3. Succeeds academically 4. Enrolls in postsecondary education 5. Graduates and enters a career

4 Put concerned people in one room, agree upon statistically definable goals, and then coordinate action and spend the dollars to hit the targets. -Peter Goodman, Huffington Post, June 11, 2012

5 Local Success Percentage of Outcomes Trending Positively Percentage % 74% 81% Report Card Year

6 But are we adding value? Should we be doing more? Can we do better?

7 Evaluation Commissioned Strive hired OMG to conduct evaluation (2011) Goal Identify role(s) Strive plays in catalyzing progress, and identify ways in which it could play an even stronger role in future success. Process Conducted 26 stakeholder interviews, 3 focus groups with collaboratives, and surveyed 461 people with 36% response rate (164). 7

8 OMG Evaluation Framing Six factors critical to the success of collaborative efforts: Purpose: the commonly held (and unique) reason for coming together, including the vision and goals, and how the group will work together to reach them. Value Added: perceptions of whether the group has added value to community-wide efforts and/or to individual organizations. Membership: understanding of what membership entails, what is expected of members and what they can expect, benefits and drawbacks of participation. Also the extent to which an appropriate variety of members is participating. Process and Structure: clarity around the operating structure and processes used for decision-making, accountability, and providing feedback/input. Staff and Resources: skills and talents of the leadership and leadership group(s), as well as other human and financial resources. Communications: clarity and sufficiency of internal and external, formal and informal communications efforts.

9 Purpose: the commonly held, unique reason for coming together vision and goals, and how group will work together to reach them. Nearly 80 percent believe the vision and goals are clear 75 percent believe they are the right ones for the community

10 Value Added: perceptions of whether the group has added value to community-wide efforts and/or to individual organizations. The Partnership has added value to community level efforts in four major roles Convener creating cradle to career framework; facilitating ongoing discussions on education issues Capacity Builder providing committees & collaborative with critical non-monetary resources such as training and facilitation to improve performance Network weaver refreshing/maintaining the network; securing stakeholder buy-in Promoter of data based decision making leading selection of shared outcomes; establishing baseline; developing annual report; changing media approach

11 Membership: understanding of what membership entails roles and responsibilities of members, benefits and drawbacks for members. Stakeholders have varied perceptions of the benefits they should and do derive from being in the partnership Those closest to the leadership of the partnership perceive the greatest benefit

12 Process and Structure: clarity around operating structure, processes used for decision-making, accountability, and providing feedback/input. Confusion exists about roles and responsibilities of the partnership s Executive Committee Interpretations of what Strive the anchor or backbone organization is accountable for, and to whom, vary widely There is lack of clarity about partnership s structure--participants without direct connections to staff want to know how to access/provide input to partnership

13 Staff and Resources: skills and talents of the leadership and leadership group(s), as well as other human and financial resources. Strive backbone staff are considered by participants to be knowledgeable, hardworking, and helpful However, at times, the role of staff is unclear to participants

14 Communications: clarity and sufficiency of internal and external, formal and informal communications efforts Opinions vary about how well-informed partners feel Communication structures and/or processes do not exist between committees/collaboratives, and are minimal between committees/collaboratives and Executive Committee Most participants praise Strive Partnership Report Card as an excellent tool to report region s progress Many feel there is little communication about Strive backbone organization s progress

15 Executive Committee Feedback Finalize key strategies around each of eight key outcomes ( success drivers ) Realign existing work Start /stop collaborative efforts based on key strategy needs Then focus on opportunities that emerged from the evaluation

16 Opportunities Issue: Lack of clarity about what membership or affiliation with Strive means, how it is defined, and who else is involved. Similarly, questions exist about the specific resources and supports that Strive can offer members. Proposed Response: Define partner vs. member vs. stakeholder Clarify roles/responsibilities and benefits of membership Develop process for becoming an official partner of the partnership Clarify staff roles and responsibilities Identify resources available to partners Expand partnership to those committed to improving shared outcomes 16

17 Opportunities Issue: Lack of clarity around the partnership s structure, roles and responsibilities of the Executive Committee and Strive staff, especially for farther out participants without direct connections to staff. Proposed Response: Refine the Executive Committee structure to establish four working groups: Collaborative Action, Continuous Improvement, Community Will, and Advocacy and Funding Alignment Add two staff positions for collaborative action and community will Develop and distribute organizational charts to all stakeholders Host semi-annual partnership events to update and get input from all partners and stakeholders. 17

18 Opportunities Issue: Varied opinions about how well-informed partners feel about what goes on in the Partnership. Internal communications mechanisms (in particular, s) are perceived as lacking a sense of continuity and a full picture of Strive s work. Proposed Response: Develop a comprehensive communications strategy Develop a better partnership with Public Allies to better align the work of the 40 AmeriCorps members in Greater Cincinnati to the Partnership s cradle to career vision and our shared outcomes 18

19 Opportunities Issue: There are some roles that participants are not sure are appropriate for Strive, or do not believe that Strive has effectively played to date, including: -- Leveraging and aligning resources -- Pursing state policy work -- Driving change when leadership doesn t exist Proposed Response: Explore the appropriateness of pursuing these activities with various members of the Partnership Determine the value-added impact of doing this work as well as the expense Identify potential resources who could do this work 19

20 Discussion Questions What should the partnership s role be in fostering collaborative action? A leader & driver of collaborative action? A catalyst for collaborative action? A supporter of collaborative action? What else are we missing? 20

21 Discussion Questions There are some roles that participants are not sure are appropriate for Strive, or do not believe that Strive has effectively played to date. Is it appropriate for a Strive Partnership civic infrastructure to: Leverage and align resources? Pursue state policy work? Drive change when a lack of community leadership exists around an issue? 21

22 Discussion Questions What roles and responsibilities are most critical for the Strive Executive Committee to play: Overall agenda setting? Decision-making about Strive s overall direction, and/or specific goals and strategies in each priority area? Defining operational expectations for committees/collaboratives? Setting a precedent by making organizational level changes? 22

23 Discussion Questions Does the role of the Executive Committee change as the partnership matures? What are the individual roles/responsibilities for Executive Committee members beyond attending Committee meetings? 23