Evaluation of the Alliance Strategy and Theory of Change: HIV, Health and Human Rights - Sustaining Community Action, 2013-2020 By Social Development Direct 1 Executive Summary (October 2015) Background: In August 2014, the International HIV/AIDS Alliance (IHAA) engaged Social Development Direct (SDDirect) to undertake an independent evaluation of its Strategy and Theory of Change, HIV, Health and Human Rights: Sustaining Community Action, 2013-2020. The overall objectives of the evaluation were: To measure the Alliance s performance in achieving the outcomes of the organisational Strategy and Theory of Change (ToC) and selected thematic areas (value for money, sustainability, relevance/effectiveness, beneficiary feedback, and gender) including key achievements and challenges. To provide evidence (and validate existing evidence) of progress against the ToC and its assumptions, including an in-depth exploration of selected key processes of change. In 2013, the Alliance launched a new strategy HIV, Health and Human Rights: Sustaining Community Action 2013-2020. The Alliance then developed a Theory of Change (ToC) to support implementation of the Strategy. The ToC broadly defines a series of outcomes that have hitherto been implicit in its HIV programming. It highlights the sequence of actions which ultimately lead to impact on health and sociopolitical outcomes. Overall, the assumptions underpinning the Alliance ToC are based on the belief that communities have the power to tackle HIV if recognised and supported to do so and that civil society capacity and resources exist at both local (community) and national levels. The Strategy Evaluation focused on three core strategic responses and five key outcome indicators: Strategic Response 1: Increase access to HIV and health programmes Outcome 1: Improved health outcomes 1. Percentage of Alliance countries contributing ten per cent or more of the national coverage for HIV prevention, care or treatment, as reported to national HIV coordinating bodies or based on population size estimation. Strategic Response 2: Stronger health and community systems Outcome 2: High performing & sustainable civil society 2a. Community based organisations and networks supported by the Alliance show increased capacity to innovate, strengthen community systems and measure results 2b. Percentage of Alliance Linking Organisations receiving five percent or more of their total funding from sources beyond Official Development Assistance Strategic Response 3: Inclusive & Engaged Societies Outcome 3: Empowered communities 3a. Percentage of Alliance countries where violence and discrimination against people most affected by HIV is recognised by the state and addressed by CBO-led individual case response systems. 3b. Globally and number of Alliance countries, where the Alliance s community and global action achieves verifiable progress towards policy goals related to HIV, health and rights. 1 Report by Barry Smith, Emma Bell and Jenny Holden, Social Development Direct (info@sddirect.org.uk)
Key Alliance achievements The introduction of the new Alliance Strategy and Theory of Change has been a valuable process to date and progress has been achieved in relation to several of the key outcome indicators above. The evaluation process confirmed the Alliance s strong commitment to a rights-based approach and its critical role as a champion of key populations the most vulnerable and marginalised groups affected by HIV. Under the banner of its 2013-2020 Strategy, the Alliance is putting social inclusion at the heart of its work. This is very much in tune with the post-2015 Sustainable Development Goals (SDG) agenda and it helps to position the Alliance as a global civil society leader at the nexus between HIV, social justice and sustainable development. Achievements and strengths Outstanding commitment and passion of both Secretariat staff and Linking Organisations (LOs) Strong focus, alignment and achievements around work with the most marginalised groups Strengthening leadership globally on the rights of key populations Strong performance on institutional funding, leadership on sustainability and funding diversification Growing Southern leadership on key issues, and a stronger Linking Organisations (LOs) leadership role Significant advance around segmentation allowing for more tailored approaches to LOs. The credibility of the Alliance as a key player and partner in mobilising a sustainable community response to HIV was affirmed by evaluation informants at both the Global Fund to Fight AIDS, TB and Malaria and UNAIDS: Its reach is so extensive that it can bring a lot of knowledge to the table There are no limits on the possibilities for collaboration with UNAIDS (UNAIDS representative). Progress, challenges and opportunities Building momentum and alignment around the Alliance Strategy and Theory of Change: There is evidence that some Alliance Linking Organisations (LOs) are increasingly taking ownership of the Strategy and ToC. The introduction of the new Strategy and ToC has helped to stimulate and inform strategic conversations across the Alliance as well as the process of strategy-making and development of theories of change at national and LO levels. However, it appears that buy-in and uptake from a wide cross-section of LOs remains mixed. The evaluation team found that further efforts are needed to make the Strategy and ToC come alive and to build ownership, momentum and alignment across the global Alliance. An outcomes-based strategic focus and approach: As the nature of the HIV epidemic evolves, amidst a fastchanging donor and policy environment, the onus is on the Alliance to articulate more clearly its offer and impact at outcome level. With their vision of a sustainable community response to HIV, health and rights, the Strategy and ToC are starting to shift the culture and focus of the Alliance from outputs to outcomes. While this shift is underway at different speeds among different parts of Alliance, there is growing acknowledgement of the need to track progress in relation to wider scale outcomes that contribute to healthy people, strong community systems, and inclusive and engaged societies. Approach to monitoring and evaluation, data and evidence: Spurred on by the Strategy and its focus on results at outcome level, the Alliance is making good progress towards a more robust, standardised approach to monitoring and evaluation, data and evidence. Although, capacity for research within the 2
Secretariat and among the LOs remains limited, important steps (such as the Alliance s research partnership with the London School of Hygiene and Tropical Medicine and the formation of a cross-alliance research group) are being taken to address this gap. The Alliance s emphasis on robust quantitative data collection should be balanced by more systematic harvesting of qualitative data about impact of their work on the lives of key populations through beneficiary feedback mechanisms, strengthened monitoring and evaluation and research capacity across the Alliance, and better linkages between the reporting and media and communications functions. Building robust reporting systems: The Alliance relies heavily on self-reporting by LOs and CBOs to verify progress on key indicators. The Alliance has recognised that more robust shared data collection and reporting systems are needed, as evidenced by the new annual reporting instruments (developed in response to the recent global data reporting error) and the introduction of unique identifier codes (UIC). Giving priority and a sense of urgency to sustainability: The Strategy has put issues of sustainability firmly on the global Alliance agenda. In the context of a rapidly changing operating and funding environment, the Alliance is playing a critical leadership role by adding a sense of both urgency and possibility around financial and political sustainability. For the period ahead, a clearer shared understanding of political sustainability should be nurtured across the Alliance. As it pursues its ambitious sustainability agenda, the Alliance needs to ensure as much as possible that both LOs and the Secretariat have access to the right mix of skills and capacity required to diversify their resource base including, where necessary, outsourced skills. Leadership and innovation on Value for Money (VfM): In recent years, the Alliance has been a civil society leader and innovator on the issue of VfM. There is good evidence of economies, efficiencies and cost savings (at both Secretariat and LO levels). But while some shared understanding and common approaches to VfM have emerged, these are not consistent across the Alliance. In the period ahead, the Alliance has an opportunity to build on its record of activism around innovative VfM approaches to promote a more integrated body of practice among the Secretariat and LOs. Building decentralised leadership and capacity: The Strategy is moving the Alliance towards a more decentralised model in which LOs take on a larger leadership and implementation role through the establishment of regional Centres of Practice in various technical and thematic areas (such as key populations, sexual and reproductive health rights and harm reduction). Regional Technical Support Hubs (TSH), hosted by LOs in 6 regions, continue to provide a high volume of south to south technical support to LOs and CBOs. Steps are being taken to develop business models for TSHs to market services to other organisations and generate income to sustain their work. Alliance principles and initiatives have been mobilised to galvanise this culture shift, but progress is uneven. There are residual perceptions and behaviours among both LOs and global staff that tend towards a paradigm of the Secretariat as donor. Creating a cohesive Alliance brand and identity: Through the Strategy and ToC, the Alliance has created opportunities to sharpen the Alliance brand and identity around a strong focus on vulnerable, socially excluded groups ( key populations ) and a sustainable community response to HIV, health and human rights. The Alliance s Positioning Strategy provides a useful road map in this regard, but it is important that the Secretariat s media and communications function should be integrated at an early stage into all strategic initiatives so as to maximise the possibilities for cohesive and creative messaging to both external and internal audiences. Introduction of more nuanced segmentation approach to LOs: The Strategy has concentrated minds on the diversity of capacities, expertise, scale and contexts of the LOs. As a result, a segmentation approach has been adopted that allows the Secretariat and LOs to move away from a homogeneous view of the Alliance and to work from a more nuanced analysis of the different assets and needs of LOs as well as the varied contributions they can make to the achievement of strategic objectives. This analysis enables tailored 3
Secretariat services and technical support to LOs and reduces the burden of unrealistic, undifferentiated expectations. It gives space for LOs to be valued for what they do, and what they can do well, despite their relative size. The segmentation approach enables more informed and strategic investments in LOs that are likely to yield greater value for money. The Alliance s positioning and programme response to HIV testing and treatment: The current focus of the global HIV response is very much on HIV treatment, including access to testing and treatment, provision of treatment services, adherence to treatment, etc. While the Alliance is undoubtedly playing an important role around treatment issues, especially in relation to the needs of key populations, there is some ambiguity and hesitation about the Alliance s engagement with testing and treatment. It is important for its future positioning that the Alliance should clarify its strategic approach and priorities around HIV treatment programming. Improving the quality of treatment access for key populations can go hand-in-hand with a greater focus on access to the cheaper, easier and more reliable testing technologies that are now available. Meaningful engagement with, and accountability to, Alliance service users: Based on interactions with Secretariat staff, LOs and CBOs and particularly community beneficiaries, the evaluation team concluded that the Alliance would benefit from more systematic mobilisation of beneficiary dialogue, feedback and accountability mechanisms. Stronger mechanisms to give voice to, and engage with, target populations (ensuring involvement of target populations at each stage of programme design, implementation and evaluation) would improve quality of services and measurement of progress towards strategic goals and outcomes. Gender and empowerment of women and girls: The Alliance is engaged widely in work relating to gender, women and girls, but it is hard to assess overall impact. Currently, Alliance tracking and reporting efforts seem to equate benefits with aggregate numbers of women and girls reached by Alliance programmes. The Alliance has certainly been engaged in some excellent work that addresses sexual identity and diversity as well as gender-based social norms. However, the Alliance has not mainstreamed its gender focus nor has it realised its full potential to improve the way the Alliance works in relation to women and girls. A more effective approach would involve understanding and tackling gender-related stigma, discrimination, and barriers to service access. There is an exciting opportunity for the Alliance to capitalise on its distinctive experience and approach around gender and sexuality to develop a gender strategy that makes the links between gendered and sexual identities, gender equality, empowerment of girls and women, and social inclusion. Accreditation as a core strategic process: The enhanced Alliance accreditation system has made great strides and plays a critical role in fostering shared values, standards and peer learning; building LO systems and capacity; and providing a basis for both comparison and collaboration. Accreditation processes can play a critical role in improving LO accountability to, and engagement with, service users. As the accreditation system is critical to realising the Strategy and achieving better outcomes, it should increasingly be positioned as an integral strategic process in the Alliance. Working as an alliance: As the Alliance works to realise its Strategy, there are some tensions between ensuring meaningful participation and collaboration between different parts of the Alliance, on the one hand, and on the other the need to deliver efficiencies in decision-making. Streamlined decision-making is undoubtedly necessary, but some staff members are concerned that sometimes key voices and work streams are insufficiently included in strategic dialogue and decision-making processes. Within the Secretariat, there is a still a widely acknowledged tendency to work in silos, and some staff worry about a disconnect between the pace of change in the Secretariat and the rather different trajectories of some LOs. Alliance work culture: Staff members, at both Secretariat and LO level, are passionate and committed to their work. There are, however, elements of Alliance culture, particularly in the Secretariat, that make it 4
difficult to say no and to stay focused. The Strategy is intentionally ambitious. At the same time, there has been considerable restructuring and streamlining of the global Secretariat with consequences for staff work loads. In the view of management, the intention of the strategy is to transform, not downsize, the role of the Secretariat and move leadership increasingly to the LOs with the Secretariat playing a facilitating role. This is a major change and will take time. In the meantime, more organisational development support is required to enable staff to work differently and to counteract the perception of being asked to do more with less. General recommendations The full evaluation report makes detailed recommendations on the key outcome indicators under review. General recommendations around the key themes, issues and processes include: Continued strengthening of the Alliance s mobilisation data and evidence, as well as its strategic focus on outcomes and impact. Strengthening meaningful engagement with, and accountability to, Alliance service users though more systematic beneficiary feedback and accountability mechanisms. Building renewed leadership, activism and common understanding around Value for Money (VfM). Continued high priority to leadership and technical support around financial sustainability. Nurturing more shared understanding of political sustainability across the organisation and LOs. Capitalising on its distinctive approach and experience with gendered identities and sexuality, the Alliance should develop a clear and comprehensive gender strategy. Stronger positioning and resourcing of the accreditation system as a central, integrated and strategic process in the Alliance. Clarifying the Alliance s positioning and programme responses to HIV testing and treatment. Working more collaboratively, effectively and efficiently as an alliance. Managing change in the Alliance work culture through targeted organisation development support. 5