PEACE DIRECT S DEVELOPING APPROACH TO MONITORING AND EVALUATION

Transcription

1 PEACE DIRECT S DEVELOPING APPROACH TO MONITORING AND EVALUATION Peace Direct is a UK-based NGO that is dedicated to supporting locally led peacebuilding in conflict areas by finding, funding and promoting programmes developed and led by civil society leaders. DfiD currently is funding Peace Direct in a project to support locally led peacebuilding. Within that project, Peace Direct is working to demonstrate the effectiveness, efficiency and value added of locally led peacebuilding. Part of this work involves identifying, defining and evaluating characteristics of locally-driven peacebuilding that support the creation of sustainable peace. Peace Direct s commitment to support local capacity to build and nurture peace also implies a commitment to develop and support locally-driven evaluation of this work. This paper summarizes key achievements in the process of developing this local monitoring and evaluation capability thus far within this project. Peace Direct s Objectives Increase the evidence base for local peacebuilders by increasing the number of local peacebuilders evaluating their work Develop locally appropriate approaches to M&E to strengthen locally led peacebuilding strategies Contribute to efforts in the peacebuilding sector globally to improve methods of measuring the impact of peacebuilding By producing local indicators, try to close the divide between international indicators and realities on the ground Challenges The majority of Peace Direct partners have not been internationally funded before and therefore lack any experience in M&E Through this project, we are learning that their peacebuilding projects rest on a substantial base of work in trust building, relationship building, and conflict transformation carried out within the community. It is this largely invisible work in emergent and transformative social change that creates the environment in which projects can be delivered effectively and sustainably. However, work at the emergent/transformative sateges is even harder to measure than projectable change seen in actual projects (see Box 1) Evaluating peace is more challenging than evaluating project results because so many actors and factors contribute to creating conflict and peace. Opportunities Through Insight on Conflict (IoC) - there is an opportunity to introduce concepts of M&E to local peacebuilders en masse. However, other approaches, such as training the IoC correspondent to conduct M&E on behalf of the IoC featured organisations may be more useful.

2 Our monitoring and evaluation work is helping us to better understand both how peace is built and sustained at the community level and thus how the achievement of peace can be evaluated more effectively. M&E is more effective when designed and implemented by local people whilst being much more cost-effective than an external evaluation. The commitment of local peacebuilders to peace is transferred to a commitment to M&E when they understand the value of M&E to their projects Localised M&E approaches are typically more transparent and inclusive of communities leading to greater accountability of the NGO to the beneficiaries. Box 1: Peace is an Iceberg: Emergent, transformational and projectable change as described by Peace Direct s external evaluator, Rosemary Cairns Emergent change is when communities and people are just beginning to recover from trauma or conflict; the most appropriate strategy is accompaniment walking with people as they slowly learn to work together once again, providing specific help as they identify their need for it. Transformative change involves unlearning such as unlearning how to be in conflict in a community and can be a challenging time for those who work with the community. Projectable change tends to happen when a community can identify its needs clearly and is able to plan and carry out a project. I visualize this as an iceberg, with projectable change being the small part visible above the waterline, resting on a large body of work in emergent and transformative change that is done largely by local peace builders but is largely unseen. Projectable change is the stage at which communities are healthy, strong and organized enough to develop and manage projects. This tends to be the area of focus for many international development projects, without always recognizing that projectable change rests on a great deal of work already done by local people. Most project budgets don t specifically include such trust-building and relationshipbuilding costs, yet these are equivalent to bricks and mortar for a house building project. At present, the only way to support such vital emergent work is through supporting the organization s operational costs as an investment in supporting this peacebuilding capacity. Example from DRC In DRC, Henri, the director of the Centre Resolution Conflits (CRC) is working on encouraging militias to return from the bush but he cannot just call up a mai mai leader and say I d like to talk about you bringing your soldiers back from the bush. Before he can even talk to that leader, he has to identify that leader s contacts and allies that takes a lot of air time on a mobile phone and travel into the bush. Once he identifies some key allies, he has to speak with them to explain what he is doing. Only when he has built a relationship with them can he speak with the leader. But that is only the start of the process. The mai mai leader may decide to come and visit Henri in person, and he may show up in a taxi one day without any notice, expecting that Henri will pay for the taxi. Then perhaps it will be necessary to meet for tea, or a meal or two. And then, once all that relationship-building has been done, he may invite Henri to come and visit him in the bush and meet his men. Henri then needs to travel there, and spend time talking with the leader and his men in their own context. There may be a number of meetings and possibly even some kind of feast once the decision is made that they will come back from the bush. And all of that happens before Henri can begin to work on what might be considered the projectable part of the activity - finding income generating activities for this mai mai group once they return home, and working with the community to accept them back.

3 PEACE DIRECT S MONITORING AND EVALUATION APPROACH Given that most local peacebuilders have had little or no experience with doing monitoring and evaluation themselves, and often work in English as a second language, our work began with the basics. In visits to projects and work during the first Peace Exchange, we explained the planning, monitoring and evaluation cycle as simply as possible. Simple analogies help people to understand these linkages, and when they understand the linkages, they begin to incorporate evaluation into their planning. Learning to think through and identify goals, steps to achieve them, and measures for assessing achievement at all stages of a project also gradually builds the capacity of local peacebuilders to have a peer-to-peer conversation with international donors about monitoring and evaluation in terms that make sense to both. Introducing the Concept The expertise of local peacebuilders is helping us to begin mapping out the indicators of peace at the community level. In Nairobi in February 2010, during the first Peace Exchange supported by Peace Direct with funding from DfID, we asked 12 local peacebuilders how they know that their communities are becoming more peaceful. They identified 65 signs or indicators and then sorted them into nine categories. Almost two-thirds of these indicators fell within the areas of increasing personal security and enhancing social relationships. Categories, ranked by total number of cards in each, were named as follows: 1. Social relations enhanced; 2. Personal security; 3. Communication and trust building; 4. Inclusion; 5. Demilitarization; 6. Rule of law; 7. Economic opportunities increase; 8. Functioning institutions; and 9. Youth empowerment.

4 Box 2: Challenging common outsider assumptions One international observer noted that this framework presented a different picture of where work should focus in the aftermath of conflict. Many projects, he observed, focus on rebuilding institutions and the economy, but local priorities clearly are on enhancing social relations, building personal security, and on trust building and communications work that requires local knowledge. Demand for M&E at the local level Participants found the participatory approach to developing criteria for measuring their work so useful that they asked for similar workshops during evaluation visits to their projects. In Sri Lanka, the Young Visionaries team then immediately began to use this same method to have YV peer groups in local communities identify indicators that would show their effectiveness, and sources of information. The team will roll up these locally-developed indicators into an overall framework for assessing their own achievement. Box 3: Enthusiasm for the approach Evaluation of the Peace Exchange showed that most participants planned to use the participatory approach to develop indicators for monitoring and evaluating their work. Eleven of 13 survey respondents found the evaluation training useful. One said Evaluation turning impossible into understandable while equipping us with super useful tool for us to take home and ready to implement. One person noted that the colonial mindset created the belief that evaluation is something done by somebody else. I was not aware of the tools and mechanisms how we can incorporate it into the process. I got many insights and tools to accommodate it into the process. Said another: I had always had trouble to come up with measurable indicators for peacebuilding work but in this workshop I learned how to measure the impact of our work. Another said that the simple definition of monitoring and evaluation will be very helpful as they have had so much trouble understanding this area. Taking it to the communities As the Sri Lanka example shows, local peacebuilders want to use the communities that they work with to identify the indicators. Following the M&E workshop at the Nairobi Peace Exchange, the Collaborative for Peace in Sudan asked that their new proposal include funds to take this approach to M&E further, involving local communities. The Collaborative plans to involve the communities it is working with to identify the indicators that they want to see. In this way, the Collaborative will use more relevant indicators, as well as offering the communities greater transparency and greater project support as communities will understand clearly how the project will benefit them. In turn, this will lead to greater accountability of the Collaborative to the community as the communities, rather than the Collaborative, will set the parameters for success.

5 A Win-Win Situation This kind of inclusive approach is a win-win situation as responsibility for M&E is shared with the communities. By choosing indicators that are identified by the communities the indicators are typically more visible making them easier to collect. For example, rather than conducting a psychosocial questionnaire to assess the reintegration of a combatant, they look at indicators that are easy to see, such as building a house or starting to farm. The ease of these indicators means that staff with little or no M&E experience can still do the evaluation. In fact, it is feasible that someone within the community could be appointed to monitor the progress. Furthermore, this approach keeps local peacebuilders connected to the local level even as their organization grows in sophistication. At the Nairobi Peace Exchange the peacebuilders expressed concern that, as they became an NGO, they lost some of their local character. Some NGOs, as they grow, become more professional and start to lose touch with their deep community roots. This approach helps to keep their priorities embedded in the communities.

6 Box 4: An example in action, CRC in DRC An example of the process in action It has taken some time to find effective ways to help local peacebuilders set effective indicators for measuring their results that are meaningful in their own local contexts and also to donors. An example of the evolution of this process comes from the CRC in Beni, eastern Democratic Republic of Congo. Finding the indicators In July 2009, the external evaluator met with CRC and Task Force representatives in Butembo to develop evaluation criteria for a DDR project funded by the John Ellerman and Baring Foundations. As the experts in knowing when DDR was effective at the community level, local people were asked to list indicators that showed them when ex-combatants were being integrated into the community, and the sources of verification. Indicator Examples: The indicators listed by the communities included such items as ex-combatants marrying, building a house, getting a job, not using black magic and being accepted by the community, as well as the source from which this information could be obtained the school teacher, chief, church minister, and so on. This list relates as much to the community s perception of the ex-combatant s integration as to the actual activities of the ex-combatants themselves. This seems to show that for communities, DDR is much more about strengthening community peace sustainably by integrating the ex-combatants so they do not return to the forest and resume wreaking havoc. Thus it is about assessing the quality, and not just the quantity, of DDR activities. Example of Ex-Combatants Re-integration Indicators: Number of ex-combatants returned to community 630 Number of houses built by them 205 % involved in community activities 90 The above indicators are a few examples that allow CRC to monitor the sustainability of the reintegration process and intervene at signs that combatants may be susceptible to a life in the bush again. For example, in a community with little desire to accept ex-combatants back, CRC can intensify its sensitization programme with that community or perhaps create joint community-excombatant livelihood projects to help with reconciliation. Example of Task Forces indicators: Number of rumours confirmed 43 Number of rumours rejected 28 Number of people bringing cases to them 129 For the Task Forces, a key indicator of effectiveness was how many rumours they were able to confirm or dispel. Apart from their role in ensuring that untrue information is stopped (a key concern where rumours can send people fleeing into the forest), the numbers of rumours brought to them reflects their credibility with local people. Given that the Task Forces include a range of local authorities, this work also helps strengthen local community governance but it is undoubtedly hard to measure that effect, especially when there is no baseline to measure it against. Practicalities Given the transportation, communication and security challenges in eastern DRC, it took some time for CRC to work out a practical way to collect information about these indicators from the individual communities. In the end, CRC/RDC developed a questionnaire that reflected the agreed indicators and visited the communities to collect statistics and where possible, stories by talking with local stakeholders. Part of this work involved making people aware that information they provide should relate to the actual project rather than more generally, so as to begin linking results to project activities. This questionnaire helped community members begin to understand the evaluative process, thus effectively pushing evaluation down a further level into the community itself.

7 Quantifying a perspective The next challenge for our partners is to put the statistics into context, so that the numbers make sense to donors in terms of their overall impact on peace at the community level and how that information relates to their theories of change. As our partners understand more and more M&E, they naturally start to look at the larger context. What does it mean for a community when many ex combatants have returned, married, and built houses? Does that make the community more peaceful and if so, how can that be measured? And to what extent is it possible to credit the return of ex-combatants specifically to CRC s work? How does one measure the result of a task force s work in investigating and dismissing a rumour, for example? Does this increase community security and if so, how does one measure this effectively? During the recent CRC annual general meeting held in Kampala, participants focused on the question: How do I know that CRC is achieving its mission? Some examples are below: Demobilization and reintegration of excombatants Peaceful cohabitation Resumed socio-economic activities Socio-economic reintegration of returnees Promotion of information about peace Assistance to people living with HIV/AIDS Promotion of human rights Improved security Community approaches Womens rights We are finding that as the process continues, the indicators are becoming more specific, often setting numerical targets as objectives for example, 1900 combatants re-integrated; 9000 IDPs assisted to return; 2000 assisted with livelihoods; etc. As they continue to gain experience, we believe that the indicators they set will continue to become much more specific, measurable, attainable, relevant and trackable. Genuine Capacity Building: Working with local peacebuilders to develop locally-relevant indicators for evaluating their work within a framework that comes from their local experience, while also making sense to international donors, is a key part of capacity building. We believe it will result in a community-based evaluation framework that will allow peacebuilders to monitor the results of their activities while also revealing the underlying but generally unacknowledged preparatory work they do to build peace. In this way, the local peacebuilders can build their own capacity whilst also building the capacity of the international community to better understand what causes sustainable peace and indicators for measuring it.

8 CONSIDERATIONS FOR THE FUTURE There is still much work to be done and is likely to be a continuous part of Peace Direct s work. For now, the priorities are to: Use the October 2010 Peace Exchange in London to work with the local peacebuilders to refine specific indicators and ways of measuring them Continue to develop our approaches and learn lessons of best practice to share in the on-going debate on how to measure peacebuilding Investigate how we can best utilize IoC to increase the number and amount of M&E from local peacebuilders Experiment with using indicators of peace as the starting point for a project planning tool rather than looking for indicators to prove a project Continue to explore ways of measuring the vital elements of conflict resolution that exist within the emergent and transformative stages of conflict Move closer to determining ways to measure the costs and benefits of locally driven peacebuilding compared to international interventions Given that making all of the above would benefit greatly from both international and local expertise and perspectives, we will actively seek to work with others, such as DfID. This combination of expertise will help elaborate the framework of evaluation that is beginning to emerge from this work.