How To Measure The Impact Of Drought Risk Reduction

Transcription

1 Community Based Resilience Assessment (CoBRA) Conceptual Framework and Methodology Commissioned by UNDP Drylands Development Centre Under the framework of Humanitarian Aid and Civil Protection Department of the European Commission s Drought Risk Reduction Action Plan Version: April 4,

2 TABLE OF CONTENTS TABLE OF CONTENTS Background Introduction Conceptual Framework Why a methodology to measure resilience? Building on the Existing Evidence Base Common Definitions of Disaster Resilience Existing Models for Disaster Resilience Community Based Resilience Assessment (CoBRA) Measuring Resilience Methodology Phase I: Preparation Step 1: Identify Target Areas and Baseline Resilience Step 2: Prepare for Field Work Phase II: Field Work Step 3: Identify and Train Field Staff Step 4: Data Collection Phase III: Data Analysis and Reporting Step 5: Data Analysis Step 6: Presenting and Using Findings Step 7: Repeat Monitoring of Impact and Change Annex 1 Existing and Emerging Resilience Models Annex 2: Components and Potential Indicators of Resilience

3 1 Background After two consecutive seasons of inadequate rainfall since 2010, the impacts of drought have dominated the humanitarian scene across the Horn of Africa (HoA) from mid The terms such as risk reduction, vulnerability reduction and resilience building are increasingly becoming the new 'hot topic' being highlighted at various drought fora and included in numerous drought-focused project documents. There are islands of innovative disaster risk reduction (DRR) practices, which have been helping the affected communities to withstand, adapt to and recover from the ongoing and past drought crises in the region. However, these achievements are insufficiently recognized and hence have often been overlooked in the Governments' priority areas of action planning processes the future. This results in duplicative efforts, reinvention of the wheel and knowledge loss in the region. Various efforts have been undertaken to document proven community-based DRR experiences so as to scale up and roll out the established measures and techniques. On the flip side of these efforts is information overload and fatigue: interested stakeholders are often caught in a deluge of good practices and lessons learnt in the absence of a clear definition of "good", "best" and "success" and common indicators/comparable data to measure tangible long-term impacts of these interventions. Pilot attempts are under way at global level to develop standardized criteria for holistically quantifying the impacts of DRR investments, throughout the cycle of prevention/mitigation, preparedness, response and recovery, not only in terms of life saving but also in terms of reducing underlying vulnerability. A quantitative impact analysis of community-based DRR initiatives is expected to help fill in the multiple gaps evident in the ongoing DRR implementation. Initiatives range from government-led/top down processes and community-driven/bottom-up measures, sectoral approaches and between development and humanitarian partners - shifting their scattered foci strategically and enhancing their coordination towards long-term resilience enhancement and livelihoods improvement. It is in this context that the UNDP Drylands Development Centre (DDC) developed the Community Based Resilience Assessment (CoBRA) project, with the financial support from the Humanitarian Aid and Civil Protection Department of the European Commission (ECHO). Under the framework of the ECHO s Drought Risk Reduction Action Plan (DRRAP), the CoBRA intends to build on the going efforts and introduce the robust analytical tools through which to measure the socio-economic and environmental impacts of community-based DRR in short and long terms systematically. In particular it focuses on quantifying the results of development and humanitarian interventions for drought in the HoA region, inter alia Ethiopia, Kenya and Uganda. 2 Introduction The overall objective of the project is to reduce drought/disaster risks and improve human livelihoods in the disaster-prone communities in general, and the HoA more specifically, by establishing an integrated enabling DRR planning and programming framework at national and regional levels, effectively promoting local resilience building and vulnerability reduction. Towards this objective, CoBRA aims at 3

4 developing a rigorous conceptual framework and standardized methodology for measuring and assessing the impacts of community-based DRR interventions on local resilience building. It seeks to introduce broadly applicable and comparable indicators/indexes that evaluate and aggregate short- and long-term changes and trends in drought resilience as a result of various interventions comprehensively in quantitative values. It will also take into consideration the qualitative measures of other less readily quantifiable impacts to allow for the deeper interpretation of results obtained from quantitative assessment by shedding light on the processes and causal relationships. Developing the quantitative methodology is the core conceptual element of the work. The methodology needs to sit within a conceptual framework that guides the overall intention of the proposed programme of work. The approach proposed will attempt to translate the most robust theoretical understanding of disaster risk reduction and resilience into a conceptual framework and practical methodology at the community/household level that can be used to: Assess the baseline resilience levels of, and between, different communities / groups of households; and Assess which interventions have impacted upon the target communities / groups resulting in a positive or negative change in resilience, particularly following a shock or stress. This conceptual framework and methodology have been developed through several iterations, incorporating feedback throughout the process. The first draft was built on a review of existing literature, and through extensive consultation. This was presented at the 2 nd Africa-Asia Drought Adaptation Forum, held in Nairobi in October 2012 and the 5 th Africa Drought Adaptation Forum held in Arusha in February Feedback from both events has informed this document. The first draft was then field tested in Uganda and Kenya, and the results have shaped this draft final approach (specific recommendations coming from the Kenya and Uganda field studies can be found in the field study reports). 3 Conceptual Framework 3.1 Why a methodology to measure resilience? In order to help communities onto a path of resilience building, rather than increasing vulnerability, it is clear that a multi-faceted approach at scale is required. This is in sharp contrast to the current fragmented, largely sectoral and project-based approach to interventions. In drought affected areas, where protracted crises with spikes in need are the norm, tools are required that help to take an integrated approach to resilience, documenting evidence of groups of interventions that have high impact. The benefits of a methodology to measure resilience are as follows: Currently no tools exist to measure resilience as a long-term multi-dimensional concept. Programme monitoring tends to be very sectoral, and national level monitoring of many development indicators is not done frequently enough, or at a scale that allows for differentiation across livelihood groups, ecological zones or wealth groups. 4

5 All actors need to prioritise interventions on the core factors affecting resilience rather than attempt all sectors equally, or only provide the assistance for which political will /funds are available (e.g. food aid). A multi-faceted approach can help to engage decision making that looks at issues more holistically, recognising the interconnection between different facets of resilience. A resilience measurement tool will doubtless include baseline indicators and data that are already being collected such as HEA, DHS, risk profiling etc. The value of a set of context specific resilience indicators is that it enables analysis of multi-sectoral issues not normally considered together. It is often the interaction of these factors, rather than any specific factor, that is undermining resilience. For example, data on the number of households with members who have completed secondary or tertiary education is rarely analysed alongside data on the diversification of household incomes or levels of conflict and insecurity. Key indicators affecting resilience are not comprehensively collected using any widespread agreed methodology, and hence there is no mechanism for consolidating and comparing findings. Conflict and governance are prime examples. 3.2 Building on the Existing Evidence Base A key component of this study is to develop a conceptual framework that would go a long way in highlighting layers and components of resilience; however, it is also essential to note that the proposed conceptual framework draws from other existing and ongoing work globally. A review of existing work on resilience measurement was used to identify some of the key principles underpinning the resilience agenda, as well as to identify existing models that have been developed to capture resilience concepts Common Definitions of Disaster Resilience Numerous definitions of resilience exist 1, and for the most part, they broadly reinforce each other. The UNDP defines building resilience as: a transformative process of strengthening the capacity of women and men, communities, institutions, and countries to anticipate, prevent, recover, adapt and/ or transform from shocks, stresses, and change 2. DFID s definition also links resilience with long term development: Disaster Resilience is the ability of countries, communities and households to manage change, by maintaining or transforming living standards in the face of shocks or stresses - such as earthquakes, drought or violent conflict without compromising their long-term prospects" (DFID, 2012). In other words, when a group of people experience a shock or stress, what pathway do they follow? Those who collapse, or recover but are worse than before, are not resilient, and are likely to fall deeper into vulnerability. Those who bounce back, or bounce back better, can be said to be on a resilience pathway. Within this, it is important to note that resilience, like vulnerability and risk, is a dynamic concept. In addition resilience is a multi-dimensional concept that requires the simultaneous measurement of 1 UNISDR 2009; John Twigg; Characteristics of Disaster Resilient Communities 2007; DFID 2011; etc. 2 UNDP Resilience Position Paper 2013 (Draft) 5

6 several factors, both short and long term. This goes against the current orthodoxy of monitoring and evaluation practice, which tends to be highly sectoral. Finally, it should be noted that the term disaster-resilience, rather than drought-resilience is being used here. Although drought is recognised as the most frequent and major stress experienced by communities in the drylands of the HoA, it is not the only one. While it is important that shocks and stresses are not pre-determined or limited, this needs to be balanced with the need to define what a community is resilient to, and whether it actually experiences disaster or is just chronically vulnerable Existing Models for Disaster Resilience Existing resilience models can typically be categorised into two groups: models that attempt to capture/ describe a system-wide approach to resilience (for example, DFID/TANGO, Practical Action, Fraser); and models that attempt to define and measure the characteristics of resilience at a community level (for example, FAO, Oxfam, and Tulane). The framework presented here draws from and builds upon some of these various models 3. However, it also differentiates from these models in that it is designed to be a participatory, community based methodology. It is designed as a package of interventions that can be applied in many contexts, rather a one-off study. Annex 1 contains a brief summary of other existing models. The specific mandate of this assignment is to design a quantitative impact assessment of interventions on resilience at the community or household level. As such, the system-wide models encompass a much wider range of factors than those captured in this study. However, it is very important to relate the model developed here to the wider system, as the findings that come out of this approach will need to be informed by other processes and actors. The models that attempt to define the characteristics of resilience at a community level are much more relevant to the specific mandate of this assignment, as ultimately changes in these characteristics as a result of programming will provide a quantitative impact assessment at a community or household level. However, consultation on these characteristics indicated that these should come from the communities themselves, and preferably be grouped using existing categories, rather than introducing a new set of categories. System-Wide Approaches System-wide approaches seek to define a range of activities, actors and processes that are part of a resilience building system. For example, a recent paper by TANGO International, funded by the World Bank, DFID and USAID, presented a composite framework for assessing resilience, with a specific focus on food security shocks in Africa see Figure 1 below. 4 It is adapted from the DFID Disaster Resilience 3 Frankenberger,T. and Nelson, S. (Feb. 2013) Background Paper for the Expert Consultation on Resilience Measurement for Food Security ; and Summary of the Expert Consultation on Resilience Measurement for Food Security TANGO International 4 Frankenberger, T., T. Spangler, S Nelson, M Langworthy (2012). Enhancing Resilience to Food Security Shocks in Africa. Discussion Paper,

7 Framework (2011), TANGO Livelihoods Framework (2007), DFID Sustainable Livelihoods Framework (1999) and the CARE Household Livelihood Security Framework (2002). The overall objective of the TANGO resilience assessment framework is to enable policy makers and practitioners to have a comprehensive understanding of the factors and processes influencing vulnerability and resilience at the household and community levels. There are others that also achieve a similar objective, and the TANGO model is presented in Figure 1 to provide an example of what a resilience framework can look like, and to describe where this study fits within a wider systems approach. Figure 1: The TANGO Resilience Assessment Framework Because the specific mandate of this assignment is focused on community or household level resilience, it is very much focused on the adaptive capacity section of the TANGO model (the three blue ovals in the figure below though we rename them as discussed in subsequent sections). Specifically, the aim is to measure the ability of households to cope with shocks or stresses, by determining and measuring the characteristics of those households over time and track if they are on a resilience pathway, or a vulnerability pathway. Characteristics of Resilience There are also numerous ongoing efforts to capture the various characteristics of resilience at a community or household level. Here there is more diversity and far less consensus, with a wide range of characteristics represented across the different models, such as: income, food security, assets, access to basic services, social safety nets, ecosystem health, livelihood strategies, adaptive capacity, governance, and stability, to name a few. 7

8 The methodology presented here is intended to build on existing tools and processes this is critical if it is going to be practical and achieve buy-in from humanitarian and development actors. While there are many factors that define resilience, a key objective of this tool is to select those that give a reasonably good representation of resilience, acknowledging that, on the one hand, resilience is multi-faceted and not just represented by one or two indicators, and on the other hand, a tool that captures all facets of resilience would be too cumbersome to be of any practical use. With this in mind, the Sustainable Livelihoods Framework (SLF) is used as a method of categorising or mapping the potential characteristics of resilience. It defines five key categories (human, natural, financial, social and physical) that can effectively encompass the indicators of resilience that are reported in the literature. It is also widely used by development actors to examine the different facets of a community. It is not intended to define the building blocks of resilience, but rather provides a way of mapping indicators into familiar categories that can facilitate comparison across groups. The Figure 2 and Table 1 present the SLF, showing the five categories of livelihood assets, sitting within the wider framework. Figure 2: The SLF Framework 5 5 DFID Sustainable Livelihoods Guidance Sheets 8

9 Table 1: Definitions of SLF Components (adapted from DFID) Dimension DFID Definitions Physical Capital The basic infrastructure (water supplies, roads, railways, telecommunications) that people use to function more productively. Human Capital The sum of skills, knowledge, labour and good health that together enable people to pursue different livelihood strategies and achieve their livelihood outcomes. Financial The cash that enables people to adopt different livelihood strategies. This can be in Capital the form of savings, or a regular source of income such as a pension or remittance. The inputs that support livelihoods, as well as the producer goods (tools, equipment, services) that contribute to the ability to increase financial capital. Natural Capital The natural resources (land, forests, water) and associated services (e.g. erosion protection, storm protection) upon which resource-based activities (e.g. farming, fishing etc.) depend. Social Capital Access to and participation in networks, groups, formal and informal institutions. Peace and security. Governance and political relationships. 3.3 Community Based Resilience Assessment (CoBRA) Building on the various models and components of resilience described above, and based on stakeholder consultation and feedback, as well as field testing, a conceptual framework for a quantitative impact assessment of interventions to build resilience was developed and refined. The premise for the approach presented here is that, in order to measure resilience, and the impact of interventions on resilience, we need to first understand our baseline. Are households resilient at a given point in time, and how does that change following interventions? The model presented here is very much based on understanding resilience from a community perspective how do households define and prioritise the characteristics of resilience (or, in other words, those households that are able to cope with a shock or stress without external assistance)? What do existing resilient households look like, and how did they get to be resilient? Importantly, the approach aims to learn from positive experience by identifying resilient households as a starting point and examining how they are able to cope with shocks. The conceptual model for this work is adapted from the EC-FAO model 6, and is represented in Figure 3 below. This model was adapted because it was felt to be most relevant to the work being undertaken here, as it very much focuses on impact assessment at a community level. It assumes that resilience can be measured at a given point in time (T 0 ) as both a composite measure of five components of resilience and overall based on a universal measure (see below). Both composite and universal measurements should be evaluated again (T 1 ) following shocks/stresses, as well as interventions, modified by household response mechanisms and external policy. Resilience is not static, and it is multi-dimensional therefore monitoring resilience needs to happen over a longer time frame, and should include ongoing updates at regular intervals. The approach is also designed to be participatory and very much based on evidence from the community level. 6 Livelihood Strategies and Resilience to Food Insecurity: An empirical analysis for Kenya: Alinovi, D Errico, Mane and Romano; European Report on Development

10 Figure 3: Community-Based Resilience Assessment (CoBRA) Capital and Capacity One of the key differences that emerges from the literature and the consultation is that resilience is not only about capital (the key focus of the SLF categories) but also about capacities. Capital tends to describe the assets, skills and services that households and communities have, in other words, more static factors. Capacity indicators measure more dynamic factors the ability of households to expand and contract their capital in response to stress or change, and rely on skills and linkages to adapt in a positive way. Clearly adaptive capacity is an essential part of what helps us to achieve resilience, but not an end in itself like adequate food, income, water, education etc. In this model, adaptive capacity should and can be measured but within or as part of the various livelihood outcomes. Similarly, it is very important to identify negative coping mechanisms. Many households could be resilient in the short term, because they are able to cope by engaging in unsustainable or unethical practices for example, increased sale of firewood/charcoal where forest management is an issue, or early marriage of daughters. It is important to identify communities where resilience may be supported by negative adaptive capacity, as these will need to be re-classified as not resilient. 3.4 Measuring Resilience The task of measuring resilience within this framework requires identifying the indicators of resilience, as well as finding data to measure. There is a clear tension between ensuring the findings are robust, whilst balancing the limited resources available to do this analysis. The conceptual framework for this study is based on the premise that resilience can be measured in two ways: 10

11 (1) An overall or universal measure or indicator(s) of resilience enables us to understand whether resilience is increasing, decreasing or staying the same; and (2) Composite and contextually specific indicators of resilience enable us to understand how local drivers of resilience are expanding or contracting, and the impact of interventions on those drivers. A Universal Measure of Resilience In order to quantitatively measure resilience, there needs to be a consensual definition as to which households are resilient and which ones are not, and importantly the differences between these two groups. At what point do you cross over to being a resilient household? Unfortunately there is currently no overall or universal indicator(s) threshold of resilience which can be used to measure absolute and comparable levels of resilience (and non-resilience) across contexts and in aggregate i.e. is the percentage of population in village/ district / country X who are resilient growing or declining, and how does this compare with village / district / country Y. The hypothesis in this study is that many households would define themselves as resilient when they are able to feed their families adequately every day, and meet basic needs (on a consistent basis, and in times of shock/stress as well as normal times, and without external relief). Being able to quantifiably measure this ability to cope would then give us our universal indicator of resilience. One of the best monitoring tools tracking precisely this is the Household Economy Approach (HEA). The HEA uses detailed household level data to compare conditions in a reference year to conditions in the current or modelled year, and assesses the impact of changes on households ability to meet a set of defined minimum survival and livelihoods protection requirements. Figure 4 demonstrates the model more concretely in visual terms, and demonstrates how income sources are combined to identify the point at which a household no longer has a food deficit and therefore can, in theory, subsist without external assistance. For the purposes of this study, the livelihoods protection threshold could be redefined as the resilience threshold a household that is able to maintain itself above a resilience threshold in bad times could be considered a resilient household whilst those that fall below are not. HEA data exists in most countries in Africa, to varying degrees, and is mapped on a livelihood zone basis. Hence the level of analysis cannot identify specific households or communities that are resilient; however, it can define livelihood zones and wealth groups within them that are resilient both generally, and in times of drought or other shocks. Hence it provides an excellent meta-indicator to identify hotspots where significant proportions of a community are not resilient. In cases where comprehensive or recent HEA data is not available, food security and similar surveys, assessments (such as seasonal needs assessments conducted by Government agencies and WFP) can be used to provide panel data. These assessments often do not capture as wide a range of variables as HEA, they rarely break down populations into as many localised food security groupings, nor can they be modelled for different shock events. However, they can provide an indication of the groups and regions that are unable to feed their families. This again provides a useful proxy for inability to cope with a shock (or even in good years), i.e. areas that are not resilient. 11

12 Figure 4: HEA Emergency Response Thresholds Compared to Total Income Levels The Survival Threshold represents the total income required to cover: a) 100% of minimum food energy needs (2100 kcals per person); b) The costs associated with food preparation and consumption (i.e. salt, soap, kerosene and/or firewood for cooking and basic lighting); and c) Any expenditure on water for human consumption. The Livelihoods Protection Threshold represents the total income required to sustain local livelihoods. This means total expenditure to: a) Ensure basic survival (above); b) Maintain access to basic services (e.g. routine medical and schooling expenses); c) Sustain livelihoods in the medium to longer term (e.g. regular purchases of seeds, fertilizer, veterinary drugs, etc.); and d) Achieve a minimum locally acceptable standard of living (e.g. purchase of basic clothing, coffee/tea, etc.) Development of Indicators for Components of Resilience Once the areas and populations that contain significant non-resilient populations have been identified, the next step would be to identify those factors that are building resilience (or not) in those communities. Whilst the HEA livelihood protection threshold is a good overall indicator of resilience, it does not by itself explain why households have become resilient or not. Other areas of development, such as water and health care access, or social networks, or good governance, or ecosystem health, could all contribute to resilience. In other words, if you have access to good health care and your cost of illness is therefore lower, you have more disposable income, which increases your capacity to feed your family. If you benefit from good governance, you can advocate with your local officials, and push for developments in your community. This will facilitate greater income/assets and/or lower costs, contributing to your ability to meet basic needs. Consequently the second step in measuring resilience is to understand which of these factors should be measured and tracked in order to ultimately increase the proportion of resilient households. The SLF categories are useful here to group and map the multi-dimensional components of resilience this is important to allow cross-comparison between different communities or regions Within each of these five categories, the specific indicators of resilience need to be identified, and could vary quite substantially depending on the specific context. For some standard indicators, significant information and data for the target areas is likely to exist. However, these data are rarely available below a province or district level, and may be collected less frequently than the incidence of the shock. Consequently they do not help to understand the variations between communities and households that 12

13 can affect resilience, especially in light of programming decisions which can be heavily affected by local context. Therefore this methodology proposes the development and measurement of indicators at a community level hence local communities are asked to develop the indicators that define resilience, and then provide the data to measure how far they have been achieved. Spider or radar diagrams can then be used as a tool for mapping the multi-dimensional indicators that comprise resilience in that context. The diagram below illustrates the relative resilience scores from the Kenya field testing, using the five components of resilience. The left spider diagram in Figure 5 shows how communities ranked the relative importance of their defined indicators of resilience, and the right diagram indicates the proportion of the community that meets these criteria in the current year and in a crisis year. Figure 5: Community Ranking (first diagram) and Scoring (second diagram) Assessing the Impact of Interventions These diagrams would be repeated at regular intervals, possibly annually, and compared with the range of interventions undertaken in the area, to quantitatively assess their impact on resilience. Along similar lines, control communities and communities with a specific intervention could be the subject of analysis, in order to compare resilience across the two and infer the impact of specific interventions. It should be noted, however, that resilience is a holistic assessment, not a project specific measurement. As such, the approach would ideally be used as a regular monitoring framework to track progress along a resilience pathway. Its value lies in the ability to identify areas that have been relatively well addressed, as well as gaps requiring further programme development, which can then guide holistic planning processes at community, district and wider levels. 13

14 4 Methodology The methodology described below is designed to support the conceptual framework, as presented above. A much more detailed set of guidelines for implementation of the methodology will be developed as part of this work, and therefore the description presented here simply presents an overview. Phase I: Preparation Phase II: Field Work: Data Collection Phase III: Data Analysis and Reporting Step 1: Identify target areas and establish baseline resilience identify the information already available analyze the information available based on steps for analyzing resilience Identify the information required but not yet available. Step 2: Prepare for Field work Finding out about the geographical area i.e., livelihood analysis Ensuring that communities are informed well in advance. Logistical arrangements Step 3: Identify and train field teams Identification of stakeholders Assigning roles and responsibilities Simulation exercise for the field team Step 4: Data collection 4(a) Key informant interviews (KIIs) with local government and agencies 4(b) Focus Group Discussions (FGDs) 4(c) Key Information Interviews (KIIs) with resilient households Step 5: Data Analysis Step 6: Presenting and Using Findings Step 7: Repeat monitoring of impact and change 4.1 Phase I: Preparation Step 1: Identify Target Areas and Baseline Resilience The first step is to establish the target population(s) to be assessed, by whom and for what particular objective. Identifying the meta-indicator of resilience to be used in assessing the current baseline level of (non) resilience is also a key task. This will enable the organisation(s) doing the assessment to track changes in overall resilience levels over time. It is also a useful way to correlate how well the components of resilience (to be identified by the CoBRA) do impact upon overall resilience. As outlined in the conceptual framework above, the HEA livelihoods protection threshold is proposed as a robust measure of the percentage of households that cannot cope (in both normal and crisis years). However as mentioned, it may be more practical to use an indicator that is regularly collected or widely accepted. For example in field testing in Kenya, the proportion of households in need of humanitarian 14

15 assistance in the last crisis year (2011) was used. This figure is produced annually by the Government led Kenya Food Security Steering Group (KFSSG). All drought affected areas of the HoA have similar assessments or data sets that enable the identification of areas where large proportions of the population are regularly or chronically not resilient. These assessments are undertaken in both crisis and non-crisis period, providing a range of percentages. As a crude proxy, food security assessments, such as the World Food Programme s Comprehensive Food Security and Vulnerability Analysis (CFSVA) studies 7 etc. could be used. Each user will have slightly different objectives for using the CoBRA methodology and these should be clarified at the start to establish the level and frequency of field work. Field work can be undertaken to do one or more of the following: Gather baseline information on the characteristics and levels of resilience to influence programme or policy design in a specific area (e.g. an NGO s area of operation); Conduct comparative analysis of the characteristics and levels of resilience between different livelihood zones / geographic areas etc. (e.g. a county / provincial authority comparing zones or districts within its area); Understand the characteristics and strategies of existing resilient households in a particular area; Evaluate the impact a specific (or range of) intervention(s) by comparing target and control communities; and/or Undertake regular assessments (e.g. annual) to track overall trends and community perceptions of resilience and the priority interventions, services or support they require. Develop a common decision making framework to maximise the synergy between the work of humanitarian and development actors in an area The user s commitment to repeated or longer term use of the methodology s approach should also be considered in the levels of training and amount of baseline data collected. Step 2: Prepare for Field Work Once the communities and objectives of the field work are established, a detailed plan (including timeframe, responsible parties and budget) for data collection in the field needs to be prepared. The plan should include consideration of the following for each community/ livelihood group: Sampling methodology this will dictate the number of sites to be visited and the number of focus group discussions (FGDs) and key informant interviews (KIIs) to be carried out at each site and in total. The implementation guide has more detail on sampling methodology. 7 The World Food Programme (WFP) conducts country-level Comprehensive Food Security and Vulnerability Analyses (CFSVA) to provide an in-depth picture of the food security situation and the vulnerability of households in a given country. These are baseline surveys, conducted in normal times (not crises), in countries that are subject to vulnerabilities. The CFSVAs are intended to identify food insecure and vulnerable populations within a county, provide insights into why they are food insecure or vulnerable, and identify appropriate assistance to reduce vulnerability and food insecurity. In addition to obtaining basic descriptive information on the scope of food insecurity and vulnerability, the assessments are intended to undertake analyses to identify the root causes of food insecurity, and analyze the risks of all types of shocks and their potential impacts on the most vulnerable. 15

16 Resources and budget required the number and timescale within which FGDs and KIIs are to be completed will dictate the number of facilitators required for what time period. It will also enable estimations of the supporting logistics. Resources available where there are several agencies or organisations committed to undertaking a CoBRA, opportunities to pool or share resources (e.g. staff and vehicles) should be established. The costs of CoBRA training and technical support a minimum of two days of classroom training will be required for FGD facilitators along with technical support throughout field work and to undertake the analysis. 4.2 Phase II: Field Work Step 3: Identify and Train Field Staff The quality of data collected will depend to a large extent on the skills and experience of the team selected to undertake field work. The team leaders should be technical specialists who have successfully completed a CoBRA ToT training (to be developed). If an organisation is committed to repeated use or institutionalisation of the CoBRA methodology it may be best to train and utilise existing local staff rather than hire a consultant and external facilitators. The process also provides them with the opportunity to gain first-hand experience of the interests, needs and priorities of their target communities. A well-balanced team would consist of a mixture of men and women, fluent in local languages and who have experience in community mobilisation and facilitation. Good training and ongoing support and supervision are pre-requisites. Step 4: Data Collection Following training, the field work is comprised of three main data collection processes (two using the same tool): 1) district level key informant interviews (KIIs); 2) focus group discussions (FGDs); and 3) resilient household KIIs. Data collection Process 4a: Local Level Key Informant Interviews (KIIs) 4b: Focus Group Discussions (FGDs) 4c: Resilient Household Key Informant Interviews (KIIs) Description Structured interviews / checklists to be completed with key government/ UN/NGOs and other agency staff operational in the area. The purpose is to map the level and scale of local services, interventions and monitoring data collected in district generally and target communities. Separate groups of between men and women representing households from the target (and control) communities. Number of FGDs per location / livelihood zone and overall to be determined by sampling framework. Semi-structured interviews with adult members of households identified as resilient (approximately two informants per site). Detailed guidance and data collection formats will be developed for each process. The information provided by each process is outlined below. 16

17 Step 4a: District Level KIIs These KIIs attempt to systematically map the level and scale of past, ongoing and planned interventions and other services benefiting the target communities. Clearly local authorities as the primary duty bearers are critical in this process. It is also important that, if the local authority is not leading the assessment, they are informed of the process and (where necessary) their permission is obtained. The interview format is effectively a series of checklists covering general and sectoral services and programmes available. The KII checklists also collect information on the type and frequency of monitoring data collected by the local authority and other agencies in the area. This information is valuable when analyzing why and how resilience assessments differ and the respective impact of different interventions. It can also further assist in identifying target and control communities within the same livelihood zones for selecting FGD sites. Step 4b: Focus Group Discussions This is the core data collection mechanism for CoBRA. following process: Each FGD takes participants through the Explanation of the process and managing expectations - It is very important that the rationale and purpose of the FGD is explained to participants. It is particularly important that participants understand the FGD is an information gathering exercise to enable a better understanding of their communities aspirations and challenges. It is not linked in any way to current or future programming, although findings may be used by organizations and policy makers when planning in the future. Consensus on the Community being discussed The term community is used repeatedly in the FGD. Facilitators need to ensure that all participants have a common understanding regarding the type and number of households referred to by community. Understanding Resilience and Crisis - Resilience is not a common term in most communities and many local languages have no specific translation. Given the first exercise in the FGD is to consider what a resilient community would look like, it is important that the facilitators spend some time discussing what the term means to and within communities in that particular context. The same is true for the term crisis with reference to the community as a whole. Again it is important to establish the frequency, range and severity of crisis years (or periods) experienced in a particular community. In some contexts there may be no well defined crisis period that systematically hits the wider community or livelihood zone. Where no clearly defined crisis can be identified it will be difficult to score communities attainment of resilience characteristics in a crisis period (see step 4 below). Develop a list of outcome statements describing resilience drawing on the above, participants are encouraged to develop statements that describe a resilient community. Facilitators encourage as many positive impact statements as possible (without duplication) that would describe their community if ALL households had achieved resilience. Participants prioritise and rank these statements - Once the long list of statements is complete, each participant is given six beans or stones and asked to rank their three priority statements about resilience. They put three beans on the most important, two on the second and one on the third. The statements are then put in priority order from the highest to the lowest total bean score. Communities rate their current status towards attaining these statements in normal and drought years - For the top scoring statements, participants are then asked what proportion of households in this community has attained each statement. This is asked first for a normal year/period and then again for a crisis year/period. This area needs strong facilitation as many 17

18 statements cannot be so easily scored e.g. forests would be well managed. Statements may also score zero in both normal and crisis years e.g. we would have access to a paved road. An example list of resilience statements and scores for one FGD is shown in Table 2 below. Table 2: Example Resilience Characteristics/ Statements and Scoring for an FGD % households for Prioritised Characteristic of Total bean score which this is true Resilience normally All households have members educated to tertiary level All households have access to clean water all year round All households enjoy peace and security at all times The rangelands are managed well so there is always good grazing Households have access to saving and credit facilities Community would have its own wellfunctioning market to sell livestock % households for which this is true in a crisis period 28 5% 5% 21 40% 5% 18 90% 25% 11 70% 15% 9 10% 10% 8 0% 0% Communities identify the types of households in their community that have (fully/partially) achieved resilience - The group is asked to consider - who are the households for which the prioritized statements remain true even in a crisis period? The group is asked to describe the characteristics of such households and what they have done to become or stay resilient. Identification of resilience building interventions - Participants are asked to list all support and interventions that their community has benefited from currently or in recent years 8. They are not prompted by facilitators about any specific intervention or project. Once the list is complete, participants are asked to rate the top three most effective interventions in building resilience and explain why. They are also asked to name and justify interventions not currently in place but that could be implemented to build resilience, explaining why. The intention here is not to showcase specific interventions, but rather to learn from the specific types of activities that the communities feel are helping them to build resilience, as this gives important evidence for scaling up. This will clearly also need to be incorporated with knowledge from other stakeholders on interventions that are not as visible, but which may have played an important role. Intervention specific discussion - A further session can be added to the FGD to explore in more detail any specific resilience building intervention being implemented by an organisation supporting the CoBRA. This would be particularly important where the intervention(s) was not rated highly or even mentioned in the long list. This also enables organisations to get particular feedback on an intervention only implemented / operating in some of the selected sites. Step 4c: KIIs with Resilient Households FGD participants and local facilitators are used to identify members of the resilient households described by the FGD process. These key informant interviews are semi-structured to explore the following information about resilient households: 8 Participants are asked to consider community and private sector initiatives not just Government/NGO/UN etc. 18

19 The composition, educational level and livelihood/economic activity of each member. Which community identified characteristics of resilience they meet What factors in addition to those raised by the FGD members have contributed to the households resilience? Pathways to resilience - was the household always resilient or has it become resilient over what timescale? Actions or strategies they take/took to cope with shocks and crisis affecting that community. What interventions or circumstances helped them to become resilient? What interventions and support would best assist others in their community to achieve their resilient status? 4.3. Phase III: Data Analysis and Reporting Step 5: Data Analysis Data collected from all KIIs and FGDs is entered into standard excel spreadsheet formats for compilation, aggregation (where possible) and analysis. Key stages in the analysis of field data include: o Mapping communities resilience statements against sustainable livelihoods framework (SLF) categories. o Sum and weight/normalize bean scores for all statements. o Score the achievement of priority characteristics in normal and crisis periods and plot on radar diagram, according to the SLF categories (comparisons can also be made within SLF categories if relevant, for instance between health and education as components of human capital). o Disaggregate results as required between livelihood groups or target and control sets of communities depending on sampling strategy. o Compile descriptions/ characteristics of resilient households. o Compile list of interventions mentioned group according to Government / NGO / UN / Community / Private sector. Rank highest to lowest mentioned. o Map actual services and interventions against those mentioned / highly rated by communities. Quantitative results from the above analysis are then analysed alongside the more qualitative descriptions and explanations provided by FGD participants and resilient households to form the development of conclusions and recommendations. In particular findings can be used to identify formal indicators that could be incorporated (or are currently collected) by systematic monitoring processes 9. These indicators can be both capital (static) or capacity (dynamic) that measure both the current status of attainment and its potential or actual capacity for change see Annex 2 for examples. Step 6: Presenting and Using Findings The findings of the CoBRA can be written up using a standard reporting format and in a summary two page report card (a standard format will be provided in implementation guidelines). The full report is likely to contain the following: 9 Education or Health Management Information Systems (E/HMIS); Early Warning/ Food Security Information Systems; Living Standard or Household Income and Expenditure Surveys; etc 19

20 Background and Introduction objectives of the CoBRA, background information on the target population; context and recent/ recurrent crisis. Methodology the details of how the CoBRA was carried out Current overall levels of resilience / non-resilience of the target communities based on secondary data, including justification and explanation of meta-indicator used. CoBRA Findings o Prioritised community characteristics/ statements of resilience in the respective target populations. o Communities assessments of percentage of households that attain these characteristics in current and crisis periods (disaggregated as required). o The characteristics of resilient households in the respective target populations. o Potential indicators (capital and capacity) to monitor changes in resilience characteristics (highlighting who does or could collect such data). o Interventions and services most highly rated by (different) communities. o Any gaps or weaknesses in services / interventions. Conclusions o How do local communities define resilience? o o Who are resilient households and how do they stay and become resilient? To what degree are the services and interventions currently provided to communities building resilience? Recommendations o Which indicators (capital and capacity) should be tracked to monitor resilience in this context. o Policy and programming implications of the findings for various stakeholders o including interventions to be expanded, started or scaled down. Repeat monitoring recommendations. The conclusions and recommendations may vary depending on the specific objectives of the CoBRA. The CoBRA report can be a useful tool for stakeholders in designing programming, strategies or polices for an area. This wealth of information can play a key role in identifying characteristics of households that are resilient, gaps in action, and interventions that seem to make a critical difference. However, it should be restated that this methodology is only one component of a full resilience framework the findings should be used in conjunction with wider analyses and in coordination with other actors. So, for example, if access to education is identified as a key component of resilience, more qualitative analyses will be required to identify the best way to facilitate that access, any institutional or policy barriers that need to be addressed, etc. Summary Score Card The full CoBRA report will also be summarised into a standard two page Report Card that provides the highlights of the findings. This enables findings of one CoBRA to be easily compared with those done elsewhere, or previously. It also assists in ensuring the key findings and recommendations can be widely disseminated and read by many stakeholders. Step 7: Repeat Monitoring of Impact and Change Repeat monitoring will be essential to track the transformational changes in resilience levels over time and the impact (if any) of new or scaled up interventions. Timing and frequency of the data collection process is repeated will vary depending on the objectives, resources and timeframe of the implementing partner(s). However, it should be noted that resilience is a long term change and so collecting data 20

21 more frequently than annually may not yield much change in results. In addition, given that standard definitions regularly cite resilience as a communities ability to withstand, absorb or recover from shocks repeat monitoring during or just after a shock is also recommended. Again shocks, even where frequent, rarely occur even annually. Note that maintaining resilience levels after or during a shock could be a positive outcome reflecting well timed DRR or humanitarian interventions. CoBRA fieldwork findings should always be compared against the overall resilience measure or metaindicator identified. This will enable implementers to see if the percentage of resilient households is increasing (or decreasing) in line with changes noted on the radar diagram tracking the components. There could also be a key role for such a methodology to be integrated into existing government or other agency data collection processes to facilitate repeat monitoring at scale. CoBRA users could also lobby for the resilience indicators identified in the analysis of locally identified resilience characteristics to be collected as part of standard monitoring processes, if they are not already existing indicators. Radar diagrams for communities with interventions, and control communities, can provide a useful comparison to see if resilience indicators are changing over time, and hopefully improving as a result of intervention. 21

22 Annex 1 Existing and Emerging Resilience Models (A) System-wide Descriptive Models 1) TANGO / DFID Model This is a widely cited model that comprehensively maps the components of and factors affecting resilience (Please refer to Figure 1 in Section 3.2.2). It draws on livelihood models and climate change adaptation thinking in the inclusion of many factors. The TANGO/DFID model helps to conceptualise resilience as a dynamic process which ultimately coalesce to put households on positive or negative pathways 10. 2) Practical Action Vulnerability to Resilience (V2R) Practical Action s Vulnerability to Resilience (V2R) framework 11 (see figure below) highlights the key areas the affect a household or communities ability to be vulnerable or resilient and the interrelationships between them. The framework seeks to guide development programming in ways that address the core factors that underlay vulnerability. Building resilience is seen as a process that moves people permanently out of vulnerability. This is achieved by strengthening livelihoods, disaster preparedness, building adaptive capacity and addressing different areas of the governance environment. 10 Frankenberger, T., Langworthy, M., Spangler, T., and S. Nelson (2012). Enhancing Resilience to Food Security Shocks in Africa. Discussion Paper. November Pasteur, K (2011). From Vulnerability to Resilience: A framework for analysis and action to build community resilience Practical Action. 22

23 3) Fraser et al (2011) This model identifies three critical factors in influencing resilience; socio-economic assets, agroecosystem and institutional capacity. It recognizes that reinforcing benefits of all three in maximizing a household s resilience as characterized by its ability to adapt, food security and income. In the model, it is argued that the pathway to resilience is not necessarily linear and households may be placed in all parts, and move in all directions, around the cube 12. The inclusion of agro-ecosystem dimension reinforces the importance of environmental health as part of systems resilience. (B) Models or Studies attempting to Measure Resilience Several models have been (or are still being) developed or undertaken in attempt to measure multidimensional aspects of resilience. Some studies measure change over time, during or after a shock; or between target and control populations. The dimensions identified for analysis vary as does the type of analysis. A fuller summary and analysis can be found in the background and summary documents for the Expert Consultation in Rome held in February 2013 organized by FAO and WFP with funding from USAI and European Union Fraser, E.D.G., Dougill, A.J., Hubacek, K., Quinn, C.H., Sendzimir, J., and M. Temansen Assessing vulnerability to climate change in dryland livelihood systems: conceptual challenges and interdisciplinary solutions. Ecology and Society 16(3): 3. http//dx.doi.org/ /es Frankenberger,T. and Nelson, S. (Feb. 2013) Background Paper for the Expert Consultation on Resilience Measurement for Food Security ; and Summary of the Expert Consultation on Resilience Measurement for Food Security TANGO International 23

24 4) FAO (Alinovi, Mane and Romano) FAO s model involves development of a suite of latent variable indices that are derived from a number of observable indicators. These indices are then used to derive a single resilience index that is a weighted sum of the factors generated using Bartlett s scoring method and the weights are the proportions of variance explained by each factor 14 The six factors identified are then scored and weighted and plotted on a radar diagram. Scores can be plotted for locations or livelihood groups. This approach has been used to measure resilience in Kenya (using data from the Kenya integrated household budget survey) and in Palastine using a public perception survey 15. 5) University of Florence (Ciani and Romano 2013). The study conducted by the University of Florence expands on the approach developed by FAO 16 by applying it to a specific shock event. It measures food security resilience of rural households affected by Hurricane Mitch in Nicaragua in 1999 and produces a single agricultural resilience index, which is a composite index made up of 11 latent variables estimated through factor analysis. These variables were: Income and Food Access; Access to Basic Services; Agricultural Assets; Non-agricultural Assets; Household Production technological level ;Public transfers; Private transfers; Adaptive capacity; Physical connectivity; Economic Connectivity; Household demographics. Though based on the FAO model, the inclusion of additional characteristics enables the assessment of which households are able to tap into alternative options for taking advantage of opportunities and accessing the resources needed in order to deal effectively with shocks, i.e., to adapt (Alinovi et al. 2008, 2010). 15 Alinovi, L., Mane, E. and Romano, D Measuring Household Resilience to Food Insecurity: an Application to Palestinian Households ; EC-FAO Food Security Programme and Livelihood Strategies and Resilience to Food Insecurity: An empirical analysis for Kenya ; Ciani, F. and Romano, D Testing for Household Resilience to Food Insecurity: Evidence from Nicaragua University of Florence 24