Developing Climate Change Adaptation Indicators

Transcription

1 Developing Climate Change Adaptation Indicators Approaches and lessons learnt across Europe Wolfgang Lexer, Markus Leitner (Environment Agency Austria) IN-SEI, MoEP, Jerusalem, Israel, 28 th October 2015 istockphoto.com/clintspencer Content Overview of national MRE systems for adaptation in Europe Purposes and triggers Adaptation indicators Roles and benefits Indicator types Challenges and limits Approaches taken in European countries to cope with the challenges (plus examples) Lesson learnt and key messages 1

2 Introduction Monitoring, reporting and evaluation (MRE) of adaptation in European countries is linked to national adaptation strategies / action plans evolving in a very dynamic way taking varied and diverse approaches across countries Overview: national adaptation strategies in Europe 22 European countries have a politically adopted NAS in place 12 countries have developed a national adaptation action plan (NAP) Source: Climate-Adapt European Climate Adaptation Platform ( 2

3 Status of national adaptation policy processes across Europe Monitoring & evaluation 13 European countries report that they are in the implementation or the monitoring and evaluation stage of the adaptation policy cycle Decision Source: EEA Report 4/2014: National adaptation policy processes in European countries Status of national adaptation MRE in Europe Overview Source: EEA Technical report x/2015: Monitoring, reporting and evaluation of adaptation at national level Lessons from across Europe (Draft as of October 2015) 3

4 Key concepts and terms Monitoring, reporting and evaluation are distinct, but closely linked processes Monitoring: continuous process of tracking and examining progress in planning and implementing adaptation interventions as well as climate change impacts, other drivers of vulnerability or resilience, contextual factors (by means of data, indicators, etc.) Evaluation: systematic and objective assessment of the effectiveness of adaptation plans, policies, actions at defined points in the policy/program cycle (e.g., ex ante, mid-term, ex post) Further evaluation criteria may include: efficiency, relevance, equity Based on qualitative and quantitative data (incl. from monitoring) Reporting: formal communication of monitoring and evaluation information voluntary/internal; external/statutory, e.g. UNFCCC or EU reporting requirements Status of MRE schemes in Europe Eight countries are currently implementing a monitoring, reporting or evaluation system: Austria, Finland, France, Germany, Lithuania, Spain, Switzerland, the UK Six additional countries are currently developing MRE systems 11 countries are planning to do so in the future Source: EEA Report 4/2014: National adaptation policy processes in European countries 10 countries are implementing or developing indicator-based MRE-systems (as of 2014) 4

5 Purposes of national MRE systems Overarching goals Learning: adaptation as a still relatively new field and a cyclical process => what works well (or not), in which circumstances, and why? Accountability, transparency: legitimation of adaptation as a public policy with public funding Effectiveness: are adaptation interventions effective in accomplishing their goals? Is progress towards desired outcomes being made? Efficiency: Appraisal of costs and benefits Equity: Distribution of climate impacts and effects of adaptation measures across social groups; addressing needs of socially vulnerable groups Purposes of national MRE systems Drivers and objectives in European countries Tracking progress in implementation of national strategies/action plans and effectiveness of policies/actions Enhancing the knowledge base: detecting trends in climate impacts and vulnerabilities Policy learning: Revision of national adaptation strategies/action plans Application of MRE results into policy and planning Improvement of mainstreaming, coordination, governance Communication: Awareness raising, enhancing society s preparedness and adaptive capacity Legal requirements, political commitment to regular MRE in NAS documents, administrative good practice European and international reporting requirements 5

6 Main purposes in European countries Source: EEA Technical report x/2015: Monitoring, reporting and evaluation of adaptation at national level Lessons from across Europe (Draft as of October 2015) Adaptation indicators: why they matter Roles of indicators in the adaptation process: General: Representative parameter for measuring/characterizing relevant attributes of a system/entity in order to provide meaningful information about its state (reducing complexity of the real world) Critical means by which adaptation processes, outputs and outcomes are tracked and assessed Signposts of implementation progress; clues on whether and how change is occurring and if outcomes are being achieved Aid (auxiliary means) for reviewing the degree of accomplishment of adaptation objectives 6

7 Benefits of adaptation indicators Allow comparison (between countries, regions, sectors, climate risks) Help identifying vulnerability hot spots and prioritizing adaptation investments Data-based indicators perceived as objective and robust Quantitative indicators are able of communicating clear and simple messages (appealing to policy makers) Standardized indicators allow repeated measurement and establishment of time series => identification of trends Indicator systems for various policy fields and topics exist in most countries => use of pre-existing indicators is often possible, beneficial in terms of efficiency, and useful regarding mainstreaming of adaptation Adaptation indicators: challenges and limits Lack of universal, clearly defined and measurable objective Climate mitigation: reduction of GHG emissions and/or conservation of storage capacities of carbon sinks Changes are easier to track through relatively straight-forward quantitative measures (carbon dioxide equivalents) Climate adaptation: more varied, diffuse and subjective objectives; diverse dimensions and aspects No single measure of adaptation What to monitor and how? How to measure reduced vulnerability/ increased resilience? What constitutes successful adaptation? Indicators can only capture partial aspects of complex (and vaguely formulated) adaptation goals Choice of aspects to be monitored/evaluated and selection of indicators is crucial! 7

8 Adaptation indicators: challenges and limits Careful selection of indicators is essential Everything is an indicator of something, but nothing is an indicator of everything (Cairns et al. 1993) Overriding requirement: relevance, meaningfulness, ability to show trends Trade-offs between scientific accuracy and practicability Scientific quality requirements: (statistical) validity, credibility, robustness, replicability, traceability, transparency, legitimacy => can be demanding and expensive Practice-related requirements: user-friendly, practicable, feasible, costefficient, easy to manage, accepted, easy to understand and to communicate Compromises are inevitable => pragmatic approaches are required! Adaptation indicators: challenges and limits Long-term time scales: Long-term nature of climate change processes; time lags until adaptation actions taken become effective Trackingadaptation impacts and outcomes is difficult and at present mostly not possible yet Uncertainties: Incomplete understanding of the climate system; future development of socio-economic drivers of vulnerability Makesevaluating adequacy ofadaptation actions from today s perspective difficult Establishing baselines Adaptation occurs within a dynamic context (long timeframes, uncertainties, climatic and non-climatic drivers) Socio-cultural factors may change over time as well: values, perceptions (e.g., extreme events may change the public perception of acceptable risk levels) Points of reference against which adaptation progress is measured may change as well => shifting baselines 8

9 Adaptation indicators: challenges and limits Attribution problem: Uncertainties, long time scales, and many other external factors (incl. other policies, autonomous adaptation, context conditions) make it difficult to attribute changes in vulnerability/resilience to specific adaptation interventions Indicators cannot establish plausible causal linkages between adaptation policies, actions and outputs on different levels, and their impacts and outcomes (behavior changes of target groups; changes in vulnerability/resilience levels) Efforts to mainstream adaptation into other sector policies make it even harder to disentangle the contribution of adaptation to given outcomes Indicators can describe or characterize trends, but they cannot explain and provide the whole narrative Indicators can only provideindications ofprogress and performance => whether the track is likely to lead to desired outcomes in the future Makes monitoring and evaluating effectiveness/ performance of adaptation (outcomes/impacts) particularly difficult Over-reliance on indicators can lead to false conclusions Adaptation indicators: challenges and limits Aggregation of indicators: can lead to loss of information and over-simplification Lack of data, economic feasibility, resource constraints Limitations in data availability and data quality (spatial coverage, spatial and temporal coherence, periodical data collection, validity, etc.) Collecting new data and applying new indicators is often not feasible in terms of cost-benefit ratio Use of pre-existing indicators as proxies can raise questions regarding relevance, validity, attribution Compromises between demands and feasibility are often necessary 9

10 In a nutshell: Summary of M&E approaches, methods & challenges Bringing together: o What to monitor? o How to monitor? o How to cope with the conceptual, methodological and practical challenges? Zoller, M. et al. (2015): Tracking climate change in Switzerland (presentation at the ECCA Conference, May 12 th 2015, Copenhagen) Types of adaptation indicators From an adaptation perspective Indicators can be used to measure: Climate impacts (e.g., changes in frequency and/or intensity of drought periods, changes in riverine runoff regimes) Adaptation policy responses and actions (e.g., funding of agricultural irrigation, revised water management plans, measures to reduce water consumption) Vulnerabilities (e.g., rate of residential development in a floodplain, share of climate-sensitive tree species in forest ecosystems) Realisedclimate losses (annual damage costs from flooding, annual agricultural productivity losses from drought periods, change in heatrelated mortality) Example Germany: 102 indicators across 15 fields of action => mostly impact and response indicators 10

11 Types of adaptation indicators From a policy cycle and evaluation science perspective Process-based indicators: tracking progress in planning and implementing adaptation actions, based on defined key stages in a process that could realistically be expected to contribute to positive adaptation outcomes Output-based indicators: tracking direct results of adaptation policy or actions, without assessing yet if these results actually lead to better adaptation outcomes (e.g., sectoral adaptation plan prepared, information provision, installment of an early warning system, x km of upgraded coastal defenses constructed) Outcome-based indicators: measuring explicit effects of adaptation actions taken against adaptation goals (measures for changed vulnerability/resilience/adaptive capacity levels, e.g. number of buildings exposed to flood risk reduced, decrease in annual damage costs) Example M&E concept Switzerland: further differentiation between adaptation outcomes: behavioral changes of actors (e.g., reduced water consumption of households during drought periods) adaptation impacts: degree of goal achievement, effectiveness (e.g., regional water shortages during drought periods avoided) Qualitative methods: benefits and values Often used to complement quantitative indicator-based systems Qualitative approaches based on stakeholder involvement help coping with challenges and barriers of quantitative monitoring: Preventing over-reliance on quantitative indicators Helping to validate quantitative evidence Facilitating a deeper understanding of causes ( why ) and processes ( how ), barriers and enablers; supporting interpretation of quantitative monitoring results Providing a clearer picture and broader narrative about causalities and contextual factors leading to adaptation progress (or not) => helps overcoming the attribution problem Involving stakeholders responsible for implementation of adaptation fosters learning, builds commitment, and can enhance coordination efforts 11

12 Methods to gather qualitative evidence General approaches: Self-assessments by stakeholders responsible for / affected by implementation of adaptation measures Consultation of experts (including from other countries) not directly involved in implementation Methods used: Stakeholder surveys (questionnaires) Workshops In-depth interviews Consultations Expert panels or groups Aggregation of monitoring data by means of qualitative indicators Some countries aggregate and summarize monitoring results for fields of action or themes by means of qualitative indicators to illustrate progress (e.g., Finland, UK, Belgium) by blending quantitative indicators with qualitative information used as reporting means / communication method Cf. examples from Finland and UK on the following slides 12

13 Example Finland Level of adaptation rated for each sector on a five-step scale (based primarily on process-based data and stakeholder views) Source: Ruuhela, R. et al. (2013):Monitoring and Evaluation of National Adaptation Strategies. Input presentation at the C3-Alps expert-level workshop, Vienna, 7 June, Example UK 1: flooding Traffic light system to highlight directions of trends and implications for risk levels in thematic areas Assessment based on combination of quantitative indicators and qualitative information Source: Website UK Committe on Climate Change ( rg.uk/chartsdata/adaptationindicators) 13

14 Example UK - 2: agriculture and forestry Traffic light system to highlight progress for key adaptation priorities in each thematic area Assessment of trends based on combination of quantitative indicators and qualitative information Source: ASC (2015): Reducing emissions and preparing for climate change: 2015 Progress Report to Parliament Conceptual models for mapping intervention flows Conceptual models of causal relationships (climate impacts adaptation policies and actions outcomes; e.g. by working backwards from the intended outcomes) can help attributing actions to outcomes Example Switzerland: Development of the national MRE system is based on an Impact Model => demonstrates the expected cause-effect relations and underpins the intervention logic of adaptation policy making Example Netherlands: Monitoring Wheel uses critical signposts along the implementation pathway 14

15 Example: Impact Model Switzerland (overview) Zoller, M. et al. (2015): Tracking climate change in Switzerland (presentation at the ECCA Conference, May 12 th 2015, Copenhagen) Outputs to be provided by the federal government: legal framework water management policies and instruments creating knowledge base (studies, pilot projects) etc. Example: Swiss Impact Model Water Management Behavior change (outcomes) ofsubnational governance levels (cantons, regions, municipalities): sensitization, knowledge, planning Implementation of actions Effects of changed behavior (impacts): Sufficient water resources permanently available for various uses in the required quality and quantity Water distribution adapted to changing hydrological regimes and adequate for the needs of society, economy and ecology Sewage and drainage systems able to cope with more intense heavy precipitation events Transport capacity of inland water ways maintained etc. Source: Steinemann et al. (2014): Indikatoren für die Wirkungsanalyse der Anpassung an den Klimawandel. Arbeitsbericht. 15

16 Summary of MRE approaches across Europe MRE systems are motivated by a variety of purposes across Europe, depending on the national context => (policy) learning is the most constant goal 10 European countries are implementing or developing indicators as part of their MRE systems (as of 2014), and indicators play a prominent role in these MRE frameworks Focus has so far been on the monitoring dimension (much less on evaluation), with greater emphasis on indicators to track progress (of planning and implementing adaptation) Countries are using indicator sets combining indicators of different types and for different aspects of adaptation Both quantitativeandqualitative indicators are applied, with an emphasis on quantification efforts Summary of MRE approaches across Europe Types of indicators employed vary between countries, but indicators of climate impacts/vulnerabilities and responses (outputs and processes) predominate Limited number of outcome indicators reflects the respective challenges Most countries active in MRE are complementing quantitative data with qualitative evidence from stakeholder surveys or selfassessments Good experiences have been made with such mixed methods approaches drawing upon multiple data sources => supports analysis and interpretation of M&E information Most countries have taken pragmatic approaches by including pre-existing indicators in their indicator sets 16

17 Lessons learnt and some key messages Purposes for developing national MRE schemes influence the methodological approaches and use of MRE results MRE systems should be flexible to allow for learning and improvement Methods employed should be appropriate to the national context (adaptation needs, priorities, data availability, resources) => there is no one-size-fits-all approach A mixed methods approach blending quantitative and qualitative information provides a strong basis for analyzing and interpreting adaptation progress and performance Quantitative methods are important, but should be complemented with qualitative indicators revealing causalities and contextual information and explaining the narrative behind the numbers Only sets of indicators can provide a coherent and robust picture of adaptation progress Clearer formulation of the objectives of adaptation policies is in favor of identifying outcomes Lessons learnt and some key messages Conceptual models linking long-term outcomes to building blocks along the chain of interventions can help tracking effectiveness of adaptation Governance: engagement of stakeholders in developing and implementing MRE is important, but also presents challenges (e.g., the local level has so far been difficult to involve) Evaluation:few experiences are available on the actual application of MRE results for adaptation policy and practice Careful consideration of the linkages between the purposes ( why ), methods ( how ), actors ( who ) and the use of results is necessary (to avoid that comprehensive MRE systems fail to capture lessons and are not used for effectively improving adaptation policy, planning and management) 17

18 Contact & Information Environment Agency Austria Dep. Environmental Impact Assessment & Climate Change Wolfgang Lexer Markus Leitner P: P: wolfgang.lexer@umweltbundesamt.atmarkus.leitner@umweltbundesamt.at Umweltbundesamt IN-SEI Jerusalem 28 October