Developing Diagnostic Tools for National Monitoring and Evaluation Systems: A Pilot Experience in Latin America

Size: px

Start display at page:

Download "Developing Diagnostic Tools for National Monitoring and Evaluation Systems: A Pilot Experience in Latin America"

Nigel Parsons
10 years ago
Views:

1 IDEAS Global Assembly April 2011 Amman, Jordan Session 6.7 Complexity of Assessing Evaluation Approaches Developing Diagnostic Tools for National Monitoring and Evaluation Systems: A Pilot Experience in Latin America Idania Fernández, Jacques Lefevre, Nadia Croteau, Frederic Martin Pour un secteur public performant For public sector performance Por la excelencia del sector público

Monitoring and Evaluation Systems: A Pilot Experience in Latin America Idania Fernández, Jacques

2 Context in Latin America Also times of change and turbulence! Growth and overall emerging economies: Brazil, Chile, but also other countries such as Panama Latin America as a region went through the financial crisis relatively well At the same time, old issues of structural poverty and inequality still there and even bigger because benefits of growth were not shared among all social categories Faced with these persistent and even growing problems and pressure from citizens and civil society, governments felt the need for new solutions

the financial crisis relatively well At the same time, old issues of structural poverty and inequality still there and even bigger

3 Governments response to this situation in the region: New Integrated social protection programs Main poverty reduction strategy program: Conditional monetary transfers. Started in Mexico (Progreso in 1982 which became Oportunidades in 2000). Adopted later by: Brasil (Bolsa familia en el programa Fome zero en 2000) Chile (Chile Solidario en 2004) Honduras (Bono 10000, PRAF) El Salvador (Programa de Apoyo Temporal al Ingreso Familiar) Panamá (Red de Oportunidades d in 2006)

Started in Mexico (Progreso in 1982 which became Oportunidades in 2000).

4 Challenges faced by goverments Very complex programs: o Many actors, many partnerships o Multiple activities to conduct, coordinate, and monitor o Issues of identification and follow up of beneficiaries o Cost-effectiveness issue of delivery mechanism o Need to demonstrate outcomes and impacts o Subject to external and internal shocks Need for a solid results-based M&E system!

up of beneficiaries o Cost-effectiveness issue of delivery mechanism o Need to demonstrate

5 Challenges for setting up and implementing a results-based M&E system Starting or young M&E systems: o No baseline; systems only designed, but not fully implemented o Weak logical frameworks linking outputstooutcomestoimpacts, p o Data quality issues o Limited use of results o Limited human capacities o Limited culture of results o Multiple, ad hoc, and donor driven M&E systems for some time; now improvingi with ihmove towardsanational lm&e system In spite of all those challenges, M&E systems were put in place. Now the challenge is: to know where are we in terms of national M&E systems; and how to improve them?

Limited culture of results o Multiple, ad hoc, and donor driven M&E systems for some time; now improvingi with ihmove towardsanational lm&e system In

6 National M&E system diagnostic tool Project of SISCA with support from World Bank Concept paper by Ray Rist Objectives of the tool: Guide a structured and participatory diagnostic of the National M&E system Guide the formulation of a priority action plan to improve the performance of the system Dimensión del diagnostico D1 D2 D3 D4 D5 Herramienta de Diagnóstico del Sistema de Monitoreo y Evaluación de Programa de Protección Social (MESOC) Version 1 04 de febrero de 2011 Estructura del MESOC Definición Pre-condiciones para un sistema de M&E basado en resultados Mecanismos institucionales de M&E del programa Planificación y presupuesto del M&E del programa Desempeño real del componente de monitoreo del programa Desempeño real del componente de evaluación del programa Repeat this exercise every 2-3 years for a stepwise improvement in the M&E system (example of PEFA approach in public finance)

Evaluación de Programa de Protección Social (MESOC) Version 1 04 de febrero de 2011 Estructura del MESOC Definición Pre-condiciones para un sistema de M&E basado en resultados Mecanismos

7 Diagnostic tool structure - Part I: Diagnostic of the OVC National M&E system Key dimensions: o RBM prerequisites o Legal and institutional frame o M&E plan and budget o Monitoring implementation o Evaluation implementation Key questions: existence, quality, and use Quantitative assessment on an ordinal scale and qualitative evaluation to justify the quantitative assessment

implementation o Evaluation implementation Key questions: existence, quality, and use

8 Diagnostic tool structure - Part II: Action Plan to improve the OVC national M&E system Identification of existing relevant actions Identification of additional actions Prioritization of actions Characterization of actions (Tasks, Roles and responsibilities, Timetable, Budget and funding needs)

Identification of additional actions Prioritization of actions

9 Overall Process 1. Documentation review (international, national) 2. M&E group work sessions 3. Individual stakeholder sessions 4. Field visit 5. Workshop(s) for diagnostic and formulation of elements of an action plan 6. Analysis and report writing on diagnostic and action plan 7. Discussion and validation of diagnostic and action plan with stakeholders 8. Action plan implementation

Workshop(s) for diagnostic and formulation of elements of an action plan 6.

10 Issues based on experience so far Generic vs. tailor-made Optimal degree of disaggregation g (number of indicators) Choice of scales Identification and involvement of stakeholders (national/local) Ownership of the tool and of the process Indicators on risk in turbulent times

indicators) Choice of scales Identification and involvement of

11 This tool is not an end in itself; it facilitates a systematic, stepwise, evidence-based process of improvement of the national M&E system. Conclusion Pilot test in El Salvador, Honduras, and Panama.

12 Thank You!

1. Introduction. October, 2010. By Verónica Silva Villalobos with Gastón Blanco and Lucy Bassett

1. Introduction. October, 2010. By Verónica Silva Villalobos with Gastón Blanco and Lucy Bassett October, 2010 Management Information Systems for CCTs and Social Protection Systems in Latin America: A Tool for Improved Program Management and Evidence-Based Decision-Making By Verónica Silva Villalobos