Sesión 4. Rapid Evaluations in practice I. Chile Mexico

Size: px

Start display at page:

Download "Sesión 4. Rapid Evaluations in practice I. Chile Mexico"

Sheila Farmer
8 years ago
Views:

1 Sesión 4 Rapid Evaluations in practice I Chile Mexico

2 Previous Session Re Cap Four countries with RE Colombia, Chile, Mexico and US Brief background, objectives, methods, and scope of these evaluations. Some degree of standardization and synthesis in RE A common trait Pros and cons of standardization Analyze and synthesize information on design, process and impact Information comes from various sources. The importance of documental sources Some margin for field research

Some degree of standardization and synthesis in RE A common trait Pros and cons of standardization

3 Session structure Introduction The importance of reviewing examples of real RE, in order to fully understand the concept. Objectives Participants should get acquainted with the specifics of RE through the analysis of specific country experiences. Distinguish particular and general traits of RE cases. Main topics Government Program Evaluations (Evaluaciones de Programas Gubernamentales) Chile Consistency and Results Evaluation (Evaluaciones de Consistencia y Resultados) Mexico. Specific Performance Evaluation (Evaluación Específica de Desempeño) Mexico.

Distinguish particular and general traits of RE cases.

4 Example of a RE: CHILE Program: Subsidy Program for College/University Selection Test (Programa de Subsidio para la Prueba de Selección Universitaria) Evaluation of Government Programs (Evaluación de Programas Gubernamentales.)

para la Prueba de Selección Universitaria) Evaluation of

5 Brief background on RE in Chile After establishing an Index system for government programs, the Chilean administration decided that in order to strengthen accountability and performance management and budgeting, an evaluation system had to be developed. In 1997, Chile incorporated an ex-post type of evaluation called Evaluation of Governmental/Government Programs (Evaluación de Programas Gubernamentales). Since EGP/EPG s were established,and up until 2005, more than 158 public programs have been evaluated.

In 1997, Chile incorporated an ex-post type of evaluation called Evaluation of Governmental/Government Programs (Evaluación de

6 Evaluation of Governmental/Government Programs The EGP has as its starting point or theoretical background the Logical Framework approach. EGP s draw from secondary sources of information and are generally conducted in a short period of time (6 months). According to Chile s government, up until 2005 more that 60% of public expenditure via public programs was subject to evaluation.

EGP s draw from secondary sources of information and are generally conducted in a short period

7 RE in Chile Evaluation of Governmental/Government Programs First Rapid Evaluation Over 15 programs evaluated Independent/External evaluators. Based on the LogFrame Approach Scope Brief program description Evaluation results Justification Effectiveness and Quality Economy Efficiency General/Overall Performance Recommendations

Based on the LogFrame Approach Scope Brief program description Evaluation results

8 Program: Program: Subsidy Program for College/ University Selection Test EPG

18 Examples of RE Mexico Program: Child Day Care Services for Working Mothers (Programa de estancias infantiles para apoyar a madres trabajadoras (PEI)) Specific Performance Evaluation 2010/2011 (Evaluación Específica e Desempeño) Consistency and Results Evaluation 20011/2012 (Evaluación de Consistencia y Resultados)

Specific Performance Evaluation 2010/2011 (Evaluación Específica e Desempeño)

19 Brief backgroud on RE in Mexico The first rapid evaluation in Mexico was implemented in The ECR (Evaluación de Consistencia y Resultados) methodology was applied to federal social programs. Over 100 programs were evaluated with the ECR methodology in the period september 2007-march In 2008, another RE methodology was implemented. The EED (Evaluación Específica de Desempeño) was applied to over 100 programs annually from 2008 to In the period 09/2011 to 03/2012, another round of ECR (with an improved methodology) was aplied to more than 100 federal social programs.

Over 100 programs were evaluated with the ECR methodology in the period september 2007-march 2008. In 2008, another RE methodology was implemented.

20 Rapid Evaluations in Mexico 2007/2008 Consistency and Results Evaluation Evaluación de Consistencia y Resultados First Rapid Evaluation Over 100 programs evaluated External Evaluators Homogeneous Methodology Scope: Design Strategic Planning Targeting and Coverage Process and Operation Beneficiary Perception Results (impact) 100 questions 80 binary questions Written Report Executive Summary SWOT Recommendations Specific Performance Evaluation Evaluación Específica de Desempeño A synthesis of program performance Over 100 programs evaluated each year in 2008, 2009 and 2010 Homogeneous online format External Evaluators Scope: Results Output Coverage Follow up on Areas susceptible of change Written report Executive summary 2011/2012 Consistency and Results Evaluation Evaluación de Consistencia y Resultados Second round of this RE Improved methodology External Evaluators Homogeneous methodology Scope Design Planning and orientation towards results Targeting and Coverage Operation Beneficiary Perception Measurement of Impacts/Results 51 questions 34 binary questions with a scale 1-4 to be judge by the evaluator. Written Report Executive Summary SWOT Reccomendations

SWOT Recommendations 2008-2011 Specific Performance Evaluation Evaluación Específica de Desempeño A synthesis of program performance Over 100 programs evaluated each year in 2008, 2009 and 2010

21 Program: Child Day Care Services for Working Mothers EED 2010/ especificas/edd_2010_2011/sedesol.es.do

31 Examples of ER: Mexico Program: Child Day Care Services for Working Mothers Consistency and Results Evaluation (ECR) evaluaciones/consistencia/evaluaciones_de_consistencia_y_ resultados_ es.do

33 ECR: Executive Summary : Child Day Care Services for Working Mothers (Programa de estancias infantiles para apoyar a madres trabajadoras (PEI))

35 Consistency and Results Evaluation: Child Day Care Services for Working Mothers (Programa de estancias infantiles para apoyar a madres trabajadoras (PEI)) Example of Design Questions

37 Consistency and Results Evaluation: Child Day Care Services for Working Mothers (Programa de estancias infantiles para apoyar a madres trabajadoras (PEI)) Example of Planning and Orientation towards Results Questions

39 Consistency and Results Evaluation: Child Day Care Services for Working Mothers (Programa de estancias infantiles para apoyar a madres trabajadoras (PEI)) Example of Targeting and Coverage questions

41 Consistency and Results Evaluation: Child Day Care Services for Working Mothers (Programa de estancias infantiles para apoyar a madres trabajadoras (PEI)) Example of Operation questions

43 Consistency and Results Evaluation: Child Day Care Services for Working Mothers (Programa de estancias infantiles para apoyar a madres trabajadoras (PEI)) Example of Beneficiary Perception questions

45 Consistency and Results Evaluation: Child Day Care Services for Working Mothers (Programa de estancias infantiles para apoyar a madres trabajadoras (PEI)) Example of Measurement of Results questions

47 Consistency and Results Evaluation: Child Day Care Services for Working Mothers (Programa de estancias infantiles para apoyar a madres trabajadoras (PEI)) Example of SWOT Analysis questions

49 Consistency and Results Evaluation: Child Day Care Services for Working Mothers (Programa de estancias infantiles para apoyar a madres trabajadoras (PEI)) Example of Synthethic Assesment questions

Developing Diagnostic Tools for National Monitoring and Evaluation Systems: A Pilot Experience in Latin America

IDEAS Global Assembly 11 15 April 2011 Amman, Jordan Session 6.7 Complexity of Assessing Evaluation Approaches Developing Diagnostic Tools for National Monitoring and Evaluation Systems: A Pilot Experience