UNODC Independent Evaluation Unit Evaluation Quality Assessment Template

Transcription

1 UNODC Independent Evaluation Unit Evaluation Quality Assessment Template General Project Information Project/Programme Number and Name Thematic Area Geographic Area (Region, Country) Type of Evaluation (final/ midterm) Evaluator(s) Date of Evaluation Quality Assessment conducted on/by OVERALL QUALITY RATING: [Insert overall Assessment Level based on highest score] SUMMARY: [Insert summary of assessment - maximum 10 lines] Quality Assessment Criteria 1. Structure, completeness and clarity of report a. Format (headings, font) accords to IEU Guidelines and Templates for Evaluation Reports. b. Structure accords to IEU Guidelines for Evaluation Reports: Executive Summary; Summary Matrix of Findings, Evidence and Recommendations; Introduction (Background and Context, Evaluation Methodology, Limitations to the Evaluation); Findings (Relevance, Efficiency, Partnership and Cooperation, Effectiveness, Impact, Sustainability, Human Rights and Gender, Innovation); Conclusions; Recommendations; Lessons Learned c. Language is adequate in quality and tone for an official document. Assessment Levels Very Good Good Fair Unsatisfactory Please insert assessment level followed by your main comments.

2 d. Visual aids, such as maps and graphs convey key information Report is free from any grammar, spelling, or punctuation errors. e. Report adequately addresses the objectives stated in the terms of reference. f. Annexes include at a minimum: evaluation terms of reference; list of persons interviewed and sites visited; list of documents consulted; evaluation tools used. g. Report adequately addresses and complies with human rights and gender UNODC and UNEG guidelines for evaluation. h. The report contains a logical sequence: evidence-assessment-findingsconclusions-recommendations. 2. EXECUTIVE SUMMARY a. Provides an overview of the evaluation, is written as a stand-alone section and presents main results of the evaluation. Structure: Purpose, including intended audience(s); Objectives and brief description of intervention; Methodology); Main Conclusions; Recommendations. b. The evaluation matrix presents only the key and most important recommendations from the evaluation report, and specifies the stakeholder to whom they are addressed. c. Findings, sources and recommendations in the evaluation matrix are clear and cohesive. d. Maximum length 4 pages, excluding the evaluation matrix. 3. BACKGROUND OF PROJECT/PROGRAMME EVALUATED a. Report presents a clear description of the project evaluated. b. Logic model and/or the expected results chain, and /or program theory (that at a minimum identifies and links objectives, outcomes and indicators of the project) is clearly described. c. Context of key social, economic, demographic, and institutional factors is described, and the key stakeholders involved in the project implementation and their roles are identified. d. Project s implementation status is described including its phase of implementation and any significant changes (e.g., plans, strategies, logical frameworks) that have occurred. e. Composition of Evaluation Team.

3 4. EVALUATION PURPOSE AND SCOPE a. Purpose of the evaluation is clearly defined, including why the evaluation was needed at that point in time, who needed the information, what information is needed, how the information will be used. b. Report provides a clear explanation of the evaluation objectives and scope including main evaluation questions, and describes and justifies what the evaluation did and did not cover. 5. EVALUATION METHODOLOGY a. Methodology is a transparent description of the data collection methods and sources; rationale for selecting them are explained; uses qualitative and quantitative data. b. Methodology includes a sampling frame and describes rationale for selection, mechanics of selection, numbers selected out of potential subjects. c. Methodology presented allows for triangulation. d. Methodology highlights limitations of the evaluation, how these were addressed, and the impact on evaluation findings, if any. e. Human rights and gender-responsive evaluation methodology, methods and tools, and data analysis techniques are selected. f. Methodology includes a high degree of participation of internal and external stakeholders, including the Core Learning Partners, throughout the evaluation process.

4 6. FINDINGS AND ANALYSIS To ensure sound analysis and credible findings: Findings a. Have been formulated clearly and are based on rigorous analysis of the data collected. b. Address all evaluation criteria and questions raised in the ToR including relevance, efficiency, effectiveness, impact and sustainability, as well as UNODC s additional criteria of design, partnership and cooperation, innovation, and the cross-cutting themes of human rights and gender. c. Are triangulated. d. Are presented in a clear manner. e. Address any limitations or gaps in the evidence and discuss any impacts on responding to evaluation questions raised in ToR. f. Discuss any variances between planned and actual results of the project (in terms of objectives, outcomes, outputs). Analysis a. Interpretations are based on carefully described assumptions. b. Contextual factors are identified. c. Cause and effect links between an intervention and its end results (including unintended results) are explained. d. Substantive analysis of gender and human rights issues. 7. CONCLUSIONS a. Conclusions have been formulated clearly and are based on findings and substantiated by evidence collected. b. Conclusions were developed with the involvement of relevant stakeholders. c. Conclusions present a comprehensive picture of both the strengths and weaknesses of the project. d. Conclusions go beyond the findings and provide a thorough understanding of the underlying issues of the project and add value to the findings. 8. RECOMMENDATIONS a. Recommendations are formulated clearly and are based on the conclusions and substantiated by evidence collected. b. Recommendations are specific, realistic, time-bound and actionable. c. Recommendations are clustered and prioritized. d. Recommendations clearly identify a target group for action.

5 9. LESSONS LEARNED a. Lessons learned are correctly identified and add value to common knowledge. b. Lessons learned are based on specific evidence and analysis drawn from the evaluation. c. Lessons learned have wider applicability and relevance to the specific subject and context. d. Lessons presented are innovative. Scoring Element of the evaluation Points Very Good Good Fair Unsatisfactory PRESENTATION AND COMPLETENESS 10 EXECUTIVE SUMMARY 5 BACKGROUND OF PROJECT 5 EVALUATED EVALUATION PURPOSE AND SCOPE 5 EVALUATION METHODOLOGY 10 FINDINGS 35 CONCLUSIONS 10 RECOMMENDATIONS 15 LESSONS LEARNED 5 Total Maximum Score 100