UNESCO Office Kabul, Afghanistan November 2015 Terms of Reference for LEAP II Final Evaluation Consultant Post Title: Final Evaluation Specialist Organization: UNESCO Office Kabul Location: Kabul, Afghanistan Duration: 6 weeks (from 11/01/16 to 25/02/16) Nationality: International Grade: International P3 level Date Announced: November 26 th 2015 Closing Date: December 10 th 2015 1. Introduction Data from the Ministry of Interior Affairs of Afghanistan (MoI) shows that in 2010 about 70% of the 155,000- strong police force was functionally illiterate, which is a major drawback to quality policing and law enforcement. In June 2011 UNESCO launched in 19 of the 34 Afghan provinces the two-phase Literacy for Empowering Afghan Police project to assist the MoI in establishing a basic structure and system for police literacy training in the country. This was made possible through the Government of Japan s financial assistance amounting to six million US dollars since 2011. During its first phase, the project developed a network of 500 volunteer police literacy facilitators and a fullfledged team within MoI Literacy Unit that is now fully functional leading the provincial training activity. Four national-level senior master trainers and twenty provincial-level master trainers have since been trained in police literacy, which were all integrated in the MoI and have been providing literacy classes since the programme s ongoing second phase was launched in October 2013. The main objective of the second phase was to build on the existing framework established during the first phase, while enhancing the institutional capacity of MoI s Literacy Unit in planning, implementation, management and monitoring. UNESCO LEAP delivers quality literacy training for Afghan police officers while ensuring the quality of existing police literacy classes through institutional capacity building of the Literacy Unit of MoI. This capacity building process is key to the sustainability of the programme by ensuring that MoI can effectively support literacy training and classes, producing as well of literacy materials. UNESCO has also been supporting MoI in liaising with the Ministry of Education Literacy Department to ensure learners certification. Based on lessons learnt in the first phase, UNESCO has been providing additional technical support and monitoring programme activities to further strengthen MoI s capacity and ownership and ensure sustainability of the initiative. This included a sample survey to obtain accurate data on literacy levels to further develop a national police literacy strategy. The survey collected information on demography, educational attainment (if any), literacy classes and general literacy of patrolmen/women through personal interviews. Using the literacy
assessment tool developed by the survey team, the level of literacy in terms of reading, writing, understanding and numeracy is assessed. Within the framework of the LEAP project, a study has been conducted to analyze the determinants of the quality of literacy training delivery with a particular focus on the effects of incentives to facilitators. Quality of literacy training delivery is a key intermediary factor to enhancing literacy levels of the Afghan Police. The findings of this study will be used to plan new interventions as well as to adjust and tailor existing programs to sustain literacy programs within MoI. The final evaluation will integrate the findings of this study into an overall evaluative perspective of the project. Committed to the Afghanistan Millennium Development Goals, National Development Strategy and Compact, UNESCO as a specialized agency of the United Nations system promotes Education, Science, Culture, Communication and Access to Information in order to foster peace through dialogue, tackle poverty, strengthen sustainable development, uphold respect for diversity and human rights, build knowledge in societies and media pluralism and work towards gender equality, thereby helping to bring about a more successful and prosperous Afghanistan. The project is in line with: - UNESCO s mandate of supporting Member States to develop education systems to foster high quality and inclusive lifelong learning opportunities for all; - the global strategy of Education for All to ensure that the learning needs of all young people and adults are met through equitable access to appropriate learning and life-skills programmes; - the National Literacy Strategy 2. Main Objective and Responsibility of the Consultancy: The final evaluation offers the opportunity to assess the implementation of the literacy classes through the established structure as well as to identify weaknesses and strengths in current implementation strategies. The major stakeholders (MoI, UNESCO and MoE) will use the findings to identify ways in which project implementation may be improved to more effectively achieve programme goals and inputs. It will also generate recommendations to enhance the collaboration with police stakeholders and highlight the needs for further support from the donor. In making any future recommendations the evaluator will take into consideration the National Police Literacy Strategy developed under LEAP II. SCOPE The final evaluation will cover the following dimensions: - Teacher (facilitator) training - Supplementary materials and publications - Policy literacy classes - Institutional capacity development The evaluation will address the following questions (for each of the four dimensions or from an overall perspective, depending on the question) - Implementation and coordination o Who have been the main partners? o What has been the implementation and partnership approach of the project? o What were the main challenges in terms of implementation, collaboration and coordination? - Output delivery o Have the original targets been achieved? o If not, what were the main reasons? o What were the main challenges? - Outcome achievement o To what extent is there evidence that the outputs have contributed to expected outcomes? o Have there been unintended outcomes (positive or negative)? 2
o What are the main challenges to achieving the expected outcomes? - Relevance o Did the project adequately respond to the needs of the different programme stakeholders (police men and women, facilitators, MoI, etc.)? o Has the project been adequately aligned with national/institutional strategies and action plans? - Sustainability o What is the likelihood that activities, outputs (and consequently outcomes) would be sustained after the current phase of the project? o What are the key challenges to ensure sustainability? METHODOLOGY The data regarding the literacy classes will cover the 19 provinces 1) QLTS findings integrated in the evaluation 2) Desk study of the materials a. Project documents b. Documents generated by the project 3) Semi-structured interviews with key stakeholders from UNESCO, LEAP, MoI, MoE, Embassy of Japan, GIZ Interviews of former facilitators, ANP officials and MoI Literacy Unit staff will be conducted. Field class visit of the 2 nd round of classes will be organized. ROLES AND RESPONSIBILITIES The evaluation will be managed by the LEAP coordinator and conducted by an external evaluator. - LEAP team will be responsible for the logistics - The data collection will be undertaken by LEAP team at MoI (SMTs), analysis and report writing by the consultant - The consultant is responsible for the quality control 3. Deliverables: - An inception report including documentation of the current status of LEAP2 and evaluation design proposal articulating evaluation matrix and survey tools (sampling frame, questionnaire, etc.); - Guidelines for SMTs who support the data collection to feed the final evaluation - Field missions findings report; - Final evaluation report including recommendations to enhance the effectiveness and sustainability of the current delivery mechanism and inform future activities and collaboration with police literacy stakeholders. Task Timeline* Outcome/Deliverable Desk review of project document, reports and other relevant documents 11-17 Draft inception report with detailed evaluation plan and methodology prepared; including draft evaluation tools (interview sheets; questionnaires) Travel to Afghanistan 17 18 Boarding passes Briefing of evaluator by the responsible official at UNESCO LEAP and Education unit, Afghanistan 19-20 Inception report finalized, methodology and evaluation plan agreed 3
Meetings, and interviews with identified stakeholders, identified beneficiaries and collaborative partners; debriefing with the LEAP project manager and Literacy Head 21 31 Data from major stakeholders collected; Exit minutes prepared and discussed Preparation of field survey questionnaire Missions to two provinces to collect data to feed the final evaluation Preparation of draft report to include all findings- survey, interviews, meetings, desk study, etc. Receiving feedback on draft report, incorporating comments 25 4 4 14 10-21 Questionnaire Survey data/report Evaluation draft report with findings, lessons learned and results submitted to UNESCO for review Evaluation report refined Stakeholder meeting to present 23 Evaluation findings presented and findings recommendations refined Final evaluation report 25 Final evaluation report submitted to UNESCO *Subject to modification 4. Profile of the consultant: The evaluation is expected to be conducted by an external evaluator who should possess the following qualifications and skills: - Advanced university degree in social sciences, education, with specialized training in evaluation and project/programme management; - At least 5 years of international experience in designing and managing program/project evaluations including in the area of Education; - Extensive knowledge of qualitative and quantitative evaluation methods - A strong record in designing and conducting evaluations in post-conflict context - Expertise in non-formal education programme - Familiarity with Education and Adult Education in Central Asia or South Asia region; - Knowledge of the UN guiding principles on evaluation; Personal skills: - Good communication, analytical and drafting skills; - Excellent analytical and presentation skills - Ability to work in a post-conflict environment (consultant will be required to spend significant amount of time in Afghanistan) - Experience working with multiple stakeholders (governmental, non-governmental, international organizations) - Work experience in Afghanistan and/or (post)conflict environment will be considered as a strong asset - Knowledge of Dari and/or Pashtu languages will be considered as a strong asset 4
5. Supervision and reporting: The overall work under this consultancy would be supervised by Head of Education Unit and LEAP project manager. 6. Timeframe: 11 2016 25 2015 How to Apply Please send your cover letter, resume and UNESCO CV (downloadable from http://bit.ly/1go0qrg), both in English to k.human-resource@unesco.org The application deadline is December 10 th 2015 (24:00 GMT). Please note that only pre-selected candidates will be contacted. 5