CONSULTANCY Education Program & Organizational Evaluation Funded by Dan Church Aid and Christian Aid (DCA/CA) BACKGROUND Marginalized children in Phnom Penh who need to work to survive and/or supplement their family income experience interrupted schooling, while many of them have never accessed education. Even if a family aspires to send children to school, the lack of school uniforms, personal hygiene and money for the many formal fees (and informal costs) means those children are generally unwelcome in public schools. These children have been living in extremely vulnerable situations for years and in many cases, never been part of a structured education system. Mith Samlanh s education center welcomes over 600 children per year. On arrival, children are assessed by literacy level irrespective of physical age. While many begin with non-formal education, some receive remedial education (especially those who have been to school for a short time or have had interrupted schooling) with the objective of public school reintegration. The children are evaluated through testing in order to determine their literacy and study levels, after which they are directed to the appropriate level of study. Using creative, child centred, non-formal and formal educational methodologies, children are supported until they are able to join the public school system. The NFE methodologies are based on the Cambodian public school curriculum, and are regularly reviewed and approved by the National Institute of Education in Cambodia. Once students have received remedial education, a test is organized at the end of the school year to evaluate their readiness for public school reintegration. Once children are ready to reintegrate into public school, the final step in the reintegration process is to organize meetings with children, parents and teachers, ensure children are registered for school and provide students with the necessities for schooling such as school uniforms, and administrative documents. To ensure that newly reintegrated students in the public schools attend classes regularly, and to identify and prevent them falling behind, Mith Samlanh s follow-up case managers conduct regular visits to the children, their families and their teachers at school. These visits allow the teams to provide support (counselling and social support) to the children on an ongoing basis, while encouraging them to attend support classes at Mith Samlanh to ensure they keep up with school work and decrease dropout rates in the long run. These case managers also work closely with the public schools and inform public school teachers on how to identify and help students who are struggling at school and are at risk of dropping out or falling behind in their lessons. In addition, once attending public school, all children from previous reintegration
intakes are welcome to return to the Mith Samlanh education centre for ongoing support classes to help them keep abreast of their studies on a daily or occasional basis. Program s goal: Continue to provide remedial education and ongoing social and emotional support to children who have had no schooling or highly interrupted formal schooling; in order for them to gain the skills, experience and confidence required for successful and sustainable re-entry into the public school system. This is achieved through realisation of five key results, as follows: Program objectives: - Objective 1: Provide remedial education for out of school children with highly interrupted or no schooling in central Phnom Penh and outskirt areas - Objective 2: Ensure the sustainable reintegration, support and follow-up of children in public school - Objectives 3: Build capacity of school teachers and NGO partners to reinforce and improve implementation of the Non-Formal Education (NFE) Tool Box and best practice teaching methodologies Target groups: - Out of school children - Street living and street working children and youth - Marginalized children in community - Children and youth at high risk of abuse and exploitation 1. Evaluation Purpose Objectives and Intended Use 1.1 Overall purpose of the evaluation: The overall purpose is to document key progresses, challenges, lessons learned, and good practices of project implementation and feed into future program development and implementation strategy. 1.2 Objectives of the evaluation: The two objectives of the evaluation are: Objective 1: To assess Mith Samlanh s practices in Reaching, Reintegrating, and Retaining children into non formal and formal education system and give recommendations for future program development and improvement. Objective 2: To assess Mith Samlanh as an organisation and to provide recommendations to improve MS s organisational and management capacities. 1.3 Intended users of the evaluation: The intended users of the evaluation are: - Friends-International: the results from the evaluation will allow FI to better support its implementing partner Mith Samlanh in the areas where the evaluation has identified specific needs for improvement.
- Mith Samlanh Management Team: the recommendations from the evaluation will be utilized by MS s management to better structure, prioritize and strategize its program and OD developments. - Mith Samlanh education program: the recommendations from the evaluation will be utilized by MS to improve educational capacities and to plan for future projects /programs. - MS s partners: Evaluation findings will be share with key relevant MS s NGO partners who are providing non formal education and school reintegration. (e.g M lop Tapang, Komar Kikreay, Krousar Thmey, Domnok Toek..) - MS Program managers, teachers and school follow-up officers: the evaluation will be carried-out with MS s staffs involved in education program. Evaluation findings will be presented to the team and a specific workplan will be developed for each education component following project recommendations. - Public school directors and teachers: The evaluation s findings will be presented to school partner teachers and directors in order to improve the use of non-formal education tools in public school (e.g toolbox, games..) 2. Specific evaluation questions 2.1 Evaluation questions related to the Objective 1: To assess Mith Samlanh s practices in Reaching, Reintegrating, and Retaining children into non formal and formal education system and give recommendations for future project development and improvement To achieve this objective, the three main components of the reintegration process will be assessed with the following specific evaluation questions: a) Identification of the out-of school children: - Is the identification process of out-of-school children efficient? (inc. outreach, reintegration campaign, collaboration with local authorities and public schools) b) Reintegration : - How effective are MS procedures and practices to support out-of-school children in accessing remedial and non-formal education at MS? (inc. ratio teacher/student, use of the toolbox, training of teachers, teaching methodologies and program, extracurricular activities). - How effective is MS reintegration process to support out-of-school children in accessing public school? (inc. support for legal document, school materials, family IGAs, psychosocial support) c) Retention of children into the public education system: - How sustainable is the reintegration process to retain children in public school and prevent them from dropping-out? (inc. school follow-up monitoring system, student and family case management, training of school follow-up officers) Through an analyze of strengths and weaknesses of the project, the evaluation will provide recommendations on how the project can be strengthened or developed to better achieve school reintegration in the future.
The evaluation will examine the following aspects, as finalised and agreed with the evaluator and MS team. Relevance - To what extent is the program relevant to the needs of the target group? Is it still relevant? - In what way does the program contribute to the programme goals? (Geographically, thematically, including cross cutting issues such as RBA and gender) Effectiveness - To what extent were the program objectives achieved? Determine the level of achievement of objectives and results according to program indicators and identify constraints in cases of non-achievement. - Are there any unintended results or consequences of the program? - Assess the quality of the support provided by MS were the systems, methodologies and materials used appropriately and effectively? (including prevention work and access to services for care and support) - Assess the collaboration/coordination with government agencies, partner NGOs, technical working groups - Assess the staff structure and staff capacity to implement the program effectively. Eg. Is MS an effective manager of the programme? For instance with regards to expertise in the thematic area of the program strategic planning, staffing, resource management, infrastructure, monitoring, outreach /contact to the communities. - To what extent has School Reintegration Program been implemented effectively throughout the program and organization? Efficiency - To what extent was the program implemented in a cost effective manner? Ex. Could MS have achieved the same with fewer resources? Or could MS have achieved more results with the same resources? And what are the overall costs of the program compared to the number of beneficiaries? Impact - What has been the impact of the program intervention on the target group? - What are the most significant changes in the lives of the individual marginalized child and youth beneficiaries, families and communities that can be attributed to the project? - What has been the impact of the project, both intended and unintended, at the broader society level e.g. advocacy, government policy, civil society capacity and collaboration? Sustainability - Are the benefits from the program likely to continue after the program has ended? Why and why not?
The evaluator will analyse these key elements of the project to determine the success of the program and will provide recommendations on how the program can be strengthened or developed to better achieve in the future. 2.2 Evaluation questions related to the Objective 2: To assess Mith Samlanh as an organisation. Recommendations from the organizational assessment will enable MS to better identify the strength and weakness of its management and to improve its organizational capacities. Organisational structure, culture, policies and systems - What are Mith Samlanh s strengths and weaknesses as an organisation? - How do MS s internal policies, regulations and internal structure contribute to the organization s effectiveness? What changes are needed? - Do the organisational values reflect rights-based thinking? - Is the language that the organisation use in policies, guidelines, reports, meetings and other communications proper rights focussed? - Is the level of coordination and communication between different management teams satisfactory? What changes are needed? What duplication exists and how can it be eliminated? - How efficient and democratic is the current decision-making process at the management level? Can it be improved? How? - What is the level of transparency on management decisions within the organisation? - How effectively do MS use the Strategic Plan to measure MS s organisational performance and plan for the future? - Are MS supporting capacity building activities within an overall organisational development plan? What capacities will need to be improved among first line and second line management staffs? (communication, finance, HR, use of management tools), Will MS finance the implementation of change and followup? What are the specific risks the management team could face while implementing activities and what mitigation strategies are established? Participation and empowerment - How participatory is the organisation? - How are beneficiaries (right-holders) and partners involved in organisational decision-making? - Who makes the important decisions? - How easy does information travel up and down in the organisational hierarchy? - Are organisational procedures helping or blocking participatory approaches to the work? - Is the organisation listening to, and consulting with, right-holders and partners in assessments and monitoring? - Is the organisation providing information/being transparent about its work to right-holders? - Are stakeholders involved in organisational decisions that affect them? - Do all programmes promote people s right to information, expression, decision making and association?
Accountability - Are MS accountable to the people MS are working for, or just to MS donors, the board of directors, supporters and the government? Accountable to whom internally and externally? - What are the organisational accountability mechanisms towards partners and communities? How does the organisation report to partners and communities? - Is the organisation addressing the situation to understand the needs of the rightholders? - Are MS carrying out stakeholder analysis and are MS addressing the impact of these works? - What are mechanisms for organisational accountability? Equity, inclusion and non-discrimination - Does the workforce in the organisation (and that of its partners) reflect the diversity of society according to gender, age, disability, ethnicity, social hierarchy and religion? - Is diversity seen in different levels/hierarchy of the organisation? - Are MS taking a people-centred and gender-sensitive approach? - Are MS taking a rights-based commitment approach? - Do MS s Board members and staff have a basic understanding of the principles and standards of human rights? Are they committed to these standards? Management styles and decision-making processes - Is the management style participatory, transparent and inclusive? - How is delegation handled? - To which level staffs feel supported by the management of MS? - How are problems and conflicts within the organisation and with partners resolved? - What does leadership mean in the context of the organisation? Is staff informed, encouraged, motivated and guided; or is the management style more traditional? - Are policies for complaint mechanisms and protection/sexual harassment in place and enforced? - Gender Equity Policies and Audits? Human resource management - How does the organisation measure up the Performance managements systems (are the setting of work objectives more directly linked to rights based programming)? - Do MS have Learning and staff development plans? Career path planning (within the organisation) to retain those with rights-based skills? Effective? - Recruitment (rights-based fit and values) of staff and consultants?, - Are Job descriptions clear and inspiring/exciting for the staff? - To which level staffs feel engaged in everyday work? - Job satisfaction and stress levels? - Staff safety and security, Compensation? - How well do MS Introduce/orientate to new staff (rights-based orientation and training)? - How is supervision? - Grievance procedures?
In total there is 300 staff spread out through: - Six different community based programs (Andong, Chbar Ampov, Russey Keo, Central Phnom Penh, Meanchey, Takhmao) - Mith Samlanh central office with key managers, M&E, HR and Finance teams - Three more locations for two Transitional Homes and one Detox Center Financial management - Are human rights principles reflected in: o Fund raising strategies, styles and methodology? o Our investments and donors? o Budgeting and financial management? - How well do MS practise finance related policies, procedure and tools? - How well MS implement and take action on audit findings/recommendations? - Will MS finance the implementation of change and follow-up? 3. Scope of evaluation In broad terms, the scope of the evaluation is focused on the relevance, effectiveness, efficiency, impact and sustainability of the education program and the OD s capacities. 4. Alignment with other program evaluation MS programs cover 2 mains components: - Saving life : direct and emergency services to marginalized children, youth and their caretakers (outreach, basic medical care, referral to appropriate services, emergency support, harm reduction, detoxification etc.) - Building future : sustainable services leading to long term rehabilitation (Education, Vocational Training, Transitional Home, independent living, family reintegration, foster care, job placement, Income Generating Activities for caretakers, apprenticeship etc.) With regards to the saving life component, an evaluation and project review has been carried-out in August 2015 by Friends-International s Executive Director. Implementation of recommendations is ongoing. With regards to the Building future component, 3 programs evaluations are planned in 2015 under 3 different grants: - European Union : evaluation is planned in November 2015 and will cover Vocational Training, employment, apprenticeship and youth empowerment s areas - Caritas: evaluation is ongoing in September 2015 and is covering Transitional Home, independent living, family reintegration and foster care. - DAC/CA: the present evaluation is planned end October beginning November and will cover the Education program plus the Organization Development so that the 3 main programs of the building future components will be assessed. This comprehensive evaluation will allow Mith Samlanh to better measure the achievements, outcomes and impacts of the 3 programs and will enable cross fertilization of future project /program development and improvement. 5. Stakeholder involvement
The evaluation will be focused on a participatory process of particular importance is ensuring children and their families are involved in the evaluation and consulted at relevant times throughout the evaluation process. Mith Samlanh management team is consisted of: - One Program Director and 6 Program Coordinators top level management - Six Program Managers and four Project Managers middle level management 6. Method The evaluation will draw on existing sources of information including six-monthly and annual reports, proposal and budget as well as field visits and interviews with beneficiaries, staff and other stakeholders. The evaluation approach will include: Desk review: - A review and analysis of MS available documents and reports in order to analyze to what extent objectives and results have been reached. - A review of other related documents which are available in the country such as National Strategic Plan for School Reintegration, recent research reports, evaluation reports and policies. All documents will be provided by MS to the evaluator before the first day of the evaluation. Field visits, focus groups discussion and interviews: - Interviews and focus group discussion with key MS team members - Visits to project sites in Phnom Penh, arrange by MS. The evaluator will conduct interviews and focus group discussion with selected beneficiaries. - Interviews with key local authorities/ngo partners will be organized by MS 7. Expected output: - Drafting of evaluation report and key findings will be submitted to MS management team and DCA/CA for review and comment. - Debriefing workshop to present report and key findings will be conducted among MS management team, staff and donors. - The final evaluation report will be submitted to MS and DCA/CA as per agreed deadline. Note: the evaluation report will follow the DCA/CA format/evaluation policy. 8. Timing A detailed schedule for the evaluation will be agreed by MS and the evaluator by the first day of the evaluation. The consultant will work independently; however, he/she needs to consult with MS s team when producing time table and questionnaires for the evaluation. MS team will support in scheduling interviews, FGDs and meeting locations. The evaluation will take approximately 22 working days and should start ideally on the 1 st of November 2015 but no later than 15 th of November, 2015.
9. Requirements The evaluation will be led by an individual independent consultant or team; and will work independently. He/she will be responsible for all processes of evaluation including desk review, field visit, focus group discussion and interviews, draft and finalize the evaluation report. He/she is required to have the following expertise: - Experience in program and organizational evaluations - Excellent analytical and writing skills (English) - Knowledge on the local context, needs/issues of marginalized children and youth and their families - Knowledge or and/or experience in human rights and related sectors in Cambodia - Knowledge of non-formal education in Cambodia - Proven experience from NGO and CBO based development assistance in Cambodia and/or familiarity with the work of MS is an advantage - Proven evaluation skills such as sampling, participatory evaluation methodology, interview techniques. - Proven analytical and report writing skills Able to speak Khmer language fluently is an advantage - Practical, hands-on approach in conduction assesments 10. Compensation, Taxation and Reimbursement The consultant will be paid as agreed amount. The payments schedule shall be as a) 1st payment 50 % upon signed contract; and b) 2nd payment 50 % upon completion of all outputs and acceptance of final evaluation. Tax payment on consultant fee and related services will be withhold by MS as MS is fully responsible for the payment of any, and all taxes due to the Cambodian government. Reimbursement for unexpected expenses necessary to complete the contract will be provided within 5 days of submission of proper receipts, provided the expenses have been pre-approved by MS. 11. Application guidelines The evaluation will be procured through a request for proposals. Bidders should include a technical bid and financial bid. The technical bid should include proposed methodology for qualitative and quantitative data analysis, a workplan, a profile and CV of the evaluator and contact information for two references. Proposal must be submitted by email to hr@mithsamlanh.org by October 25th, 2015. Candidates are expected to comply with MS Child Protection Policy.