UNESCO Regional Office in Dakar

Transcription

1 UNESCO Regional Office in Dakar Final external evaluation of the extra budgetary project on Improvement of teacher qualification and setting up of a system for the management of learning outcomes in Guinea Bissau Final report October 2015 Rui da Silva Consultant rdasilva. @gmail.com

2 Sumário executivo O presente relatório visa apresentar os resultados da avaliação final externa do projeto extra orçamental Melhoria da Qualificação de Professores e Implementação de Gestão de Resultados de Aprendizagem na Guiné-Bissau promovido pelo Ministério da Educação Nacional da Guiné-Bissau e pelo escritório da UNESCO em Dakar, com o apoio financeiro do Governo Italiano. Este projeto decorreu de 2009 a 2015 em duas fases e tem como principais eixos de intervenção a formação inicial e contínua de professores do ensino básico. A avaliação seguiu uma abordagem quanti-qualitativa recorrendo como técnicas de recolha de dados a entrevistas presenciais e por telefone (13) e à análise documental (47). Devido à instabilidade política vivida durante o período de avaliação (Setembro de 2015) a missão à Guiné-Bissau foi cancelada, limitando desta forma a avaliação realizada. Os resultados parecem mostrar que, de uma forma global, o projeto pode ser considerado como tendo um valor acrescentado no panorama atual do sistema educativo da Guiné-Bissau, principalmente tendo em conta as mudanças ocorridas no setor da educação durante a implementação do mesmo. O projeto alcançou 5 dos 9 resultados esperado e parcialmente outros dois. De uma forma geral, o projeto alcançou os seguintes resultados: i) Produção de um balanço de âmbito nacional de competências académicas (língua portuguesa e matemática) de docentes em exercício na Guiné-Bissau que lecionam do 1.º ao 9.º ano de escolaridade (taxa de resposta na maioria das regiões superior a 90%); ii) Produção de uma avaliação da formação inicial de professores do 3.º ciclo do ensino básico (7.º -9.º ano); iii) Produção de uma avaliação das competências dos alunos do 3.º ano da Escola de Formação de Professores 17 de Fevereiro; iv) Produção de um estudo sobre uma estratégia para a formação contínua de professores do 1.º e 2.º ciclos do ensino básico (1.º - 6.º ano); v) Produção de uma proposta para o apoio à elaboração do novo currículo de formação inicial de professores do 1.º, 2º e 3.º ciclos do ensino básico (1.º - 9.º ano); vi) Produção de 5 módulos e 5 guias para professores no âmbito da abordagem por competências, tendo por base a sua harmonização com os módulos produzidos pelo Banco Africano de Desenvolvimento; vii) Realização de uma formação de formadores nos módulos produzidos para inspetores formadores das delegações regionais de educação; viii) Formação em serviço de 298 professores do 1.º e 2.º ciclos do ensino básico (1.º - 6.º ano) na abordagem por competências em 5 disciplinas (língua portuguesa, matemática, expressões, psicopedagogia e ciências integradas. Após a realização de um pré-teste e de um pós-teste, verificou-se uma diferença significativa entre os resultados de todos os assuntos testados (F(6,267)=6,83, p<0,001) com a exceção da abordagem por competências. Contudo, analisados os valores médios, verificaram-se efeitos positivos em expressões, matemática e psicopedagogia; ix) Desenvolvimento de capacidades que promoveu junto do Instituto Nacional de Desenvolvimento da Educação e das delegações regionais de educação; x) Contribuiu para o desenvolvimento de capacidades do Instituto Nacional de Desenvolvimento da Educação e do Ministério da Educação Nacional; xi) Apoio à realização dos exames nacionais de 2011; xii) Participação no esforço de coordenação e harmonização com parceiros/actores nacionais do sector da educação. No que concerne ao projeto, de uma forma global, parece que pode ser considerado como complementar às restantes intervenções do setor da educação e as suas atividades complementares 1

3 à dos restantes atores da educação (e.g. BAD, UNICEF, FEC). Contudo, o seu maior desafio durante a implementação (até ao momento desta avaliação) foi a validação por parte do Ministério da Educação Nacional do programa desenvolvido para a formação inicial de professores. Outro desafio será capitalizar os estudos produzidos de forma a contribuir para o desenvolvimento do sistema educativo. Posto isto, e com base na análise de dados, podemos considerar que as principais recomendações são: i) os materiais e estudos produzidos pelo projeto devem ser capitalizados, pelo contributo que podem fornecer ao sistema educativo no contexto de mudança atual no quadro das reformas que estão a ocorrer no setor; ii) o escritório da UNESCO em Dakar deverá melhorar os mecanismos de monitorização do projeto de forma a ter acesso a mais dados mensuráveis e produzir evidências; iii) o escritório da UNESCO em Dakar deverá incluir desde a fase de design/ planeamento uma iv) abordagem sensível às questões de género; o escritório da UNESCO em Dakar poderá apoiar o Ministério da Educação Nacional a criar um mecanismo interno que apoie a harmonização e coordenação das intervenções no setor da educação; v) o escritório da UNESCO em Dakar e o Ministério da Educação Nacional poderão desenvolver orientações na educação bilingue como estratégia para a promoção da qualidade da educação; vi) vii) viii) ix) o escritório da UNESCO em Dakar e o Ministério da Educação Nacional poderão promover a implementação de um projeto piloto de educação bilingue como estratégia para a promoção da qualidade da educação, em parceria com outros detentores de interesse do setor da educação; o escritório da UNESCO em Dakar poderá continuar a apoiar o Ministério da Educação Nacional (e.g. INDE e as delegações regionais) com base na abordagem de desenvolvimento de capacidades como estratégia para a promoção da qualidade de educação e para a redução das desigualdades regionais; o escritório da UNESCO em Dakar poderá colocar mais ênfase no nível macro das politicas educativas devido ao sistema educativo poder ser caracterizado a este nível por inconsistências e descontinuidades; o escritório da UNESCO em Dakar poderá apoiar o Ministério da Educação Nacional, em conjunto com os restantes parceiros do setor, no desenvolvimento de uma estratégia nacional de desenvolvimento profissional de professores (inicial, em serviço e contínua) tendo em consideração os desafios colocados ao sistema educativo, a legislação em vigor e os estudos produzidos; x) o escritório da UNESCO em Dakar, poderá apoiar o Ministério da Educação Nacional na regulamentação da legislação existente no país e na coordenação das intervenções dos vários parceiros no setor. Estes aspetos estão relacionados com a necessidade de regulamentação da legislação existente depois da publicação da Lei de Bases do Sistema Educativo e do Estatuto da Carreira Docente, nomeadamente no que diz respeito aos exames nacionais, aos registos escolares, à educação bilingue, ao recrutamento de professores. 2

4 Executive Summary This report aims to present the results of the final external evaluation of the extra budgetary project, "Improvement of teacher qualification and setting up of system for the management of learning outcomes in Guinea-Bissau" implemented by UNESCO Dakar in close collaboration with the Ministry of National Education of Guinea-Bissau. The project was implemented during the period of 2009 and 2015 and focused its intervention on pre service and in service teacher training for the basic education. The evaluation followed a quantitative-qualitative approach. The data collection strategy used in person/phone semi-structured interviews (13) and document analysis (47). Due to the political uncertainty during the time of the evaluation (September 2015) the field mission to Guinea-Bissau as cancelled, limiting further assessment. The results show that in general the project can be considered as having an added value in the current situation of the education system of Guinea-Bissau, particularly in regard to the changes that occurred in the education system during the project implementation. The project achieved 5 of the 9 expected outputs and partially other 2. The project overall achieved: i) Conducted a national academic skills assessment of in-service teacher (grades 1 to 9) on Portuguese language and Mathematics with a response rate in the majority of the regions higher than 90%; ii) Conducted an assessment of the 1 st and 2 nd cycles (grades 1-6) basic education pre-service teacher training institution, 17 de Fevereiro, to evaluate the academic skills of students in their 3rd year of the pre-service teacher training.; iii) Conducted the above equivalent assessment of students academic skills for 3 rd cycle (grades 7-9) basic education pre-service teacher training institution, Tchico Té, ; iv) Developed a study to further elaborate a strategy for in-service teacher training from the 1st and 2nd cycle of basic education (grade 1 to 6); v) Developed a proposal for pre-service teacher training programs; vi) Developed 5 modules and 5 guides for in-service teachers within the competence based approach harmonizing them with the modules produced by the African Development Bank; vii) Training of inspectors/trainers from 11 regional education delegation from the Ministry of National Education on the produced modules; viii) Training of 298 in-service teachers in competency based approach in 5 subjects (Portuguese, Expression, Mathematics, Psycho-pedagogy and Sciences). A pre and a posttraining test was conducted, demonstrating a significant difference in scores of all subjects (F(6,267)=6,83, p<0,001), with the exception of competence-based approach related question. However, when looking to the average scores, a significant positive effect is observed only in Expression, Mathematics and Psycho-pedagogy; ix) Contributed to INDE (National Institute for Educational Development) and Ministry of National Education regional delegations capacity development; x) The project supported the realization of the national exams in 2011; and xi) The project coordinated and harmonized the activities with other education stakeholders. The project succeeded to coordinate the proposed activities with relevant education stakeholders and can be considered complementary to other interventions (e.g. UNICEF, FEC, AfDB). However the main challenge faced by the project (at the time of this evaluation) was the validation by the Ministry of National Education of the pre-service teacher training, certification, and degree system. Another challenge regard to the capitalization of the studies produced by the project in order to operate and/or develop the education system. 3

5 Taking this into consideration and the data analysis, we can consider that the main recommendations are: i) the materials and the studies produced by the project should be capitalized by partners considering the contributions that it can provide to the education system in the context of current reforms; ii) UNESCO Dakar should determine ways forward to improve the monitoring mechanisms in order to have more measurable data and generate evidence; iii) UNESCO Dakar should include an explicit gender sensitive approach since the project design/planning stage. iv) UNESCO Dakar could determine ways forward to support the MoE in the further development of the national examination system as a tool for national planning, monitoring and management for the quality of learning; v) UNESCO and the MoE could determine ways forward for the launching within the MoE the establishment of a harmonization and coordination mechanism for education interventions; vi) vii) viii) UNESCO and the MoE could develop a position paper/orientations on bilingual education developed as a strategy for quality education; UNESCO and the MoE could determine ways forward to support the implementation of the pilot bilingual education in partnership with other education stakeholders as a strategy for quality education/learning; UNESCO could determine ways forward to continue supporting Guinea-Bissau education system and MoE (e.g. INDE and regional delegations) based in the capacity development approach promoting a strategy for education quality and reducing the regional inequalities; ix) UNESCO could determine ways forward to put more emphasis in the macro policy level because the education system can be characterized by inconsistencies and discontinuities at this level; x) UNESCO Dakar could support the Ministry of National Education, in articulation with other education stakeholders, in the development of teacher professional development strategy (pre-service and in-service) based on the challenges posed to the education system and the adopted legislation; and xi) UNESCO Dakar could assist the Ministry of National Education in the regulation of existing education legislation and in the coordination of all the education partners interventions. This stems from the fact that the education act and other legislation have been published but there is little reinforcement in their application as the Ministry of National Education must put in place appropriate mechanisms, and specific aspect from these laws that need further development regarding the teachers careers, national exams, bilingual education, school records, teacher recruitment, among others. 4

6 Table of Contents Sumário executivo... 1 Executive Summary... 3 Table of Contents... 5 Acronyms... 6 Disclaimer Introduction Overview Country and project context Theory of Change Findings and Conclusions Relevance Effectiveness Sustainability Efficiency Lessons learned Challenges Recommendations Annexes Project expected results and the achievements People interviewed Reviewed documents References Figures index Figure 1 Theory of change Table index Table 1 - Number of teacher trained Table 3 - Project expected results and the achievements phase I Table 4 - Project expected results and the achievements phase II Table 5 - Reviewed documents

7 Acronyms ADPP Ajuda de Desenvolvimento de Povo para Povo AfDB - African Development Bank ECOWAS - Economic Community of West African States EFA Education for All ESE-IPVC School of Education - Polytechnic Institute of Viana do Castelo FASPEBI - Fundação para o Apoio ao Desenvolvimento dos Povos do Arquipélago dos Bijagós FEC - Foundation Faith and Cooperation INDE - National Institute for Educational Development MDG - Millennium Development Goals MoE Ministry of National Education OECD - Organisation for Economic Co-operation and Development TISSA - Teacher Training Initiative for sub-saharan Africa ToR Terms of Reference UNDP - United Nations Development Programme UNESCO - United Nations Educational, Scientific and Cultural Organization UNFPA - United Nations Population Fund UNICEF - United Nations Children's Fund WFP - World Food Programme 6

8 Disclaimer The opinions expressed in this report are those of the Consultant, and do not reflect those of UNESCO. Responsibility for the opinions expressed rests solely with the author and does not imply any kind of endorsement from UNESCO. All data gathered and presented are dully factual and truthful. The background of the project under evaluation was provided by UNESCO- Dakar staff, as well as the implementing team in Guinea-Bissau, however, opinions, conclusions, and recommendations expressed herein are those of the author based on the information (feedback) gathered. Further claims are rightfully referenced, quoting the origin of the source. 7

9 1. Introduction The project, Improvement of teacher qualification and setting up of a system for the management of learning outcomes in Guinea-Bissau aimed to develop an effective in-service and pre-service teacher training system to sustain a body of highly qualified teachers capable of delivering quality education. This report is the final external evaluation, which presents main results of the project evaluation and recommendations for UNESCO, the government and other partners who are working in the area of teacher training in the country. In accordance with the terms of reference (ToR) this evaluation aimed to: assess relevance, effectiveness, sustainability, and efficiency of the project; measure the degree of implementation of the project and the quality of the process, products and achievements in relation to what was originally intended or subsequently formally amended; develop a better understanding of how and to what extent this project is contributing to facilitating teachers in achieving better qualification and the Ministry of Education in improving in-service and pre-service teacher training systems and policies; and formulate recommendations that will be useful for designing and implementing future similar interventions. It is also expected that the evaluation will finally be used for accountability and for learning purposes by key stakeholders. In this way, the evaluation aims to gather evidence of the project s key achievements and challenges to be used to inform decision-making processes of relevant stakeholders. It is also expected to enhance understanding of teacher s issues in Guinea-Bissau allowing to orient future similar interventions. The evaluation was conducted by a single independent consultant and followed a mixed method, quantitative and -qualitative approach. The data collection included in person/phone semi-structured interviews (13) and document analysis (47). The evaluation has the two following main limitations: i) the cancellation of the field mission to Guinea-Bissau due to the political situation during the time of the evaluation (August/September 2015). As a consequence, it was not possible to include a larger group of participants and reduced the data collection to interviews and document analysis; ii) the measurement of some of the outputs due to the absence of a baseline and/or other measurable data regarding the activities. In accordance with the Terms of Reference (ToR), the evaluation had the following questions as a guide: Relevance 1. Are the 3 components on pre-service and in-service training and pilot bilingual education consistent with the political orientations in Guinea-Bissau? 2. Is the project consistent with the C/5 and C/4 (including the relevant expected results, Strategic Objectives and UNESCO s Global Priorities), the Millennium Development Goals and other international development goals? 3. Are the project interventions adapted to the local context, including the model used for the training of teachers on bilingual education? 4. To what extent are the components of this intervention aligned with complementary to/or overlapping with those of other education actors, implementing agencies or development partners? 5. To what extent is the pre-service teacher training programme proposals relevant to the pressing demand of increasing teacher s body, taking into account the high number of unqualified in-service teachers in the country while considering their competence level? 8

10 Effectiveness 6. To what extent were the in-service and pre-service teacher trainings (grades 1 to 6) improved at policy level, at implementing level, and at the teachers level? 7. To what extent were the national capacities reinforced through the implementation of this project? 8. Does the model used for the training of teachers on bilingual education prove to be useful to improve learning and is it scalable? 9. What were the major factors influencing the achievement or non-achievement of the objectives of the project? 10. Did the project have effective monitoring mechanisms in place? 11. Has the project been cost-effective, i.e. could the outcomes have been achieved at lower cost through adopting a different approach and/or using alternative delivery mechanisms? 12. To what extent has the project been gender sensitive within its interventions? Sustainability 13. To what extent did UNESCO involve targeted beneficiaries (teachers, MoE, INDE, etc.) in the design, delivery, and monitoring of the project? 14. To what extent will national appropriation of the project results be ensured and more specifically by the Ministry of Education? In particular: a. How the 5 developed training modules and guides for in-service teachers grades 1 to 6 will be used as reference at national level to deliver quality teaching in alignment with the curricular reform and the competence-based approach adopted by the Ministry? b. How the pilot in-service teacher training on the 5 revised modules and guides can be extended nationally to reinforce the capacities of all the in-service teachers grade 1 to 6? c. How the proposal of pre-service teacher training programmes will be used as a reference standard by the Ministry of Education? 15. To what extent will the benefits of national capacity reinforcement continue after UNESCO s support ceased? 16. Are any potential donors/partners interested in maintaining/further building upon the project outputs of the components on pre- and in-service trainings and on bilingual education? Efficiency 17. Have UNESCO s organizational structure, managerial support and coordination mechanisms adequately performed in function of timely delivery? 18. Could more activities and outputs have been delivered with the same resources? 19. Has UNESCO been focusing on the areas where it is best equipped to intervene, while aligning/coordinating with other developing partners? This document is organized into seven main parts. Following these introductory notes, the document provides a brief overview, including the country and the project context, and the theory of change. Following this overview are the findings and conclusions, the lessons learned, and the challenges followed by the recommendations. The seventh and final part presents the annexes that it exposes a table with the Phase I and II project expected results and the achievements, a list of people interviewed, a list of reviewed documents and the references. 9

11 2. Overview 2.1 Country and project context Guinea-Bissau is a west African small state with 1,6 million inhabitants, with an average annual rate of population increase estimated at 3% [1, 2]. The country has one of the lowest gross domestic products per capita in the world (USD1270), poor infrastructure, weak social indicators, and a poverty rate of 64,7% [2-4]. The country relies heavily on official development assistance, particularly for public services, and education is one of the sectors which depends heavily on bilateral and multilateral aid [2, 5]. Although the official language is Portuguese, only 5% of the population use it as mother tongue and around 10% purport to speak Portuguese. Most inhabitants speak one or more indigenous languages (including creole), of which there are 20 [5-7]. The Creole is the lingua franca and is spoken by about 15% as a mother tongue and 30% 40% as an second language [7]. Guinea-Bissau has experienced significant conflict and political and military instability both prior to and following independence (1973). It is a peripheral country in the world system where a myriad of international organizations intervene due to its post-colonial condition (e.g. Word Bank, UNICEF, UNESCO, African Development Bank, UNDP, WFP, EuropeAid, UNFPA). The World Bank and OECD have classified it as a fragile state and UN as a least development country and a small developing state [9, 10]. The fragility of Guinea-Bissau is linked to its political and military histories. An example of this instability is the April 2012 military coup, ending the second turn of the presidential elections, the tenth coup since independence in 1973 from Portuguese colonial rule [11] or the recent (August 2015) dismissed of the government by the current President installing a political crisis. Conditions linked to state fragility such as poor governance, repression, corruption, low levels of social cohesion, inequality and exclusion can affect education provision [12, 13]. In Guinea-Bissau examples of such effects include: i) a net enrolment rate of 67% in primary education; ii) 13% of classrooms made of quirintim (bamboo mat) iii) lack of teacher and learning materials; iv) limited number of school days due to teacher strikes, triggered by delayed distribution of salaries; v) use of double or triple shifts, while having a significant teacher body with no pre-service training and in some cases no pedagogical training at all [1]. The project under analysis was named Improvement of teacher qualification and setting up of system for the management of learning outcomes. It aims to support the rebuilding of the Bissau-Guinean education system by focusing on improvement of in-service and pre-service teacher training (grades 1 to 6) and was implemented from January 2009 to October 2015 (see phase I and II project documents for more details for the historical development of the project). The expected impact of the Project was to support the efforts of the Government to meet the national education for all (EFA) goals by supporting the implementation of its education sector plan through teacher policy development, implementation of the in-service training plan and setting up of a national system for exam. More specially: 1. Improvement of in-service teachers training modules and teachers skills (grades 1 to 6); 2. Pilot introduction of bilingual education in Creole and Portuguese (grades 1 and 2); 3. Improvement of pre-service teacher training quality (grades 1 to 6); 4. Reinforcement of national capacities; The project had four extensions. The first one aimed to continue some components and also to conduct additional activities with additional funds. The other three extensions occurred due the Guinea-Bissau political instability. The extensions caused by the political instability delayed the implementation of the project activities and consequently budget and program outputs adaptation. 10

12 2.2 Theory of Change The theory of change is based on the fundamental logic that the key elements to improve the quality of education and achieve the national EFA goals is reinforcing national capacities, teacher training (inservice and pre-service) and setting up a national examination system. In contexts of fragility 1 such as Guinea-Bissau, it is paramount to support the transition to a more stable educational system. By adopting a flexible planning the project tried to address this. The diagram (Figure 1) intends to represent the project theory of change 2. The inputs to the programme were of 1.6 million USD of project finance with a part-time coordination team in UNESCO office in Dakar, a full-time national project officer in Bissau and technical support by national experts. The theory of change asserts that a number of activities are necessary to deliver the outputs. These include the following activities: In-service teacher training o Conducted national assessment of teachers from primary and lower secondary education (Portuguese and mathematics); o Development of a teacher in-service training plan and experimental training tools for grades 1 to 4 (Module 1 Basics of the teaching profession and Module 2 Portuguese and mathematics); o Harmonization of the teacher training modules for grades 1 to 6 and revision for alignment with the curricular reform in Guinea-Bissau on the adopted competencebased approach while considering the socio-cultural context in the country and gender issues. This concerns modules on 5 subjects: Mathematics, Portuguese, Expression, Integrated Science and Pedagogy/Didactic/Ethic and Responsibility; o Elaboration of teacher guides on the 5 subjects mentioned above; o Training of trainers/school inspectors on competence based pedagogy within the 5 disciplines. Finalization of the modules and guides; o Reproduction of the modules and guides for future trainings (dissemination). Bilingual Education o Bilingual education for grades 1 to 2; o Elaboration/revision of training materials for teachers and textbooks for the students; o Selection of schools for the implementation phase; o Training of in-service teachers from the selected schools; o Monitoring of the implementation of the pilot bilingual education in selected schools. Pre-service training o Support primary education teachers training institutions reform; o Analysis of the current situation of pre-service training programs; o Organization of a national workshop to reflect on pre-service training programs; o Elaboration of proposals for national pre-service training programs (primary and lower secondary levels). Capacities and resource management by the Ministry of Education improved o Assessment of teachers qualification in primary and secondary education; o Hiring of national experts from the Ministry of Education; o Support the implementation of national exams; o Training to the use of national exams results for management purpose; o Supervision of the technical work on in-service and pre-service training by a technical consultant; 1 Refer to the lack of capacity and/or willingness to provide basic services. 2 In the red boxes are the outputs and outcomes excluded during implementation of phase II. More information please see the project progress reports from UNESCO Dakar, March 24, 201 and from January 1, 2014 April 15,

13 o Technical advice provided throughout implementation process; o Purchase of IT equipment and consumables for INDE and Escola 17 de Fevereiro (computers, photocopying machine, scanner, printer, paper, and toner). Project implemented with quality and in a timely and cost-effective manner o Elaboration of monthly and semester report; o Elaboration of monthly work plan/timeline o Mid-term and final external evaluation; o Monitoring, coordination, and supervising missions; o Elaboration of lessons learnt given the completion of each project component, and the respective recommendations; o Elaboration of project brochure. At the next level of the theory of change the outputs are expected to lead directly to immediate outcomes: The Ministry of Education is able to design its strategy on the teaching profession development while aligning towards achieving the MDGs; Improvement of qualifications of primary and secondary teachers (in-service and pre-service training); First exam session realized and results made available for all candidates; Capacities and resource management by the Ministry of Education improved; Project implemented with quality and in a timely and cost-effective manner. At the next level (outcomes), the theory is that by increasing Ministry of National Education (MoE) capacity, setting up a national examination system and improve the in-service and pre-service teacher training, the project will support Government s efforts to meet the national EFA goals as well as its political framework ( ) for the development of the Education sector The key assumptions underlying the theory of change are: Funds are available on time; Project staff and consultants appointed on time; Appropriately qualified consultants; Teacher attended trainings; Teacher implement the teaching materials; Government commitment; MoE validate the materials produced; The project is able to positively leverage and coordinate with other agencies and projects within the sector; The enabling environment for education is improved though other projects/ agencies; Decision makers validate the use of bilingual education; Change will be detected and measurable evidencing progress on the achievements of outcomes; Equipment distributed and usable; The external environment induces to success. 12

14 Figure 1 Theory of change 13

15 3. Findings and Conclusions This section is organized taking in consideration the 4 DAC-OECD standard evaluation criteria (relevance, effectiveness, sustainability and efficiency) taken in consideration during the evaluation. In the annexes there is a resume table of the project expected results and the achievements taking in consideration the project logical framework. 3.1 Relevance 1. Are the 3 components on pre-service and in-service training and pilot bilingual education consistent with the political orientations in Guinea-Bissau? The in-service and pre-service teacher training seems consistent with Guinea-Bissau orientations namely with the Education Act (2011), the Education Sector Policy Paper (2009) and the Education Sector Plan (2009). The pilot bilingual education is not consistent with the Guinea-Bissau orientations because so far the country does not have a policy regarding the use of national languages. Nevertheless, this pilot experience was approved by INDE and recognized important due the contribution that can give to improve the education quality. The timing of the project implementation was ideal because the Guinea-Bissau education system since 2009 started important transformation with the approval of several laws and the start of the Global Partnership for Education project in the country. 2. Is the project consistent with the C/5 and C/4 (including the relevant expected results, Strategic Objectives and UNESCO s Global Priorities), the Millennium Development Goals and other international development goals? The project seems consistent with: the UNESCO s Medium-Term Strategy for (document 37 C/4) and the UNESCO's Programme and Budget for (37 C/5): Global Priorities on Africa and gender and the strategic objectives/main lines of action 1 - Supporting Member States to develop education systems to foster high quality and inclusive lifelong learning for all, 3 - Advancing education for all (EFA) and shaping the future international education agenda. Teachers is equally one of the 7 thematic areas of expected results of the Major Programme I Education s Main Line of Action 1. the Millennium Development Goals (goal 2 - Achieve Universal Primary Education, and 3 - Promote Gender Equality and Empower Women), the Education for All goals (2 - Ensuring that by 2015 all children, particularly girls, children in difficult circumstances and those belonging to ethnic minorities, have access to, and complete, free and compulsory primary education of good quality, and 6 - Improving all aspects of the quality of education and ensuring excellence of all so that recognized and measurable learning outcomes are achieved by all, especially in literacy, numeracy and essential life skills). the UNESCO Convention against Discrimination in Education and with UNESCO s guiding principles regarding mother tongue in order to contribute to quality of education in multilingual countries 3. 3 See, for example UNESCO (2008) Mother Tongue Matters: Local Language as a Key to Effective Learning. Paris: UNESCO. 14

16 3. Are the project interventions adapted to the local context, including the model used for the training of teachers on bilingual education? The project intervention seems adapted to the local context. The main factors contributing to this fact are that the executing institution for this project was INDE, the national coordinators are from Guinea-Bissau, the project tried to follow a local capacity development approach 4 by strengthening existing capacities (e.g. INDE, inspectors/trainers), and by adopting a flexible planning suitable for the local context. However, INDE experienced some difficulties in executing the project especially linked to the implementation of the inservice teacher training in the regions. These difficulties are linked to education partner s trainings articulation and logistical and administrative procedures associated to the organization of these type of trainings. 4. To what extent are the components of this intervention aligned with complementary to/or overlapping with those of other education actors, implementing agencies or development partners? As to whether the project intervention was aligned i.e. complementary/or overlapping with those of other education actors it seems that the: project tried and succeeded to coordinate with education actors (specially with UNICEF, African Development Bank [AfDB] and Foundation Faith and Cooperation [FEC]) in order to be complementary and not overlap activities. The coordination/collaboration with UNICEF was frequent in both phases of the project in order to avoid duplication of activities; with AfDB the collaboration was in regard to the harmonization of the training modules and guides; and with FEC the in-service teacher training was coordinated in order to avoid duplication of trainers and trainees, while contributing with UNESCO Peace Education manual as an additional session of their training on school management and administration.; studies financed by the project 5 were complementary to the intervention of other education actors (AfDB, World Bank, UNICEF, PLAN International, FEC, UNDP, UNFPA, ADPP, Portuguese Cooperation and Effective intervention); in-service teacher training was complementary because it seems that training contents of the other partners are not the same and also the teachers are not the same. However, it seems that the different educations stakeholders are following their own approach/strategy because of the absence of a national strategy regarding the teachers professional development; project dialogue with the other education actors (e.g. UNICEF, AfDB and FEC) and the participation in phase II in the Local Education Group meetings by the project officer representing UNESCO, seems to contribute greatly to the coordination and harmonization efforts put in place, especially in phase II. 4 See, for example De Grauwe, A. (2009). Without capacity, there is no development. Paris: UNESCO-IIEP. And/or OECD The challenge of capacity development: Working towards good practice. In DAC Guidelines and Reference Series. Paris: OECD. 5 Benavente, A., Varly, P. (2010) Balanço de competências de docentes em exercício na Guiné Bissau. / Ramalho, G. (2010) Avaliação dos Alunos do 3ºAno da Escola de Formação de Professores 17 de Fevereiro. / Campos, B. (2010) Avaliação da Formação Inicial de Professores do Ensino Secundário da Guine -Bissau. Volume I. Avaliação Institucional e Pedagógica / Ramalho, G. (2010) Avaliação da Formação Inicial de Professores do Ensino Secundário da Guiné-Bissau - Volume II / Avaliação dos Alunos-Professores / Butlen, M., Martinez, C. (2011) Vers une politique et une stratégie nationales pour la formation continue des professeurs de l enseignement fondamental en Guinée- Bissau. / Barreto, M. A. Saraiva, A. M. R., Ferreira, J. B. (2010) Apoio à elaboração do novo currículo de formação inicial de professores do ensino básico (ensino primário e secundário geral) na Guiné-Bissau. Volume I e II. / Furtado, A. (2015) Planos curriculares e programas para formação Inicial de professores do ensino básico. 15

17 Regarding in-service teacher training both the project and FEC 6 use the inspectors/trainers from the MoE as trainers. Taking this into account both the project and FEC to some extent had the same target group. However, this can result in a capacity development of the inspectors/trainers in different areas with UNESCO s training focusing in competency based approach and FEC in the curriculum themes and educational management and administration. Nevertheless, FEC does not involve all the inspectors/ trainers from the country while the project involved 82.7% of total inspectors/trainers from the MoE. 5. To what extent is the pre-service teacher training programme proposals relevant to the pressing demand of increasing teacher s body, taking into account the high number of unqualified in-service teachers in the country while considering their competence level In regards to pre-service teacher training, it seems that there is a risk of neglect in regards to the revisions and program proposal produced under this project. This stems from the fact that Escola Superior de Educação do Instituto Politécnico de Viana do Castelo (ESE-IPVC) 7 signed a protocol with MoE and collaborated in pre-service teacher training from 2010 to 2012 under a Portuguese Cooperation: PASEG. Nevertheless, it seems that the two interventions can be complementary and articulated eventually. However, because of the political instability, the MoE has not yet received the pre-service teacher training programs proposal from UNESCO at the time of this evaluation. In order to establish a pre-service teacher training, certification, and degree system, two scenarios were proposed by the consultants in the reports/studies: scenario A consists in a pre-service teacher training for 3 years, leading to a bachelor s degree; and scenario B consists in a pre-service teacher training for 2 years which give access to a professional diploma that allow the access to the teacher carrier. Under this project a reflective workshop held in May 2015 in Bissau a third scenario emerged, scenario C, which integrated training that meets the needs of in-service teachers with more than 5 years of service, with professional experience and with training courses not certified. The scenarios A and B 8 seem relevant to the pressing demand for increasing teacher body. Nevertheless, the scenario C must equally be taken into consideration. This stems from the fact that the pre-service teacher training institutions are not able to train the quantity of teachers needed per year in order to achieve the first 6 years of schooling for all by 2020 (+/- 450 teachers per year) 9. There is a high number of in-service teachers in schools with no preservice teacher training and without opportunities to be integrated in in-service teacher training schools/institutions 10. These aspects are related to the holist diagnosis on teachers issues (TISSA) that initially constituted a project component. However this component was not carried out due to an internal decision Foundation Faith and Cooperation (FEC) is a Portuguese non-governmental organization that supports the education sector in Guinea-Bissau since At this moment its implementing strategy is, among others, a national in-service teacher training Program - Programa Ensino de Qualidade em Português na Guiné Bissau. 7 From the information available, ESE-IPVC is supporting a 3 years bachelor degree on teacher training: program and training itself. 8 For more information see, for exemple Furtado, A. (2015) Planos curriculares e programas para formação inicial de professores do ensino básico. And Individual Consultant Report - Activity Report n 05/ For more information see UNESCO (2013) Guiné-Bissau - Relatório da Situação do Sistema Educativo. Margens de manobra para o desenvolvimento do sistema educativo numa perspetiva de universalização do Ensino Básico e de redução da pobreza. MENCCJD. Dacar, p For more information see Benavente, A.; Varly, P. (2010) Balanço de competências de docentes em exercício na Guiné-Bissau. UNESCO/BREDA & MENCCJD, p For more information please see Project Progress Report from January 1, April 15, 2015 and The Project Progress Report from March 24,

18 3.2 Effectiveness 6. To what extent were the in-service and pre-service teacher trainings (grades 1 to 6) improved at policy level, at implementing level, and at the teachers level? It is perceived that pre-service teacher training project activities did not have direct influence at the policy level, the implementing level, and the teachers level. However, this was not a project goal. Nevertheless this process seems to produce benefits that can be capitalized in the future. This stems from the fact that the products of the project activities can be a useful tool on pre-service teacher training certification framework, institutional memory, and proposal of a pre-service teacher training program. The in-service teacher training project activities seem to have had influence in the implementation level and the teacher s level. Although it was not part of the project s goals to have direct influence at the policy both pre-service and in-service teacher training left essential orientations, recommendations, and propositions which should be best utilized for policy reform in the teaching career as well as training. From the data available in regard to the in-service teacher training 91 inspectors and inspectors/trainers from 11 regions benefitted from training in November 2014 during a 7 day-training on the competencebased approach and on the content of the 5 revised modules. In February 2015, 55 inspectors and inspectors/trainers among the 91 previously trained received training on the revised in-service teacher training modules. These inspectors/trainers training allowed the pilot test of the guides and modules. From 20 of July to 1 of August 2015, 298 teachers in 4 locations received in-service teacher training on the revised modules. More details in the table 1. Furthermore, in October 2014 UNESCO collaborated with FEC on the training of 176 heads of classes and president of study commissions from grades 1 to 6 on (1) the basic education curricula cycles 1 and 2, (2) teaching and learning methodologies and (3) planning, as well as (4) UNESCO-ECOWAS manual on Education for a Culture of Peace, Human Rights, Citizenship, Democracy and Regional Integration. In the in-service teacher training sessions held in July-August 2015, a randomized controlled trial was conducted by randomly distributing the participants between two groups (119 teachers control group and 155 teachers experimental group). Region Table 1 - Number of teacher trained Total number of teacher trained Number of female teacher trained % female teacher trained Setor Autónomo de Bissau % Bula % Gabú % Buba % Total % Taking into consideration the data analysis report, it was observed a statistically significant increases in the total scores from the control group to the experimental group of all subjects (F(6,267)=6,83, p<0,001), with the exception of competence-based approach. However, when looking to the average scores, a significant positive effect is observed only in Expression, Mathematics and Psycho-pedagogy. In Integrated Science and Portuguese there is no significant effects and is observed a negative effect on competence-based approach. Nevertheless, we must take in consideration the duration of the training. Has stated in the study analysis report some improvements should be made to the questionnaire in order to improve future interventions data analysis. 17

19 7. To what extent were the national capacities reinforced through the implementation of this project? National capacities were reinforced by the implementation of this project. This was made possible because the implementation agency was INDE, the modules were produced by national experts, the bilingual education team was from INDE, and the national officer was from Guinea-Bissau. It seems that the consultants that worked with the national experts were critical in this capacity development. The administrative support received from UNESCO Dakar office staff in order to effectively deliver the in-service teacher training contributed in addition to capacity development of INDE s financial management. 8. Does the model used for the training of teachers on bilingual education prove to be useful to improve learning and is it scalable? The effectiveness of the bilingual education was not assessed because the implementation of this activity in the selected schools did not start in November of 2014 but in The pilot phase took place in Bula, Mansoa, Safim and Setor Autónomo de Bissau (SAB) in the selected 5 plus 1 schools, with one experimental and one control class in each school. The 7 national experts were trained on bilingual education by the coordinator of this component, a manual was produced, 10 teachers in selected schools were trained by the experts and were provided with pedagogical materials, and the teachers were visited by the trainers (4 monitoring visits from May to June 2015). 9. What were the major factors influencing the achievement or non-achievement of the objectives of the project? The Guinea-Bissau political instability, the limited communication means between Dakar and Bissau in particular when the new project officer was appointed (re-launch of phase II) and installed at INDE without electricity and internet connection, and the tardive budget allocation inherent to UNESCO s administrative procedures seems to be the project main constraints. On the other hand, the commitment of the national experts, the mobilization and dialogue promoted by the national project coordinator (phase II) with the education partners and with the project team in Dakar, and the project team in Dakar were the major factors influencing positively the achievements of the project. 10. Did the project have effective monitoring mechanisms in place? From the available data, an effective monitoring mechanism was put in place. The balance between distance and field presence seems adequate to the project implementation. Although at an early stage of phase II, it seems that the communication between Bissau and Dakar was an issue. This stems from the fact that the new project officer hired in phase II needed to adapt to the project demands and the absence of day to day contact difficult her adaptation. 11. Has the project been cost-effective, i.e. could the outcomes have been achieved at lower cost through adopting a different approach and/or using alternative delivery mechanisms? It seems consensual among project stakeholders that were consulted that the project was cost-effective and the outcomes have been achieved at lower cost. The option for national consultants, the recycling of the project computer, and the reduction of the per diem initially predicted for the in-service teacher training are some examples that illustrate this aspect. In regard to the per diem, some of the national experts raised some concerns. According to them, the amount given to some teachers was not sufficient because of the longer distance that they had to travel to the training spot. However the data collected does not allow to have a full confirmation of the accuracy of this perception. 18

20 12. To what extent has the project been gender sensitive within its interventions? In the project document forms there is no evidence that the gender issue are taken in consideration. However, during the project implementation this aspect was taken in consideration in the selection of the in-service teacher training participants and in the production of the in-service training modules and guides. 3.3 Sustainability 13. To what extent did UNESCO involve targeted beneficiaries (teachers, MoE, INDE, etc.) in the design, delivery, and monitoring of the project? In regard to the involvement of the target beneficiaries in the project design, delivery and monitoring, there are several levels of participation. However it seems that the design (phase I) and the monitoring of the project only involved the project team. The revision of project document for phase II in order to contextualize and be more aligned with national priorities being put in place was completed after consulting the MoE and education partners. The MoE and INDE had a close involvement in the monitoring and in the delivery of the project, especially in phase II. The inspectors/ trainers were involved in the project delivery. In regard to the other target beneficiaries (teacher, inspectors/trainers) there is no evidence that they were involved in the project design, delivery and monitoring. 14. To what extent will national appropriation of the project results be ensured and more specifically by the Ministry of Education? In particular: a. How the 5 developed training modules and guides for in-service teachers grades 1 to 6 will be used as reference at national level to deliver quality teaching in alignment with the curricular reform and the competence-based approach adopted by the Ministry? b. How the pilot in-service teacher training on the 5 revised modules and guides can be extended nationally to reinforce the capacities of all the in-service teachers grade 1 to 6? c. How the proposal of pre-service teacher training programmes will be used as a reference standard by the Ministry of Education? It seems that the appropriation of the project results by the MoE is not ensured due to the recent (August 2015) dismissed of the government by the current President installing a political crisis. In this way the project team is waiting for the appointment of the new government to continue the appropriation of the actions completed under the project. However, the 5 training modules, the in-service teacher training and the pre-service programs have the potential to be appropriated by the MoE but several steps must be taken in order to achieve this. The 5 training modules are being revised (language, content, and competence based approach). These modules seem relevant with the current curriculum revision, however they should be revised when the curriculum reform is finalized in order to adapt them to finalized primary education programs. 15. To what extent will the benefits of national capacity reinforcement continue after UNESCO s support ceased? The benefits of the national capacity development will continue especially in INDE and in the regional delegations of the MoE. This stems from the fact that the national experts involved in the project are integrated in INDE and the inspectors/trainers will continue their functions in the regional delegations. However, due to the evaluation limitations we can t assume that INDE and/ or the regional MoE delegations are now able of continue in a sustainable way the activities. 19

21 16. Are any potential donors/partners interested in maintaining/further building upon the project outputs of the components on pre- and in-service trainings and on bilingual education? The project outputs have potential to be maintained and/or build up by other donors/partners. However, the project team should continue to establish the links with the relevant education stakeholders in order to achieve this. INDE institutional capacity development can be further supported by the Portuguese Cooperation that is in process of design a project that has this component. From the data available (interviews and documents analysis) it seems that the training modules could eventually be maintained and/or build up by UNICEF, African Development Bank, FEC, ADPP, among other education partners that have similar project activities to further develop the educative system. The pre-service teacher training has the potential to be maintained and/or build up by the ESE-IPVC. The in-service teacher training and the bilingual pilot education have the potential to be maintained and/or build up by UNICEF. Concerning bilingual education it seems that is recognized as important for quality education in regard the country multilingual context. However donors/partners are uncertain because of the absence of a political orientation in this regard by the MoE. 3.4 Efficiency 17. Have UNESCO s organizational structure, managerial support and coordination mechanisms adequately performed in function of timely delivery? In regard with the adequate performance of the UNESCO organizational structure, managerial support and coordination mechanisms, it seems that all performed well except some aspect concerning the timely budget transfer to the national experts, the in-service teacher training activities, the teacher qualification assessment and some consultants. It seems also that in the second phase of the project since the project team increased the project performance increased. 18. Could more activities and outputs have been delivered with the same resources? From the available data it seems that this project have an overall budget allocation similar to other projects in the education sector in Guinea-Bissau (e.g. Portuguese Cooperation and FEC). In this regard the project followed the trend of the country In relation to the costs associated to monetary compensations, during phase II they were provided according to the validated United Nations System Circular-GB. These compensations were mostly less than what was provided during phase I of the project. However, at no circumstances were the lower compensation (in relation to phase I) lacking justifiable authorization/decision. During phase I international experts were recruited to implement the project activities rather than national experts, who were privileged during phase II. Nevertheless, with regard to phase I, it is not possible to assess if there are national experts with the expertise/ skills necessary to conduct all the consultancies and diagnosis to support project activities and development. The recruitment of national experts during phase II might have slowed the implementation but by this way, national capacities were reinforced as well as promoted project ownership by the beneficiaries. As a consequence, it seems that during phase I, if no time constraints, more activities and outputs could have been delivered with the same resources. Concerning phase II, it seems no more activities and outputs could have been delivered with the same resources. It also seems that human, financial, and material resources were utilized in the best way possible, in order to promote sustainability, ownership and capacity development of the project beneficiaries. 20

22 19. Has UNESCO been focusing on the areas where it is best equipped to intervene, while aligning/coordinating with other developing partners? The project focused on areas that UNESCO is best equipped to intervene. The timing of the project implementation was ideal because the Guinea-Bissau education system since 2009 started important transformation with the approval of several laws and the start of the Global Partnership for Education project in the country. It seems also important to link in the future the project outputs with the qualification referential for basic education teachers in ECOWAS countries, currently being developed by UNESCO. In conclusion, by taking in consideration all the aspects referred in this report we can considerer that the project had an added value to the Guinea-Bissau education system. The project also reinforced the national capacities and was adapted to the local context and consistent with the Guinea-Bissau and UNESCO orientations. The only aspect that is not consistent with the Guinea-Bissau orientations is the pilot bilingual education because so far the country does not have a policy/political position regarding the use of national languages. However UNESCO worked closely with the MoE and INDE which finally granted support to the pilot the bilingual education component during project implementation. The project achieved 5 of the 9 expected outputs and partially other 2. Main project achievements are: i) conducted a national assessment of in-service teacher from grades 1 to 9 academic skills (Portuguese language and mathematics) with a response rate in the majority of the regions higher than 90 %; ii) conducted an assessment of the students academic skills from the 3rd year of the pre-service teacher training institution from the 1st and 2nd cycles of basic education (grade 1 to 6), 17 de Fevereiro; iii) Conducted an assessment of the pre-service teacher training institution, Tchico Té, from the 3rd cycle of basic education (grades 7 to 9) and of the students academic skills from the 3rd year of the pre-service teacher training institution Tchico Té; iv) developed a study on a strategy for in-service teacher training from the 1st and 2nd cycle of basic education (grade 1 to 6); v) developed a proposal for pre-service teacher training programs; vi) developed 5 modules and 5 guides for teachers within the competence based approach harmonizing them with the modules produced by the African Development Bank; vii) training of inspectors/ trainers from the Ministry of National Education regional delegation, on the produced modules; viii) training of 298 in-service teachers on competency based approach in 5 subjects (Portuguese, Expression, Mathematics, Psycho-pedagogy and Sciences). A pre and a post-training test was conducted, demonstrating a significant difference in scores of all subjects (F(6,267)=6,83, p<0,001), with the exception of competence-based approach related question. However, when looking to the average scores, a significant positive effect is observed only in Expression, Mathematics and Psycho-pedagogy; ix) contributed to INDE (National Institute for Educational Development) and Ministry of National Education regional delegations capacity development; x) the project supported the realization of the national exams in 2011; and xi) the project coordinated and harmonized the activities with other education stakeholders. 21

23 The project succeeded to coordinate the proposed activities with relevant education stakeholders and can be considered complementary to other interventions (e.g. UNICEF, FEC, AfDB). The pre-service teacher training programs produced by the project had not yet been validated by the MoE because of the recent political crisis which caused an emergency demolishment of the Government, and at time of this evaluation no Minister of Education had yet been appointed. It was also proposed three scenarios to establish a pre-service teacher training, certification, and degree system that should be taken into consideration in order to respond to the challenges of the education system. The project also contributed to the improvement of the in-service teacher training of 298 teachers (grades 1 to 6) from the Sector Autónomo de Bissau, Bula, Gabú and Buba regions. The teachers were trained in themes regarding the modules produced with the project support and a pre-training and post-training test was conducted to randomly assigning groups to assess training effects. The studies produced by the project have the potential to improve the pre-service and the in-service teacher training in terms of policy and in the improvement of the training system. However, they need to be capitalized by the MoE and education partners/stakeholders in the near future in order to operate and/or develop the education system and confirm their potential. As to whether the project contributed to the MoE improving in-service and pre-service teacher training systems and policies: this is under way. This stems from the fact that the project team is waiting for the appointment of a new government, in order to continue mobilizing for an adequate appropriation of the actions completed under the project. The project outputs have potential to be maintained and/or build up by other donors/partners. However, the project team should continue the appropriation and /or establish the links with the relevant education stakeholders in order to achieve this. 4. Lessons learned Guinea-Bissau s political instability requires flexible project planning; The capacity development approach (as opposed to unidirectional aid project without taking into account local capacities) seems more sustainable and produces more positive impact. This stems from the fact that it can improve the performance and skills of individuals, improve the performance of organizations and of the public administration; and improve the social, political and economic environment. However there is less measurable impact, due to the multidimensional process which depends on various means to achieve the desired results. This is divided into different levels, and is a long-term process that occurs through a series of changes, many of which relate to values, attitudes and norms and which are difficult, if not impossible to measure 12 ; The fact that the national coordinators are from Guinea-Bissau seems to be an advantage; More harmonization and coordination are needed in the education sector taking further advantage of the Local Education Group; 12 For more information please see: De Grauwe, A Without capacity, there is no development. Paris: UNESCO- IIEP. And/or Davies, Lynn "Learning for state-building: capacity development, education and fragility." Comparative Education 47 (2): doi: /

24 All teacher training requires detailed planning and national coordination in order to not duplicate interventions and beneficiaries; According to the national experts and trainees more training days and more often training were needed for the in-service teacher training. However the training period should be reviewed as it was inappropriate given the raining season and harvest period. 5. Challenges The lack of validation by a Minister of National Education (the country experienced a unpredicted and sudden disablement of the Government) of the pre-service teacher training program, certification and degree system, delaying a progressive step towards the development of this component for both the project and the country itself; Better capitalization of the studies produced by the project in order to operate and/or develop the education system; Implementation of project activities by INDE due to/as a result of some difficulties in executing the project especially linked to education partner s trainings articulation, logistical and administrative procedures associated to the organization of the in-service teacher training; Coordination and harmonization with ESE-IPVC in regard to the pre-service teacher training activities; Implementation of the bilingual education activities, particularly in regard to the absence of a national position paper/orientations on bilingual education; The Guinea-Bissau political instability; The communication between Dakar and Bissau in particular when the new project officer was appointed (re-launch of phase II); Budget allocation inherent to UNESCO s administrative procedures. 6. Recommendations UNESCO Dakar could determine ways forward to continue supporting Guinea-Bissau education system and MoE (e.g. INDE and regional delegations) based in the capacity development approach promoting a strategy for education quality and reducing the regional inequalities; UNESCO Dakar could determine ways forward to put more emphasis in the macro policy level because the education system can be characterized by inconsistencies and discontinuities at this level; UNESCO Dakar could determine ways forward to support the MoE in articulation with other education stakeholders, in the development of teacher professional development strategy (preservice and in-service) 13 based in the Lei de Bases do Sistema Educativo [Education Act], the 13 This strategy should include the pre-service and the in-serve teacher training. For more information see for example Day, C.; Sachs, J. (2010) International handbook on the continuing professional development of teachers. Berkshire: Open University Press. And UNESCO (2014) EFA Global Monitoring report 2013/4. Teaching and learning: Achieving quality for all. Paris: UNESCO. 23

25 Estatuto da Carreira Docente [Teacher Careers Statute], the national curriculum, the challenges posed by 9 years of compulsory schooling, and the findings from the studies carried out by this project; UNESCO Dakar and the MoE should determine ways forward to capitalize on the project materials and studies produced in order to use them to operate or develop the education system; UNESCO and the MoE could determine ways forward for the launching within the MoE the establishment of a harmonization and coordination mechanism for education interventions; UNESCO Dakar and the MoE could develop a position paper/orientations on bilingual education developed as a strategy for quality education; UNESCO Dakar and the MoE could determine ways forward to support the implementation of the pilot bilingual education in partnership with other education stakeholders as a strategy for quality education/learning. The results from bilingual education component supported efforts done in the past by FASPEBI (Fundação para o Apoio ao Desenvolvimento dos Povos do Arquipélago dos Bijagós) in Bubaque and Catió. It is recommended, however, that future interventions should continue in order to pilot this strategy in other regions of the country. In the future more schools/classes from Setor Autónomo de Bissau should be included (this would evidence more impact due to the concentration of the country s population (25.1%); UNESCO Dakar could determine ways forward to support the MoE in the further development of the national examination system as a tool for national planning, monitoring and management for the quality of learning; UNESCO Dakar and the MoE could determine ways forward to support the development of training modules and guides for grades 7 to 9; UNESCO Dakar could determine ways forward to assist the MoE in order to further develop the regulation of existing education legislation concerning specific aspects about national exams, bilingual education, school records, teacher recruitment, among others; UNESCO Dakar could determine ways forward to finding non-monetary compensation strategies and/or include some of the project activities in the annual plan with MoE budget allocation, in order to promote the participation of target beneficiaries in activities when external funding is not available; UNESCO Dakar should determine ways forward to improve the monitoring mechanisms in order to have more measurable data and generate evidence; UNESCO Dakar should determine ways forward to improve the project indicators following, to the extent possible, the SMART approach (specific, measurable, achievable, relevant and time-bound); UNESCO Dakar should determine ways forward to continue the adoption of a flexible planning in order to be suitable to Guinea-Bissau context; UNESCO Dakar could determine best ways to improve internal administrative procedures specially inherent to budget allocation; UNESCO Dakar should include an explicit gender sensitive approach since the project design/planning stage. 24

26 7. Annexes 7.1 Project expected results and the achievements Table 2 - Project expected results and the achievements phase I Phase Expected results Indicators Progress/achievements Observations Number of teacher in service assessed More than 90% of the in-service teachers were assessed; Phase I 1 - Evidence based teachers qualification Results of the assessment of teacher s qualification - The assessment allowed the creation of 3 teachers profiles that can be taken in consideration in the definition of a national network of in-service teacher training; - 1 report produced in Balanço de competências de docentes em exercício na Guiné Bissau - Is necessary that the in-service teacher training in the future be certified. Phase I Phase I 2 - An in-service teachers training program for primary and secondary education. More teachers trained with better qualifications. 3 - New programs for teachers trainees in primary and secondary education. Building of a pre- Number of teachers to be trained Number and popularity of usage of the training modules and tools Assessment of trained teacher qualification Content of new training programs n/a n/a n/a - New training programs were proposed. - Produced document: - Vers une politique et une stratégie nationales pour la formation continue des professeurs de l enseignement fondamental en Guinée-Bissau. (2011) About 400 teachers were trained on module I (see evaluation report phase I) - 2 reports produced in Apoio à elaboração do novo currículo de formação inicial de professores do ensino básico 25

27 service training, defining the contents. More teachers trained with better qualifications. Selection of the students n/a (ensino primário e secundário geral) na Guiné- Bissau. Volume I and II - Avaliação da Formação Inicial de Professores do Ensino Secundário da Guiné-Bissau - Volume I. Avaliação Institucional e Pedagógica (2010) Phase I 4 - Diagnostic of the pre-service training system of Secondary School (including the students achievement). Development of curriculum, Management of resources Results of the assessment of the existing institutes of training - 3 studies produced -Avaliação da Formação Inicial de Professores do Ensino Secundário da Guiné-Bissau - Volume II. Avaliação dos Alunos-Professores (2010) - Avaliação dos Alunos-Professores - Avaliação dos Alunos do 3ºAno da Escola de Formação de Professores 17 de Fevereiro (2010) Phase I 5 - Support to the implementation of national exams. Support to use these data as fact of a result based system for quality od learning management and monitoring Selection of the students 263 students assessed Availability of a professional development n/a scheme Nomination of a team in charge of the exams The team was nominated Standardized exams The exams for grade 9 and 12 were produced produced (2011) Distribution and administration of exams The exams were implemented (2011) produced developed Data management procedures developed n/a Results diffusion path n/a - The project supported the implementation of the national exams held in 2011 for grades 9 and 12. Because the National Exam Centre established by the MoE did not functioned appropriately and the political instability this result was achieved partially, beyond UNESCO control Phase I 6 - Development of local capacity and of management Development of management practices in training area and implementation of a monitoring education quality process Application of learning in MoE s activities n/a n/a - Not available data to assess these aspects 26

28 Enables links and information diffusion between the different actors of the project n/a Table 3 - Project expected results and the achievements phase II Phase Expected result Outputs Progress Observations Phase II 1 - The Ministry of Education is able to design its strategy on the teaching profession development while aligning towards achieving the MDGs A holistic diagnosis study on teacher issue is carried out according to the methodological guide TTISSA n/a This result was not carried out due to an internal decision approved by the Ministry of Education and by the donor Training Modules for skills training of in service teacher for Grade 1 to 6 improved - The 5 training modules and guides are available and revision is under way concerning the competency based approach issues, linguistic accuracy and scientific content accuracy - 91 inspectors and inspectors/trainers (November 2014) Phase II 2- Improvement of qualifications of primary and secondary teachers (in-service and pre-service training) In-service teachers skills Grade 1 to 6 improved - 55 inspectors and inspectors/trainers (February 2015) - In October 2014, 176 heads of classes and president of study commissions from grades 1 to 6 was also trained; - In July/August 2015, 298 teachers received inservice teacher training on the revised modules Reference Model for bilingual education in Creole and Portuguese developed - Reference model for bilingual education produced 27

29 - 5 plus 1 schools were selected for the pilot phase; - 10 teachers from the schools were trained; - The teachers were visited 4 times by the trainers (from May to June 2015). Phase II Phase II 3 - First exam session realized and results are available for all the candidates 4 - Capacities and resource management by the Ministry of Education improved Pre-service teacher training programs in Guinea Bissau improved Capacity of decentralized staff to manage this exam session built Material support provided to the decentralized structures and to the Secrétariat National des Examens nationaux (SNE) Capacity of the Ministry of Education strengthened in support to the implementation of the Education Sectoral Plan. - The validation of the pre service teacher training programs is still under way. The project team is waiting for the appointment of a new government to continue an adequate appropriation of this output n/a n/a - The capacities of the INDE staff were reinforced in the development of training modules, in the organization of in-service teacher training, and on bilingual education. - The regional inspectors and inspectors/trainers capacities were reinforced in the competency based approach. This result was not carried out due the National Exam Centre established by the MoE is not functioning Phase II 5 - Project implemented with quality and in a timely and costeffective manner The project is monitored and evaluated to ensure quality of implementation and achievements - The project was monitored combining progress reports, field visits and distance monitoring by project team in Dakar and permanent monitoring by national project officer. - There are available monthly reports, annual progress reports and phase I evaluation reports A brochure with the project main achievements, lesson learnt and recommendations are being prepared to be published by the end of the project. 28

30 7.2 People interviewed Mrs. Akemi Yonemura - UNESCO Office in Dakar project team (project supervisor) Mrs. Catherine Collin - UNESCO Office in Dakar project team (project consultant as project manager) Mrs. Marise Évora Sagna - UNESCO Office in Dakar project (project consultant for technical backstopping); Mrs. Marcelina Santos Ba - Project national coordinator (phase II); Mrs. Cristina Brugiolo - UNICEF Guinea-Bissau chief of education; Mr. Amadeu Badinca - National expert (mathematics); Mr. Domingos Mandau - National expert (sciences) Mr. Mateus Ialá - National expert (didactic/pedagogy) Mr. Fernando Demba Baldé - National expert (expressions) Mr. Alexandre Furtado - National consultant (technical assistant); Pe. Luigi Scantamburlo - Bilingual Education consultant; Mr. Fábio Sousa - Portuguese Cooperation Country Office Coordinator; Mr. Mamadu Saliu Djassi - MoE DGPASE director. 7.3 Reviewed documents Table 4 - Reviewed documents Year Subject Author Title n/a Project document phase I n/a n/a Project document phase II n/a n/a Project document phase II revised n/a n/a Project document Marise Sagna 1994 PhD Thesis Carol Benson 2009 Report Bártolo Campos; Alexandre Furtado. Phase 1 Project Document-Jan Dec 2010 Phase 2 Project Document USD Phase 2 Project Document_revised_ Le tableau comparatif des modules élaborés, harmonisés et revus par l UNESCO projet Amélioration de la qualité de l enseignement et par la BAD Education III Teaching beginning literacy in the mother tongue : A study of the experimental Kiriol/Portuguese primary project in Guinea-Bissau. Política Docente na Guiné-Bissau. Banco Mundial MoE document MoE Despacho n.º CE/OM/ Report Ana Benavente 1er Rapport De Mission - Du au

31 2009 Report Pierre Varly Rapport De Mission Du 21 Novembre Au 12 Décembre 2009 Guinée-Bissau - Evaluation des compétences des enseignants Unesco Breda 2010 Phase I mid-term evaluation report Patrick Nkengne Nkengne Rapport Evaluation Mi-Parcours Projet UNESCO Guinee-Bissau v4_version finale 2010 Progress report Jean-Mathieu Laroche 2010 Report MENCCJD 2010 Journal article Carol Benson 2010 Journal article Carol Benson 2010 Report 2010 Report 2010 Report Ana Benavente Pierre Varly Maria antónia Barreto Alzira Saraiva José Brites Ferreira Maria antónia Barreto Alzira Saraiva José Brites Ferreira 2010 Report Bártolo Campos 2011 Report Max Butlen Christian Martinez 2010 Report Ana Benavente Progress reports for Italy_mars2010_final Three-year Plan for the Development of Education: How multilingual African contexts are pushing educational research and practice in new directions. Language and Education, v. 24, n. 4, p , 2010/06/ ISSN Real and Potential Benefits of Bilingual Programmes in Developing Countries. Balanço de competências de docentes em exercício na Guiné Bissau Apoio à elaboração do novo currículo de formação inicial de professores do ensino básico (ensino primário e secundário geral) na Guiné-Bissau. Volume I Apoio à elaboração do novo currículo de formação inicial de professores do ensino básico (ensino primário e secundário geral) na Guiné-Bissau. Volume II. Avaliação da Formação Inicial de Professores do Ensino Secundário da Guine -Bissau Vers une politique et une stratégie nationales pour la formation continue des professeurs de l enseignement fondamental en Guinée-Bissau. Rapport de mission en Guinée Bissau pour l Unesco du 11 avril au 1er mai Rapport sur la formation des opérateurs de terrain 30

32 et la passation des tests et questionnaires 2010 Report Pierre Varly Rapport de mission pour l UNESCO en Guinée Bissau du 11 avril au 1er mai Appui aux opérations de terrain/ Mise en place de la saisie 2010 Report Glória Ramalho Avaliação dos Alunos do 3ºAno da Escola de Formação de Professores 17 de Fevereiro 2010 Report Glória Ramalho Avaliação da Formação Inicial de Professores do Ensino Secundário da Guiné-Bissau - Volume II. Avaliação dos Alunos-Professores 2010 Report Bártolo Campos Avaliação da Formação Inicial de Professores do Ensino Secundário da Guiné-Bissau - Volume I. Avaliação Institucional e Pedagógica 2011 Phase I evaluation report François Robert, BISSAUPROJETQ-EVALF Progress report Jean-Mathieu Laroche Progress reports for Italy_mars Progress report Jean-Mathieu Laroche Progress reports for Italy_mars Progress report Akemi Yonemura 534GBS1000 report 2013_approved by donors 2013 Report MENCCJD Guiné-Bissau - Relatório da Situação do Sistema Educativo. Margens de manobra para o desenvolvimento do sistema educativo numa perspetiva de universalização do Ensino Básico e de redução da pobreza National Consultant report Alexandre Furtado Actions développées par le consultant entre 1 à 30 Novembre Consultant report Maria José Nóvoa 2014 Progress report Akemi Yonemura, 2015 Progress report Akemi Yonemura modules and 5 guides National experts 2015 Bilingual education materials Luigi Scantamburlo BREVE RELATÓRIO FINAL 1ªETAPA Guinea-Bissau Progress Reports for Italy Amendment FNL_2014_Oct_1 Project Progress Report Guinea- Bissau Teacher Training Dakar Jan_2014_Apr_2015_ _final Didactic/pedagogy, Portuguese, Expression, Integrated Science, Mathematics Curso para os professores

33 Report on pre-service teacher training Curricular plans and revised programmes Marise Sagna, National experts UNESCO consultant review recommendations_msagna_ Planos curriculares e programas para formação inicial de professores do 1º-3º CICLOS INDE INDE 2015 Individual Consultant Report Marise Sagna, 2015 National experts report 2015 National experts report Amadeu Badinca Domingos Gomes Domingos Mandau Mateus Ialá Fernando Demba Baldé Amadeu Badinca Domingos Gomes Domingos Mandau Mateus Ialá Fernando Demba Baldé Implementação da experiência piloto do ensino bilingue Activity Report n 05/2015 Period: august 2015 Seminário de formação de formadores em módulos e guias elaborados em pedagogia de integração APC Relatório das atividades dos experts dos meses de maio/ junho de Mission report Marise Sagna Mission Report Project document Xinxin Yu Data Analysis Report 2015 National Consultant report 2015 National consultant report 2015 National consultant report 2015 National consultant report Alexandre Furtado Luigi Scantamburlo e Grupo de Trabalho Alexandre Furtado Luigi Scantamburlo e Grupo de Trabalho 2015 Logistics Evaluation Report Marise Sagna Principales actions développées pendant la période du 1 Janvier au 10 Février 2015 Relatório Final (Bilingual Education) Planos curriculares e programas para formação Inicial de professores do ensino básico Relatório Final - Projeto Piloto para o Ensino Bilingue Crioulo Guineense - Português In-service Teacher Training on Competence Based Approach- Bissau, Bula, Buba, Gabu 20th July to 01st August,

34 7.4 References 1. MENCCJD, Pedido Financeiro ao Fundo da Parceria Global para a Educação. 2011, Ministério da Educação Nacional, Ciência, Juventude e Desportos: Bissau. 2. Ministério da Economia Plano e Integração Regional, Segundo Documento de Estratégia Nacional de Redução da Pobreza. 2011, Ministério da Economia, Plano e Integração Regional,: Bissau. 3. MENCCJD, Eléments de Diagnostic du Système Educatif Bissau-guinéen: Marges de manœuvre pour le développement du système éducatif dans une perspective d universalisation de l enseignement de base et de réduction de la pauvreté. 2009, República da Guiné-Bissau - Ministério da Educação Nacional: Bissau. 4. UNESCO Institute of Statistics. UIS statistics in brief. General profile - Guinea-Bissau [cited 2013 december 30]; Available from: e=eng&br_country=6240&br_region= Monteiro, J., A Educação na Guiné Bissau: Bases para uma estratégia sectorial renovada. 2005, República da Guiné-Bissau - Ministério da Educação Nacional: Bissau. 6. Observatório da Língua Portuguesa. Falantes de Português Língua Materna [cited Dezembro]; Available from: 7. Benson, C., How multilingual African contexts are pushing educational research and practice in new directions. Language and Education, (4): p O Regan, D. and P. Thompson Advancing Stability and Reconciliation in Guinea-Bissau: Lessons from Africa s First Narco-State. ACSS Special Report No. 2, OCDE Resource Flows To Fragile And Conflict-Affected States Annual Report Banco Mundial, Global Monitoring Report Millennium Development Goals: Confronting the Challenges of Gender Equality and Fragile States. 2007, Banco Mundial: Washington. p Sousa, J.S., Guiné-Bissau: A destruição de um país. Desafios e reflexões para uma nova estratégia nacional. 2012, Coimbra: Author. 12. INEE, Minimum Standards for Education: Preparedness, Response, Recovery 2010, New York: INEE. 13. Brannelly, L., S. Ndaruhutse, and C. Rigaud, Donors Engagement: supporting education in fragile and conflict-affected states. 2009, Paris: UNESCO-IIEP,CfBT Education Trust. 33