Evaluation of MCA Namibia s Livestock Marketing Efficient Fund (LMEF)

Transcription

1 Evaluation Design Report MCA-N Contract: MCA/COM/RCQ/5E01001 Evaluation of MCA Namibia s Livestock Marketing Efficient Fund (LMEF) 14 March 2012 Commissioned by the Millennium Challenge Account Namibia with funding from the Millennium Challenge Corporation

2 Preface/acknowledgements First of all I would like to thank MCA-N for entrusting me with the interesting and challenging task of evaluating the Livestock Market Efficiency Fund (LMEF). I look forward to a sustained positive cooperation in the coming years. Secondly I would like to thank MCA-N staff and representatives of the LMEF grantees for the assistance provided during my first mission to Namibia and preparation of the evaluation design. Finally I would like to point out that, although I am employed by Consultants for Development Programmes (CDP) in the Netherlands, I was selected and contracted by MCA- N in a personal capacity and the opinions in this report are my own, and not necessarily those of CDP or MCA-N. Paul Sijssens Consultants for Development Programmes Achter Clarenburg JH Utrecht The Netherlands LMEF Evaluation Design Final Report. March 2012 i

3 Contents Preface/acknowledgements... i List of acronyms... iii Executive summary... iv 1. Introduction Objective of the evaluation design and planning Process of the evaluation design and planning Summary information on the projects LMEF general Grant 2010/01: Unleashing the potential of livestock sector in Oshana and Ohangwena Regions Grant 2010/02: Development of export opportunities for beef products from the Caprivi Region Grant 2010/03: Sero-epidemiological and Parasite Survey in the Northern Communal Areas in Namibia Grant 2010/04: Baseline Survey of Animal Nutrition in the Northern Communal Areas of Namibia Evaluation methodology and approach Objectives of the evaluation Characteristics of Real Time Evaluation Main criteria for evaluation Research questions, data requirements and sources of information Tools for data collection Risk assessment for evaluation Evaluation work plan Quarterly evaluation missions Final evaluation of grant cycles Final comprehensive evaluation Annex 1 Terms of Reference Annex 2 Itinerary Annex 3 Documents consulted Annex 4 Summary sheets on the grant projects Annex 5 Field visit reports Annex 6 Proposed alternative payment schedule LMEF Evaluation Design Final Report. March 2012 ii

4 List of acronyms AGRA-PSD CBPP CBRLM CBT CDP CVL DAC DEES DVS FGD FMD HACCP KII LMEF LPF MAWF MBN MCA-N MCC MEATCO NCAs OECD QPR RTE ToR AGRA Professional Services Division Contagious Bovine Pleuropneumonia Community Based Rangeand and Livestock Management Commodity-Based Trade Consultants for Development Programmes Central Veterinary Laboratory Development Assistance Committee Directorate of Engineering and Extension Services Directorate of Veterinary Services Focus Group Discussions Foot and Mouth Disease Hazard Analysis Critical Control Point Key Informant Interviews Livestock Market Efficiency Fund Livestock Producer Forum Ministry of Agriculture, Water and Forestry Meat Board of Namibia Millennium Challenge Account Namibia Millennium Challenge Corporation Meat Corporation of Namibia Northern Communal Areas Organisation of Economic Co-operation and Development Quarterly Progress Report Real Time Evaluation Terms of Reference LMEF Evaluation Design Final Report. March 2012 iii

5 Executive summary This report presents the design for the evaluation of the Livestock Market Efficiency Fund (LMEF) in response to the Terms of Reference (Annex 1) issued by Millennium Challenge Account Namibia (MCA-N). The evaluation design and planning took place from Monday 6 February to Friday 2 March 2012 and consisted of document review, familiarisation field visits and the actual design of the evaluation. The objective of the evaluation is to assess the impact of the LMEF on the intended beneficiaries mainly in terms of income or other quantifiable benefits. The ToR indicates that the evaluation is to take place while the project is being implemented and that it must consist of a series of evaluation missions, observing the project activities as they are being implemented in the field. Although not explicitly called as such in the ToR, it means that de facto the assignment will have to take the form of a Real Time Evaluation (RTE). The evaluation methodology will follow the assessment of the five main criteria defined by the Organisation of Economic Co-operation and Development/Development Assistance Committee (OECD/DAC): relevance, efficiency, effectiveness, impact and sustainability. For each of these criteria, research questions were defined. Some of these questions were taken from the ToR, others were added to allow sufficient analysis of the evaluation criteria. Subsequently, data requirements and sources of information were indicated, separately for the LMEF grant projects and for the LMEF as a whole. Tools for data collection were defined and a provisional work plan was drawn up. It contains three main elements: (i) quarterly evaluation missions, (ii) the final evaluation of the grant cycles and (iii) the final comprehensive evaluation of the LMEF. LMEF Evaluation Design Final Report. March 2012 iv

6 1. Introduction 1.1 Objective of the evaluation design and planning According to the Terms of Reference (ToR) for the evaluation of MCA Namibia s Livestock Market Efficiency Fund (LMEF), the first phase consists of evaluation design and planning. The consultant was to review the LMEF Grant Manual and all relevant grant documents and available literature and data to determine the methodology needed to carry out the evaluation, including any additional data needs aside from that which can be gathered from grantees and which can be obtained through Key Informant Interviews (KIIs) and Focus Group Discussions (FGDs), and prepare a full evaluation design report. The ToR for the evaluation is attached as Annex Process of the evaluation design and planning The evaluation design and planning was done during the consultant s first mission to MCA-N in the context of the LMEF evaluation, which took place from Monday 6 February to Friday 2 March On Tuesday 7 February a briefing meeting was held with MCA-N staff and representatives of the four LMEF 1 st cycle grantees. A first batch of background documents was handed to the consultant and the following days were spent on review of the documentation. Regular meetings were held with the contract manager, Mr Indongo Indongo. A first field visit to the Northern Communal Areas (NCAs) took place from February. In Enghandja Village, Oshikango Constituency, a training on emergency procedures and animal nutrition, organised by Komeho, was attended. A second mission to the NCAs took place from February. During this visit both grant projects implemented by AGRA Professional Services Division (AGRA-PSD) were visited 1. Due to the closure of the abattoir in Caprivi no visit was made to a project site of the Meat Board of Namibia (MBN), but a meeting was held with the project coordinator in Windhoek and a DVD was obtained from MBN that provides the purpose of the project. Short visit reports are attached as Annex 5. At the end of the mission a debriefing meeting was held at the MCA-N office in Windhoek on Friday 2 March. A detailed itinerary and list of people met is given in Annex 2. An list of consulted documents is given in Annex 3. 1 But one of the project sites couldn t be reached due to the road condition. See Annex 5.3 LMEF Evaluation Design Final Report. March

7 2. Summary information on the projects In this section short descriptions of the LMEF grant projects are given. Summary sheets on each of the grant projects are attached in Annex LMEF general The overall aim of MCA-N s Agriculture Project is to reduce rural poverty through investments that achieve a sustainable increase in the economic performance of the agricultural sector. It comprises three main activities: (i) livestock, (ii) land access and management, and (iii) indigenous natural products. The livestock activity, to which the LMEF belongs, seeks to bring the marketing opportunities for farmers who live north of the Veterinary Cordon Fence closer to the opportunities enjoyed by farmers south of the Fence. The LMEF will support demand-driven actions aimed at improving the performance of the livestock sector in the NCAs. The objective of the LMEF is to increase incomes to livestock producers and ensure the continued growth of the livestock sector. It is expected that this objective will be achieved through improving livestock marketing, eliminating barriers to existing and new markets and improving the marketability of livestock for broad application to the livestock industry in the NCAs to ensure the competitiveness and sustainability of the livestock sector Grant 2010/01: Unleashing the potential of livestock sector in Oshana and Ohangwena Regions The project aims to create 12 case study NCA village communities in Ohangwena and Oshana Regions which show how NCA farmers can increase the income that they earn from their livestock by adopting improved management of their livestock and so improving the quality at point of sale. This will be done by delivering training on, and support to establish, improved livestock farming practices, infrastructure management and other key actions. A network of expertise will be established in these communities through community mobilisers, who will offer encouragement and support in their own communities as well as being able to offer their expertise to other livestock farmers. This project will then be promoted to encourage other communal livestock farmers to follow their example. The project is implemented by Komeho Namibia Development Agency and has a duration from 1 April 2011 until 30 April Grant 2010/02: Development of export opportunities for beef products from the Caprivi Region For the Caprivi region, where Foot and Mouth Diseases (FMD) eradication is not realistic, an alternative approach is needed in order not to exclude cattle farmers from future market access. The approach which will be tested and evaluated in this project is an integration of the Hazard Analysis Critical Control Point (HACCP) approach and the Commodity-Based Trade (CBT) concept to achieve effective food safety and animal disease risk management. In order to address not only the safety but also the quality of the product, the project will involve cattle producers who commit to 2 It should be noted that the achievements of these objectives relies to a great extent on the support by MCA- N under traceability and other such broader developments in Namibia s livestock sector LMEF Evaluation Design Final Report. March

8 improved animal management practices and the delivery of quality animals to a local export abattoir. Phase 1 of the project is mainly consisting of theoretical research work which needs to be done to develop detailed value chain descriptions, market analysis and protocols relating to HACCP procedures, production protocols, etc. Another important component is the FMD virus testing done in an internationally accredited FMD. The 50 farmers who are being mentored own a total of 6,000 7,000 cattle and are distributed throughout all the constituencies of Caprivi. The project will eventually influence 12,000 households with 156,000 cattle across the entire NCAs. The project is implemented by Meat Board of Namibia and originally had a duration from 15 March 2011 until 31 December 2011 (phase 1). The project timeframe has in the mean time been extended. 2.4 Grant 2010/03: Sero-epidemiological and Parasite Survey in the Northern Communal Areas in Namibia The project concerns the collection of baseline epidemiological data and the development of investigation protocols and standard operating procedures for DVS staff for the investigation and diagnosis of abortion problems in goats and infectious fertility problems in cattle. These can then be utilised at a later stage to control infectious diseases and parasite infection which will lead to improved fertility and production, which in turn should lead to increased income and reduced overstocking and rangeland degradation, since smaller numbers of livestock can be kept. The data will be made available to the Directorate of Veterinary Services (DVS) for the development and implementation of control and eradication programmes for specific infectious diseases and to advise farmers on the appropriate use of anthelmintics (internal parasite remedies), as well as develop and implement human hygiene programmes in co-operation with the relevant health authorities (toilets and regular de-worming) to eliminate human tapeworm infestation and concurrent bovine and porcine cysticercosis. The research data will automatically become part of the epidemiological database of DVS. The project is implemented by AGRA-PSD and has a duration from 1 May 2011 until 30 April Grant 2010/04: Baseline Survey of Animal Nutrition in the Northern Communal Areas of Namibia The project attempts to identify and alleviate nutritional problems of livestock animals free-ranging in the NCAs of Namibia. It will supplement the Community-Based Rangeland and Livestock Management Project of MCA-N, which attempts to increase the quantity of grazing on offer by improved grazing management. Namibian experience has shown that once the quantity of grazing is satisfactory, the quality thereof (i.e., the nutritional value of the grassland) becomes the next factor limiting animal production. Thus, the baseline survey samples soil, drinking water, forage plants and livestock animals directly to determine their nutrient status and identify nutritional deficiencies, excesses and other problems (Objective 1). These will then be addressed in a lick supplementation pilot trial (Objective 2), before the message will be passed on to the primary producer in the NCA in an extensive outreach effort (Objective 3). The project is implemented by AGRA and has a duration from 1 April 2011 until 30 April LMEF Evaluation Design Final Report. March

9 3. Evaluation methodology and approach 3.1 Objectives of the evaluation According to the ToR (provided in Annex 1) the objective of the evaluation is to assess the impact of the LMEF on the intended beneficiaries mainly in terms of income or other quantifiable benefits. The evaluation should also look at the impact of LMEF in terms of: reducing costs and losses associated with marketing livestock in the NCAs; alleviating other challenges to successful commercial marketing of livestock that are present in the current supply chain beyond the farm gate due to the lack of disease free status; and identifying and eliminating barriers to increasing volume of livestock and livestock products sold into existing markets and accessing additional markets destinations. Finally, the evaluation should look at whether the LMEF as a whole has achieved its stated objectives. A preliminary set of research questions was given (included in Section 3.4 below) in the ToR to achieve the objectives. The ToR indicates that the evaluation is to take place while the project is being implemented and that it must consist of a series of evaluation missions, observing the project activities as they are developing in the field. Although not explicitly called as such in the ToR, it means that de facto the assignment will have to take the form of a Real Time Evaluation (RTE). 3.2 Characteristics of Real Time Evaluation The easiest definition of RTE is an evaluation that is carried out while the project is in full implementation and feeds back its findings to the project for immediate use. A RTE, with its recurrent field missions and regular feedback to the project, has the following characteristics, which have to be considered for the design of the evaluation: RTE refers to a series of iterative evaluations; RTE is interactive: the evaluator discusses findings with the project implementers and takes account of their views in the evaluation reports; RTE can be used to check compliance with broader issues such as relevant donor and government policies; RTE brings in an external perspective, analytical capacity and knowledge. The evaluator acts as the stranger who sees more because of his distance from day-to-day activities. RTE is different from monitoring as it asks not only if project implementation is according to plans, but also whether the plans are appropriate and in line with donor and/or national policies. 3.3 Main criteria for evaluation In the consultant s approved technical and financial proposal for the evaluation of LMEF, it was proposed to include in the evaluation methodology the assessment of the five main criteria defined LMEF Evaluation Design Final Report. March

10 by the Organisation of Economic Co-operation and Development/ Development Assistance Committee (OECD/DAC): relevance, efficiency, effectiveness, impact and sustainability. The criteria are defined as follows: Relevance: the extent to which the project is suited to the priorities and policies of the donor, the appropriateness of the project to the problems, needs and priorities of its target groups/beneficiaries, and the quality of the design through which the objectives are to be reached. Efficiency: this measures the outputs, qualitative and quantitative, in relation to the inputs. Effectiveness: the contribution made by the project s results/outcomes to the achievement of the project purpose. Impact: the effect of the project on its wider environment (change). This involves effects resulting from the project on the local social, economic, environmental and other development indicators. Note that these effects can be both intended and unintended, as well as positive and negative. Sustainability: the likelihood of a continuation of benefits produced by the project after the period of external support has ended. For each of these criteria specific questions can be defined, for each of the grant projects as well as for the LMEF as a whole. These questions are elaborated in the next section. 3.4 Research questions, data requirements and sources of information Research questions are compiled from three sources: (i) the Terms of Reference for the evaluation of the LMEF (which in turn come from M&E Plan for the MCA-N Compact), (ii) the M&E plans of the different grant projects, and (iii) from the OECD/DAC evaluation criteria given above. In the following tables, research questions are defined for each evaluation criterion, specified for grant projects and for the LMEF as a whole. For each research question the respective data requirements and sources of information are given. Table 3.1 Research questions, data requirements and sources of information for LMEF grant projects Research questions Information needed Source of information Relevance Are the project objectives consistent with, and supportive, of LMEF objectives? Are the projects consistent with the national goals or objectives of government and MAWF? Analysis of project objectives Project proposal LMEF Grant Manual Does the project respond to the needs of the target groups? Is the intervention logic clear and logical? Are the risks and assumptions holding true? Are risk management arrangements in place? How were key stakeholders involved in the design process? Problem analysis Project design Project design Risk analysis Project design Project design process LMEF Evaluation Design Final Report. March Project proposal Progress reports Observations and interviews Project proposal Project proposal Progress reports Observations and interviews Project proposal Stakeholder interviews

11 Efficiency Are activities implemented as scheduled? If there are delays how can they be rectified? Are a work plan and resource schedule available and used by the project management? Are inputs provided/available at planned cost (or lower than planned)? How well are activities monitored by the project and are corrective measures taken if required? Effectiveness Have the planned results to date been achieved? What are the major factors influencing the achievement or non-achievement of the objectives? What is the quality of outputs? What is the likelihood of the project objectives to be achieved? Impact What has happened as a result of the project or what is likely to happen? What real difference has the project made to the beneficiaries? How many people have been affected? Are any external factors likely to jeopardise the project s direct impact? Sustainability Are the services/results affordable for the target groups at the completion of project? How far is the project embedded in local structures? To what extent are relevant target groups actively involved in decisionmaking concerning project orientation and implementation? What support has been provided by the relevant national or local government? Work plan Implementation reports Project financial data M&E plan M&E reports Implementation reports M&E reports Project design Implementation reports M&E reports Implementation reports M&E reports M&E reports Project design Project design Project design MAWF Strategic Plan Vision 2030 or NDP3 or 4? Progress reports Project staff interviews Observations Progress reports MCA-N finance officer Project proposal Progress reports Observations and interviews Progress reports Observations and interviews Observations and interviews Surveys Progress reports Observations and interviews Progress reports Observations and interviews Observations and interviews Surveys Progress reports Observations and interviews Surveys Progress reports Observations and interviews Stakeholder interviews Project proposal Progress reports Observations and interviews Project proposal Progress reports Observations and interviews Project proposal Progress reports Observations and interviews LMEF Evaluation Design Final Report. March

12 Table 3.2 Research questions, data requirements and sources of information for LMEF as a whole 3 Research questions Information needed Source of information Relevance Are the LMEF objectives consistent with, and supportive, of the MCA-N objectives? LMEF design LMEF grant manual MCA-N compact Does the LMEF respond to the needs of the target groups? LMEF design LMEF grant manual Grant projects Are the risks and assumptions holding true? Are risk management arrangements in place? Efficiency Are all contractual procedures clearly understood and do they facilitate the implementation of the project? Is a work plan and resource schedule available and used by the project management? Are funds committed and spent in line with the implementation timescale? How well are activities monitored by the project and are corrective measures taken if required? Effectiveness Does the LMEF contribute to disease free status for the NCAs? If so, how and to what extent? To what extent has the LMEF as a whole achieved its stated objectives? Impact To what extent does the LMEF contribute to increased incomes as well as other measures of livelihood among beneficiaries? Does the LMEF contribute towards the identification and elimination of existing marketing barriers and other challenges to successful commercial marketing of livestock in the NCAs? If so, what is the impact (e.g., increased volume of livestock and livestock products sold into existing markets in the NCAs and in existing and new market destinations)? Sustainability Has the application of the LMEF led to any multiplier effects in terms of replication of grantee projects, extension of project outcomes, and dissemination of information? Risk analysis LMEF design LMEF design Grant contracts Implementation report Work plan Implementation reports Project financial data M&E plan M&E reports LMEF design Implementation report Implementation report Implementation reports M&E reports Implementation reports M&E reports Implementation reports M&E reports Observations and interviews LMEF grant manual Progress reports Observations and interviews Progress reports Observations and interviews Progress reports Project staff interviews Observations Progress reports MCA-N finance officer Project proposals Progress reports Observations and interviews LMEF grant manual Grant projects Progress reports Observations and interviews Grant projects Progress reports Observations and interviews Progress reports Observations and interviews Surveys Progress reports Observations and interviews Surveys Progress reports Observations and interviews 3 This table is not yet final, because the Agri team is still allocating the LMEF grants to the traceability and the bulls scheme LMEF Evaluation Design Final Report. March

13 3.5 Tools for data collection Document review Official project documents, and especially Quarterly Progress Reports (QPRs), are a primary source of information for the evaluation. The QPRs provide information on progress of implementation, adherence to or deviation from the work plan, risk analysis, and M&E information. Field observations and beneficiary consultation Observation plays a particularly important role in RTE. The evaluator may gather information directly from observation that would not be available indirectly through key informants. Observation is also a good means of triangulating findings from progress reports. During field work the consultant will meet with project beneficiaries and get their views. It is essential that the consultant engages in beneficiary consultation, as this is an important added value of RTE. Consultation can take various forms: key informant interviews, general meetings or focus group interviews. Key informant interviews While real-time evaluations should make extensive use of observation, key informant interviews (KIIs) are likely to be an important additional source of information. Most key informant interviews will take the form of semi-structured interviews. The recurrent field missions of RTE offer the opportunity to interview people more than once, to capture any learning that takes place as the project is progressing. Focus group discussions Focus groups can provide the evaluator with qualitative information on a range of issues. Ideally it involves five to twelve people in a discussion of their experiences and opinions about a topic. Surveys Surveys are generally not practicable as part of real-time evaluations as they take too long to prepare, conduct and process. However, it is expected that it will be possible to make use of surveys by MCA-N and LMEF grantees, which are done as part of regular monitoring. 3.6 Risk assessment for evaluation A risk emerging from the ToR is the emphasis the impact of the LMEF on the intended beneficiaries, mainly in terms of income or other quantifiable benefits. A first preliminary assessment of the four approved and ongoing grant projects indicate that it will be quite difficult, if not impossible, to measure the impact of these projects. The two projects implemented by AGRA-PSD are research projects. They are collecting data on diseases and nutrition and build capacities of local institutions. Whether these activities will have impact on livestock farmers will depend on the extent to which the governmental services will apply the learnt techniques and make use of the collected data. Supposing this will indeed be the case, it will only have a measurable impact on livestock farmers incomes after the end of the Compact. The project implemented by MBN not only has been LMEF Evaluation Design Final Report. March

14 suspended due to FMD outbreak in Caprivi, it also has an important political dimension. If the proposed approach will be accepted in Namibia, and more importantly by meat importing countries, it will have measurable impact. It is doubtful, however, whether that will be the case during the lifetime of the project. The training project implemented by Komeho is the only project of the current four where measurable impact may occur. If farmers adopt (some of) the extension messages it may improve their cattle and thereby their wealth and income. Again, this will take time and measuring the increased income will be difficult. However, Komeho has included measuring household income in its M&E plan and will also monitor the adoption rate of extension messages. To mitigate this risk concerning the focus on impact, the evaluation design takes a wider look at the project. It will not single out impact, but will look at impact as one of the five OECD/DAC evaluation criteria. The characteristics of Real Time Evaluation also ensure that the evaluation takes an appropriate form, where findings feed back to the implementers for immediate use. In addition an estimation of expected impact after the end of the Compact can be made. One risk related to Real Time Evaluation is the dependency on the stakeholders that are being evaluated. The consultant not only needs to rely on the data provided by the implementers, but also will need their assistance to engage with project beneficiaries and other stakeholders, like extension staff or local authorities. This risk will be mitigated by establishing a good working relationships with the implementers (MCA- N and LMEF grantees) and maintaining regular and open communication. It is expected that the implementers will be aware of the participatory and iterative nature of the evaluation process and that open and realistic exchange of information is the best way for all to collaborate in it. LMEF Evaluation Design Final Report. March

15 4 Evaluation work plan The LMEF evaluation approach can be summarised as an iterative process of regular evaluation missions, following, assessing and documenting implementation progress based on document review and triangulation with field observations and interviews. It contains three main elements, after the current design phase: (i) quarterly evaluation missions, (ii) the final evaluation of the grant cycles and (iii) the final comprehensive evaluation of the LMEF. Table 4.1 gives the proposed work plan for the evaluation. The different phases are elaborated in the following sub-sections. The work plan, which deviates from the work plan in the ToR, calls for a revision of the payment schedule of the evaluation assignment. An alternative payment schedule is attached as Annex Quarterly evaluation missions The largest part of the work plan consists of quarterly evaluation missions, starting in May 2012 and ending in March These quarterly missions are in line with the ToR for the LMEF evaluation, with one adaptation. The ToR mentions eight quarterly reports for at least three LMEF grant cycles. Since the grant cycles to a large extent will be running concurrently, and are all likely to end more or less at the same time, it is not feasible nor practical to have separate reporting systems for the different grant cycles. The work plan therefore consists of quarterly missions and report, during which each on-going project, regardless of its grant cycle, will be evaluated. In the course of the evaluation process the different evaluation questions as listed in Section 3.4 above will be answered. For obvious reasons questions regarding impact (and sustainability?) can only be answered towards the end of the evaluation, while questions on relevance can be answered in an early stage. Other criteria, such as effectiveness and efficiency will gradually become more prominent as the evaluation progresses. The evaluation progress reports are therefore growing documents, which will be enriched by each evaluation mission. It is also expected to have a learning function for the parties under evaluation. It is the intention of the evaluator to make this a participatory and shared process, where responses and opinions of the evaluated organisations are integrated in the reports. For each quarterly mission the following common observations are made, aiming for the missions to be as effective and efficient as possible: emphasis will be on field observations and meetings/interviews in the grant project areas; to allow for effective preparation, MCA-N and/or the LMEF grantees are requested to share progress reports and other relevant documentation with the consultant prior to the start of the field mission; the consultant will communicate his travel dates timely to the LMEF grantees. The grantees in turn are requested to indicate possibilities for field work in the indicated period of time to allow for planning of field work before the start of the mission; each mission will start with a short meeting at MCA-N in Windhoek and end with a debriefing of MCA-N and LMEF grantees; LMEF Evaluation Design Final Report. March

16 a draft quarterly evaluation report will be shared with MCA-N and LMEF grantees for comments. 4.2 Final evaluation of grant cycles The last quarterly evaluation mission is scheduled to take place when the grant projects have come to an end. This is provisionally 4 scheduled for April/June This mission will lead to a final report of the evaluation of each grant cycle (or combined; this is to be decided). In line with the ToR the draft final report will be followed by a stakeholders workshop, a final report and production of information materials. 4.3 Final comprehensive evaluation The last stage of the evaluation is the production of a final comprehensive report for the evaluation of the LMEF. In line with the ToR, the draft comprehensive evaluation report will be produced by 21 July It will be followed by a stakeholders workshop, a final report and production of information materials. All deliverables are then expected to have been produced by 1 September 2014, which will bring the LMEF evaluation to an end. 4 Since the second and third grant cycles have not been awarded yet, there is no final date for the completion of these cycles. However, given the end date of the MCA-N compact it is expected to be around April 2014 LMEF Evaluation Design Final Report. March

17 Table 4.1 Proposed work plan for LMEF evaluation Dates Activity Deliverable Due date Feb/Mar 2012 May/Jun 2012 Aug/Sep 2012 Nov/Dec 2012 Feb/Mar 2013 May/Jun 2013 Aug/Sep 2013 Nov/Dec 2013 Feb/Mar 2014 Apr-Jun 2014 Jul-Sep 2014 Evaluation design and planning 1 st quarterly evaluation mission 2 nd quarterly evaluation mission 3 rd quarterly evaluation mission 4 th quarterly evaluation mission 5 th quarterly evaluation mission 6 th quarterly evaluation mission 7 th quarterly evaluation mission 8 th quarterly evaluation mission Final evaluation grant cycles Final comprehensive evaluation Draft evaluation design report 5 March 2012 Comments by MCA-N 13 March 2012 Final evaluation design report 20 March 2012 Information materials 3 April 2012 Quarterly progress report 20 June 2012 Quarterly progress report 20 September 2012 Quarterly progress report 3 December 2012 Quarterly progress report 20 March 2013 Quarterly progress report 20 June 2013 Quarterly progress report 20 September 2013 Quarterly progress report 3 December 2013 Quarterly progress report 20 March 2014 Draft evaluation report 30 May 2014 Stakeholders workshop and report 30 June 2014 Final evaluation report 14 July 2014 Information materials 21 July 2014 Draft evaluation report 21 July 2014 Stakeholders workshop and report 11 August 2014 Final evaluation report 25 August 2014 Information materials 1 September 2014 LMEF Evaluation Design Final Report. March

18 Annex 1 Terms of Reference 1. Summary Terms of Reference Evaluation of MCA Namibia s Livestock Market Efficiency Fund (LMEF) The objective of this Terms of Reference (TOR) is to define the terms for the evaluation of Millennium Challenge Account Namibia s Livestock Market Efficiency Fund (LMEF or the Fund ). Using data collected by LMEF grantees, the Consultant, and other sources, the evaluation will assess how each of the grants have achieved their stated objectives, especially (where applicable) in terms of impact on the intended beneficiaries. The evaluation should also look at whether the LMEF as a whole has achieved its stated objectives, particularly in terms of its contribution to reducing costs and losses associated with marketing livestock in the Northern Communal Areas (NCAs); alleviating other challenges to successful commercial marketing of livestock that are present in the current supply chain beyond the farm gate due to the lack of disease-free status; and identifying and eliminating barriers to increasing volume of livestock and livestock products sold into existing markets and accessing additional markets destinations; and other relevant questions as determined based on the specific details of the grants. 2. Background Information 2.1 The Millennium Challenge Corporation (MCC) The Millennium Challenge Corporation (MCC) was established in January 2004 as a United States government corporation to implement the Millennium Challenge Account (MCA). MCC s mission is to reduce poverty by supporting sustainable, transformative economic growth in developing countries, which create and maintain sound policy environments. MCC is designed to support innovative strategies and to ensure accountability for measurable results. For additional information on MCC please visit Namibia s Millennium Challenge Compact The MCA Namibia (MCA-N) Compact, which provides grant funding for public investments in Education, Tourism and Agriculture, was signed on 28 July 2008 between the Republic of Namibia and the US Government, acting through the MCC. An amount of US$304.5 million will be available for development in the target sectors, over and above current Government allocations and assistance from other development partners. The Goal of Namibia s Millennium Challenge Compact is to reduce poverty through economic growth in the Education, Tourism and Agriculture sectors. To accomplish the Compact Goal of increasing income, the MCA-N Programme aims to achieve the following objectives: i) Increase the competence of the Namibian workforce (knowledge, skills and attitude); ii) Increase the productivity of agricultural and non-agricultural enterprises in rural areas. The Compact aims to improve the quality of education and training for the underserved populations, and attempts to capitalize on Namibia s comparative advantages to increase the incomes of the poor Namibians, predominantly in the northern areas of the country. LMEF Evaluation Design Final Report. March

19 In Education, the Programme seeks to bring the quality of the work force closer to the requirements of industry and the labour market at large. This is in line with the aspirations of Vision 2030 and the 3rd National Development Plan s strategic objective for productive and competitive human resource and institutions, whose values informed the programming, content and activities of the Education and Training Sector Improvement Programme (ETSIP). The MCA-N Education Project contributes to ETSIP. It aims to improve the competency and knowledge of young Namibians by supporting new and innovative methods of learning in addition to the more traditional approaches to education and improve physical infrastructure for learning and teaching in schools, regional study and resource centres and Community Based Skills Development Centres. The Tourism Project seeks to bring conservancies in high potential tourism areas into the mainstream of the tourism business, increasing the financial and in-kind benefits to rural conservancy members. Internationally-acknowledged best practice models for community-based tourism and community-based natural resource management underpin the Tourism Project. This is the first MCC project with an explicit focus on tourism, acknowledging the sector s tremendous poverty alleviating potential in rural areas of Namibia. The Etosha National Park will serve as a model for increased participation by the adjacent conservancies. MCA- N s Tourism Project will facilitate access to the Park for the conservancies around it through exclusive access concessions, to be awarded through the 2007 Concession Policy. The MCA-N Tourism Project will also enhance regional and international marketing of Namibia as a tourism destination. The Agriculture Project comprises three main activities in livestock, land access and management and indigenous natural products. The Livestock Activity seeks to bring the marketing opportunities for farmers who live north of the Veterinary Cordon Fence ( the Fence ) closer to the opportunities enjoyed by farmers south of the Fence. The focus will be on improving veterinary services aimed at obtaining animal disease-free status for the NCAs in order to gain access to international markets. The accompanying public investment in rangeland management will improve access to grazing and farming practices. The overall objective is to increase the farmers cash income derived from large stock and small stock farming in the NCAs. Through the Land Access and Management Activity the Communal Land Boards, traditional authorities and other key stakeholders will be empowered to better manage the available resources. Increasing direct participation of the primary producers in the processes of value addition to raw, natural products such as Marula, Kalahari Melon Seed, Hoodia, Devil s Claw and Ximenia is the key objective of the Indigenous Natural Products Activity. The value chain approach that the INP sector has adopted will ensure that the activities are responsive to the fast-changing market realities for natural products. At present, most of the revenue from selling products made of these valuable commodities is generated outside Namibia. The MCA-N Programme will facilitate an increased participation by the primary producers in the final processing, packaging and marketing of these products. Please visit for the detailed Project Descriptions. This TOR is related to the Tourism and Agriculture projects as well as to MCA-N s M&E Plan. 2.3 The MCA Namibia Management Structure MCA-N is the accountable entity, responsible for the overall management of the Compact s implementation. MCA-N is a legal entity within the National Planning Commission (Office of the President) comprised of a Board and a Programme Management Unit. MCA-N s Board provides LMEF Evaluation Design Final Report. March

20 oversight and strategic guidance and is ultimately responsible for the Programme s success, while the Programme Management Unit is responsible for the implementation of the Compact, including programme management, financial management and reporting, and coordination of M&E. The Programme is legally governed by the Compact and its supplemental agreements. For additional information please visit Consultancy-Specific Background The objective of the LMEF is to increase incomes to livestock producers and ensure the continued growth of the livestock sector. It is expected that this objective will be achieved through improving livestock marketing, eliminating barriers to existing and new markets and improving the marketability of livestock for broad application to the livestock industry in the NCAs to ensure the competitiveness and sustainability of the livestock sector. Given the lack of disease-free status in the NCAs and limited market access, farmers face high costs in the formal marketing of animals. Combined with the open access grazing systems, poor market incentives lead to overstocking of rangelands and degradation of the resource base and, consequently, to poor livestock nutrition and productivity. The estimated average off-take rate in the NCAs is only 7%, compared to 25% in the regions south of the Veterinary Cordon Fence. The LMEF should therefore impact on the livestock-producing communities in the NCAs through demand-driven actions that will improve the incomes of livestock producers. This will enable the livestock producers benefiting from the Fund to overcome the constraints of a lack of flexibility and efficient market outlets, and a poorly-functioning communal land tenure system. Specifically, this sub-activity seeks to improve livestock incomes in the NCAs by: Reducing costs and losses associated with marketing livestock in the NCAs; Alleviating other challenges to successful commercial marketing of livestock that are present in the current supply chain beyond the farm gate due to the lack of disease free status; and Identifying and eliminating barriers to increasing volume of livestock and livestock products sold into existing markets and accessing additional markets destinations. The implementation of the LMEF activity is being conducted through a two-phased process. The first phase involved (i) a synthesis of existing market studies where key market strategies are identified and recommendations for market improvements are provided and (ii) consultations with stakeholders within the livestock industry. This first phase resulted in the development of a Livestock Market Efficiency Fund Manual ( the Manual ), which provides clear guidelines on: use of funds; eligibility; evaluation criteria; proposal format; costing/cost-sharing guidelines; environment and social assessment guidelines; deliverables; ownership of new technology if applicable; and dissemination of results for broad and effective application and adoption livestock industry stakeholders. The second phase involves the implementation of proposals selected after each of a series of formal calls for concept papers (and then full proposals) from the industry for market improvement strategies. The following are practical examples of what concept papers/proposals may include: LMEF Evaluation Design Final Report. March

21 Research/projects addressing traditional/cultural/educational issues hampering livestock marketing in the NCAs; Activities leading to the achievement of disease-free status in the NCAs, thereby opening markets; The expansion of trade access to regional and global beef markets; The provision of livestock market information; Livestock vaccination programmes which meet consumer requirements; Improved rangeland nutrition, thereby addressing consumer requirements; The provision of improved genetic quality in bulls and rams; and The upgrading and/or constructing of holding pens and loading ramps at crush pens currently used by veterinarians for vaccinations, so that these sites can be used for cattle auctions. 3. Objective of the Evaluation and Related Research Questions 3.1 Objective Using data from the LMEF grantees and other sources, the evaluation will assess the impact of the LMEF on the intended beneficiaries mainly in terms of income or other quantifiable benefits. The evaluation should also look the impact of the Fund in terms of its contribution to reducing costs and losses associated with marketing livestock in the NCAs; alleviating other challenges to successful commercial marketing of livestock that are present in the current supply chain beyond the farm gate due to the lack of disease free status; and identifying and eliminating barriers to increasing volume of livestock and livestock products sold into existing markets and accessing additional markets destinations. Finally, the evaluation should look at whether the LMEF as a whole has achieved its stated objectives. 3.2 Research Questions To what extent does the LMEF contribute to increased incomes as well as other measures of livelihood among beneficiaries? Does the LMEF contribute to disease free status for the NCAs? If so, how and to what extent? Does the LMEF contribute towards the identification and elimination of existing marketing barriers and other challenges to successful commercial marketing of livestock in the NCAs? If so, what is the impact (e.g., increased volume of livestock and livestock products sold into existing markets in the NCAs and in existing and new market destinations)? Has the application of the LMEF led to any multiplier effects in terms of replication of grantee projects, extension of project outcomes, and dissemination of information? To what extent has the Fund as a whole achieved its stated objectives? LMEF Evaluation Design Final Report. March

22 Other questions, as relevant the specific details of each grant, will be added by the Consultant. The Consultant should become familiar with the LMEF and related grant projects and propose further ideas for evaluating the Fund. It will likely be necessary for the Consultant to come up with a simple mini evaluation design for each grant, each with its unique research questions. 4. Scope of Services 4.1 Overview of the Scope The scope of this task is to design and implement an evaluation of the LMEF using the most rigorous quantitative methodology possible, supplemented with qualitative data collection and analysis. The Consultant shall be responsible for full design and implementation of the evaluation but will need to work closely with MCA-N s Director: M&E, Director: Agriculture, Manager: M&E, Manager: Agricultural Grants, and the LMEF grantees. The approach must be approved by MCA-N before further work can proceed. The priority research questions initially identified were provided in section 3.2; however, as noted, the Consultant shall be expected to work with relevant stakeholders to verify, refine, and focus them as necessary, to maximize the evaluation s learning potential. The Consultant will participate in oversight of the grantees data collection activities to ensure that they meet the needs of the evaluation, and shall be expected to provide substantial evaluationrelated guidance and input to the grantees or other entities that may have existing data that contributes to the evaluation. MCA-N will facilitate this process. A high level of collaboration with other stakeholders as well as review and involvement in implementation, data collection and data quality issues is required. The evaluation work will be performed in three phases as follows: Phase 1: Evaluation Design and Planning Phase 2: Evaluation Implementation, Management and Analysis Phase 3: Communication and Finalisation 4.2 Tasks and Deliverables for Each Phase Phase 1: Evaluation Design and Planning (applicable at baseline, with updates to the Evaluation Design Report to be made at each grant round as needed) The Consultant shall review the LMEF Grant Manual and all relevant grant documents and available literature and data to determine the methodology needed to carry out the evaluation, including any additional data needs aside from that which can be gathered from grantees and that which can be obtained through Key Informant Interviews (KIIs) and focus group discussions (FGDs), and prepare a full evaluation design report. MCA-N must approve the evaluation design before the Consultant can move to the next phase. Specific Tasks for Phase 1 LMEF Evaluation Design Final Report. March

23 Review MCA-N s M&E Plan and relevant programme documents and survey documents to date to become familiar with the Compact and the LMEF Grant Manual and related project documents. The consultant shall at a minimum review the following: o o o o o MCA-N Compact Description; MCA-N Monitoring & Evaluation Plan; MCA-N Agriculture Project Description; MCA-N LMEF Grant Manual; and approved LMEF project descriptions. Conduct a literature review of relevant prior evaluations and research on livestock marketing in the NCAs and prepare an annotated bibliography of the identified documents; Conduct initial planning, discuss scope, refine research questions, and reach agreement on the evaluation design, related methodologies, and evaluation implementation with MCA-N s Director: M&E, Director: Agriculture, Manager: M&E, Manager: Agriculture Grants, and the LMEF grantees; Conduct field trips to the LMEF grantees intervention areas for familiarisation purposes, coordinated with the Fund s grantees; Prepare full evaluation design report, including at a minimum: o o o o o o Detailed evaluation methodology and approach; Refined research questions and proposed analytical model; List of data sources expected to be used for the evaluation and any data cleaning that the Consultant expects to perform as well as a plan for addressing and monitoring data quality for data sourced from grantees; Implementation plan and timeline for evaluation (i.e., a Work Plan and Staff Engagement Plan); Risk assessment for evaluation, and proposed mitigation measures; and Recommendations for incorporating evaluation design into the implementation plan for the LMEF grant projects. The design report will initially focus on what is known about grants approved to date and what is expected for future grants, but it may need to be revised to account for additional grants, depending on their content. Meet with MCA-N management to present the proposed methodology, discuss and get feedback on the evaluation design, evaluation methodologies and evaluation implementation issues and incorporate any changes and suggestions to it; LMEF Evaluation Design Final Report. March

24 Prepare a set of informational materials about the evaluation and its approach, which can be used by MCA-N in presentations, discussions with stakeholders, etc., in order to communicate clearly about the evaluation. Key Deliverables for Phase 1 (1) Evaluation Design Report that, at a minimum, addresses the following: a. Detailed evaluation methodology and approach; b. Refined research questions and proposed analytical model; c. List of data sources expected to be used for the evaluation and any data cleaning that the Consultant expects to perform as well as a plan for addressing and monitoring data quality for data sourced from grantees; d. Implementation plan and timeline for evaluation (i.e., a Work Plan and Staff Engagement Plan); e. Risk assessment for evaluation, and proposed mitigation measures; f. Recommendations for incorporating evaluation design into the implementation plan for the LMEF grantees projects. As noted previously, the design report will initially focus on what is known about grants approved to date and what is expected for future grants, but it may need to be revised to account for additional grants, depending on their content. (2) Informational materials that, at a minimum, include the following: a. A PowerPoint presentation explaining the basics of the evaluation, the evaluation design s advantages over other designs given the context and other key points from the evaluation design report; and b. A Frequently Asked Questions document covering key points about the evaluation approach, i.e., a document explaining the basics of the evaluation in a question-and-answer format and in language that is easily accessible to members of the general public Phase 2: Evaluation Implementation, Management and Analysis (applicable to the evaluation for each grant cycle 5 As noted previously, the Consultant may not begin Phase 2 until MCA-N has approved the Evaluation Design Report deliverable from Phase 1. Specific Tasks for Phase 2 5 A grant cycle is defined as the point from which a formal Call for Concept Papers is released and the end of the two-year period for grants approved on the basis of that particular Call (or grant round). It is anticipated that there will be three (3) grant rounds (i.e., three points in time over the course of the Compact when a formal Call for Concept Papers is issued). Depending on the availability of LMEF monies, a further (i.e., fourth) Call for Concept Papers may be considered. The end of the third (and possible fourth) grant cycle is expected to more or less coincide with the end of the Compact LMEF Evaluation Design Final Report. March

25 Once the design report has been approved (again, it will initially focus on what is known about grants approved to date and what is expected for future grants, but it may need to be revised to account for additional grants, depending on their content), the Consultant may proceed with the agreed-upon work plan, which will inform the specific tasks to be completed in this phase, which should be performed in accordance with the following guidelines: Communicate the evaluation progress regularly according to a frequency to be agreed upon with MCA-N; Produce minutes of key meetings and distribute the minutes to meeting participants and to MCA-N within five (5) days after the meeting; Collect data (conduct KIIs and FGDs); Continue to participate in oversight of any other relevant data collection and related data quality measures to ensure that they meet the needs of the evaluation; Update the design report elements as necessary for any issues or needed adjustments based on the course of grantees project implementation, clearly identifying and justifying any recommended changes; Once relevant data collection has been completed, analyse the data and other information to answer the evaluation questions; and Prepare and submit quarterly progress reports. Key Deliverables for Phase 2 (1) Updated Evaluation Design Report (if necessary); (2) Quarterly progress reports whose content shall at a minimum include: summary of implementation activities, performance against evaluation work plan and timeline, any issues and concerns that have arisen and mitigation measures taken, any adjustments made based on the course of the implementation, clearly identifying and justifying any recommended changes; and (3) Draft Evaluation Report that, at a minimum, addresses the following: a. Detailed evaluation methodology and approach; b. Refined research questions and analytical model used; c. List of data sources used for the evaluation and any data cleaning that the Consultant performed as well as how data quality was addressed and monitored; d. Descriptions of how the evaluation and how the LMEF activity were implemented; e. Findings from the evaluation; f. Lessons learned and recommendations (especially with regard to how the positive findings might be sustained post-compact); and g. Conclusion. LMEF Evaluation Design Final Report. March

26 4.2.3 Phase 3: Communication and Finalisation (applicable to the evaluation at the end of each grant cycle and at end-compact) The Consultant may not begin Phase 3 until MCA-N has approved the updated Evaluation Design Report and Draft Evaluation Report deliverables from Phase 2. Specific Tasks for Phase 3 Specific tasks for Phase 3 would be informed by the agreed-upon work plan in the Evaluation Design Report but will include, at a minimum: planning and conducting a stakeholders workshop to discuss the findings presented in the Draft Evaluation Report; recording stakeholders comments and suggestions for possible incorporation into the Final Evaluation Report; updating the Draft Evaluation Report accordingly; and submitting a Final Evaluation Report. Key Deliverables for Phase 3 (1) Stakeholders workshop and related workshop report that includes, at a minimum: a. An executive summary that sums up the full report; b. Background and goals of the workshop; c. List of workshop participants; d. A synopsis of the proceedings; e. A summary of all feedback received from workshop participants; f. Primary recommendations and conclusions; g. A roadmap as to how the outcome of the workshop will be applied to the evaluation and/or to its deliverables. (2) Final Evaluation Report that includes the minimum requirements laid out for the Draft Evaluation Report (see section 4.2.2); (3) Informational materials that, at a minimum, include: a. A PowerPoint presentation explaining the basics of the evaluation, the evaluation design s advantages over other designs in this context, the findings of the evaluation, lessons learned and how the positive findings might be sustained post-compact, and key conclusions drawn; and b. Frequently Asked Questions document covering the key points noted in point a. above, i.e., a document explaining the basics of the evaluation, the evaluation design s advantages over other designs in this context, the findings of the evaluation, lessons learned and how the positive findings might be sustained post-compact, and key conclusions in a questionand-answer format and in language that is easily accessible to members of the general public. 5. Duration of the Assignment From the time of contracting through to the end of the Compact (15 September 2014). LMEF Evaluation Design Final Report. March

27 6. Payment Schedule Item number Deliverables Due date Payment amount (% of contract value) Payment amount (US$) Phase 1: Evaluation Design and Planning (applicable at baseline) 1 Evaluation Design Report 4 weeks after contract effective date 15 tbd 2 Informational materials 2 weeks after MCA-N approves Evaluation Design Report 1 tbd Phase 2: Evaluation Implementation, Management and Analysis (applicable for grant cycle 1) 3 Quarterly progress reports (approx. 8 total) per agreed-upon work plan 1 tbd 4 Draft Evaluation Report per agreed-upon work plan 5 tbd Phase 3: Communication and Finalisation (applicable at the end of grant cycle 1) 5 Stakeholders workshop and related workshop report 4 weeks after MCA-N approves Draft Evaluation Report 1 tbd 6 Final Evaluation Report 2 weeks after the stakeholders workshop report is approved by MCA-N 7 Informational materials 1 week after Final Evaluation Report is approved by MCA-N 10 tbd 1 Tbd Phase 2: Evaluation Implementation, Management and Analysis (applicable for grant cycle 2) 8 Quarterly progress reports (approx. 8 total) per agreed-upon work plan 1 tbd 9 Draft Evaluation Report per agreed-upon work plan 5 tbd Phase 3: Communication and Finalisation (applicable at the end of grant cycle 2) 10 Stakeholders workshop and related workshop report 3 weeks after MCA-N approves Draft Evaluation Report 1 tbd 11 Final Evaluation Report 2 weeks after the stakeholders workshop report is approved by MCA-N 12 Informational materials 1 week after Final Evaluation Report is approved by MCA-N 10 tbd 1 Tbd Phase 2: Evaluation Implementation, Management and Analysis (applicable for grant cycle 3) 13 Quarterly progress reports (approx. 8 total) per agreed-upon work plan 1 tbd 14 Draft Evaluation Report per agreed-upon work plan 5 tbd Phase 3: Communication and Finalisation (applicable at the end of grant cycle 3) LMEF Evaluation Design Final Report. March

28 15 Stakeholders workshop and related workshop report 3 weeks after MCA-N approves Draft Evaluation Report 1 tbd 16 Final Evaluation Report 2 weeks after the stakeholders workshop report is approved by MCA- N 10 tbd 17 Informational materials 1 week after Final Evaluation Report is 18 Draft Evaluation Report (comprehensive, incorporating all grant rounds and the LMEF as a whole) 1 tbd 21 July tbd 19 Stakeholders workshop and related workshop report 3 weeks after MCA-N approves Draft Evaluation Report 3 tbd 20 Final Evaluation Report 2 weeks after the stakeholders workshop report is approved by MCA- N 21 Informational materials 1 week after Final Evaluation Report is approved by MCA-N 15 tbd 2 tbd 7. Evaluation Criteria: Qualifications and Experience A Consultant will be selected based on qualifications and proven experience to do the work. Applications are invited from individual consultants who are recognised evaluation experts who can demonstrate the ability to be responsible for the technical and methodological leadership of the evaluation. Applicants should, at a minimum, possess the following qualifications, skills and professional experience: A Master s degree in Agricultural Science, other relevant social sciences, or statistics. At least 5 years experience of designing and implementing evaluations, and having worked on quantitative as well as qualitative analysis of data. Preferably 5 years experience in managing or overseeing evaluation projects in agricultural settings (preference will be given to those with experience in agricultural marketing). Excellent communication skills and experience in working with a wide range of individuals in government, private sector and civil society. Good written and verbal communication skills in English. Excellent knowledge of SPSS or similar statistical analysis software as well as other software used in the analysis of qualitative data. LMEF Evaluation Design Final Report. March

29 Annex 2 Itinerary Date Activity People met Mo 6 Feb Departure from the Netherlands Tu 7 Feb We 8 Feb Th 9 Feb Fr 10 Feb Mo 13 Feb Tu 14 Feb We 15 Feb Th 16 Feb Arrival Windhoek Inception/kick-off meeting Elizabeth Feleke, MCC Deputy Resident Director Eline van der Linden, MCA-N Deputy CEO Kandi Shejavali, MCA-N Director: M&E Indongo Indongo, MCA-N Manager: M&E Helmke Sartorius von Bach, MCA-N Director: agriculture Mwazi Mwazi, MCA-N Manager: livestock Irene Nunes, MCA-N Grants manager Victor Paereli, MCA-N Manager: procurement Rainer Hassel, AGRA-PSD, grantee Susanne Thalwitzer, Meat Board, grantee Axel Rothauge, AGRA-PSD, grantee Gabriel Hangara, Komeho, grantee Document review Document review Document review Planning meeting MCA-N Document review Evaluation design Document review Evaluation design Document review Evaluation design Travel to Ongwediva Fr 17 Feb Filed visit to Enghandja village See Annex 5.1 Su 19 Feb Mo 20 Feb Tu 21 Feb We 22 Feb Travel to Windhoek Evaluation design Report writing Meeting with Meat Board Travel to Tsumeb Susanne Thalwitzer, Project coordinator Th 23 Feb Travel to Outapi Rainer Hassel, Project coordinator Fr 24 Feb Field visit to Omusati Region Rainer Hassel, Project coordinator Sa 25 Feb Travel to Ongwediva Su 26 Feb Planning meeting Indongo Indongo, MCA-N Manager: M&E Mo 27 Feb Travel to Nkurenkuru Indongo Indongo, MCA-N Manager: M&E LMEF Evaluation Design Final Report. March

30 Tu 28 Feb We 29 Feb Th 1 Mar Fr 2 Mar Sa 3 Mar Meeting with Agra team Short meeting with Gopa team Field visit to Kankundi Field visit to Kankundi Travel to Windhoek Debriefing preparation Debriefing meeting MCA-N Departure from Windhoek Arrival in the Netherlands Axel Rothauge Cornelis van der Wal Angelina Kanduvarisa Martha Sheepo Heinrich Piolok, Project Manager Alex Endunde, Regional Field Manager Esther Lusepani, Community Development Expert Johannes Penny Akwenye, MCA-N CEO Kandi Shejavali, MCA-N Director: M&E Indongo Indongo, MCA-N Manager: M&E Mwazi Mwazi, MCA-N Manager: livestock LMEF Evaluation Design Final Report. March

31 Annex 3 Documents consulted General documents: MCA-N: Livestock Market Efficiency Fund Grant Manual; April 2010 MCA-N: Monitoring and Evaluation Plan, Revision 2; 2 February 2011 MCA-N: Indicator Tracking Table MCA-N: Thematic Analysis Report Livestock; November 2008 MCA-N: Back to office report: Baseline survey of nutrition in the Northern Communal Areas of Namibia collection of hot-wet season samples Kankundi site, Kavango Region; February 2012 CTA, Agritrade: Beef sector; July 2011 University of Pretoria, Bulletin Foot and Mouth Disease: Southern Africa; December 2011 Grant project MCAN/LMEF/2010/01: Unleashing the potential of livestock sector in Oshana and Ohangwena Regions: Grant Agreement Inception Report, 30 April 2011 Quarterly Progress Report 1, April - 30 June 2011 Quarterly Progress Report 2, 1 July Sept 2011 Desktop study on available training materials, 9/30/2011 Updated M&E plan, June 2011 Grant project MCAN/LMEF/2010/02: Development of export opportunities for beef products from the Caprivi Region: Grant Agreement Inception Report, 26 May 2011 Quarterly Progress Report 1, 21 July 2011 Grant project MCAN/LMEF/2010/03: Communal Areas in Namibia: Sero-epidemiological and Parasite Survey in the Northern Grant Agreement Inception Report, 24 May 2011 Quarterly Progress Report 1, 23 August 2011 Quarterly Progress Report 2, 7 November 2011 Comments and Responses Trail on QPR1 LMEF Evaluation Design Final Report. March

32 Comments and Responses Trail on QPR2 PPT Presentation First Review Workshop, 2 November 2011 Procurement Report 1, 1 May July 2011 Procurement Plan 2, 1 August October 2011 Manual for Animal Health Technicians, R.H. Hassel; 2011 Grant project MCAN/LMEF/2010/04: Baseline Survey of Animal Nutrition in the Northern Communal Areas of Namibia: Grant Agreement Inception Report, 13 May 2011 Quarterly Progress Report 1, 7 July 2011 Quarterly Progress Report 2, 21 October 2011 Quarterly Progress Report 3, 7 January 2012 Comments and Responses Trail on Inception Report Comments and Responses Trail on QPR1 Comments and Responses Trail on QPR2 Comments and Responses Trail on QPR3 LMEF Evaluation Design Final Report. March

33 Annex 4 Summary sheets on the grant projects Annex 4.1 MCAN/LMEF/2010/01 Annex 4.2 MCAN/LMEF/2010/02 Annex 4.3 MCAN/LMEF/2010/03 Annex 4.4 MCAN/LMEF/2010/04 LMEF Evaluation Design Final Report. March

34 Annex 4.1 MCAN/LMEF/2010/01 Project ID Project Name Implemented by Partners/stakeholders MCAN/LMEF/2010/01 Unleashing the potential of livestock sector in Oshana and Ohangwena Regions Komeho Namibia Development Agency Ministry of Agriculture, Water and Forestry/MAWF (Directorate of Engineering and Extension Services/DEES, Directorate of Veterinary Services/DVS) Oshakati Pharmacy MEATCO Regional Councils Traditional authorities Input suppliers Project period 01/04/ /04/2013 Grant awarded N$ 1,864,501 Location Objectives Ohangwena and Oshana Regions The project aims to improve livestock marketability in the Oshana and Ohangwena Regions, through training and capacity building of livestock farmers. The specific objectives are: To secure and increase livestock farmers income in target NCA s - by improving livestock farming practices in target NCAs so that cattle better meet market needs To strengthen the direct links between communal cattle farmers and key stakeholders To establish a network of expertise in modern cattle farming methods in communal areas to facilitate sustained, ongoing improvement. To establish case studies that other communal cattle farmers will wish to follow Methodology Begin with six months preparatory work, appointing project staff, securing buy-in from all stakeholders and targeted village communities, confirming of training needs, preparing the details of the core training and support to be provided mobilization of beneficiaries for project implementation, and strengthening links with key stakeholders. Then implement in three batches of four communities: - Initial introductory training and identifying a community mobiliser or mobilsers - Programme of training direct to communal farmers and providing more detailed training to community mobilisers. Monitoring system Carrying out a needs review /visioning exercises, baseline studies, and development and maintenance of data banks of selected monitoring criteria gathered during and at the end of the project. LMEF Evaluation Design Final Report. March

35 Project summary The project aims to create 12 case study NCA village communities in the two regions which show how NCA farmers can increase the income that they earn from their livestock by adopting improved management of their livestock and so improving the quality at point of sale. This will be done by delivering to these 12 communities training on and support to establish improved livestock farming practices (livestock production, animal health, rangeland management, infrastructure management (water and fencing) and other key actions which assist improvement), strengthening the direct links between communal cattle farmers and key stakeholders (e.g. improving links to the market and market information) and ensuring good infrastructure support (training in technical skills linked to fencing and water for example). A network of expertise will be established in these communities through community mobilisers who will offer encouragement and support in their own communities as well as being able to offer their expertise to other livestock farmers. This project will then be promoted to encourage other communal livestock farmers to follow their example. It will involve consecutively delivering to three sets of four village communities, six months practical training; more intensive training to a selected mobiliser for each community to act as a champion for the improvements to be carried out, with a slightly higher level of skills than their peers; then providing follow up mentoring support on a less intensive basis. The methodology will also ensure that feedback is gathered from participating farmers and used constructively to improve the project. Action will also be taken to ensure that accurate information is gathered at the start, during and the end of the project to meet reporting requirements and measure its impact. QPR1 Appointed Project Manager (the only deliverable due at this point) Detailed project implementation plan for the preparation period Plan for implementation of training has also been drafted Training materials were obtained from almost 30 organisations/sources To support monitoring and evaluation of the project, a summary of the action needed and templates for reports at each key event (training, site visits, surveys etc) have been prepared and are now being reviewed. No progress was made on Indicators as per Grantee M&E plan during this reporting Quarter QPR2 12 final target communities selected As was not anticipated, the formal requests for support made to the Ministry of Agriculture, Water and Forestry were rejected. The reason being that the Ministry is unable to provide staff to the project because the Ministry staff s are overburdened with implementation of Ministry s programs and projects in the regions. Desktop study report on available training materials Prior to commencing with the desktop study, the project staff conducted training needs assessments with twelve target communities LMEF Evaluation Design Final Report. March

36 Thereafter project staff summarized the communities training needs identified into nine key topics Forty-four training materials comprising of manuals, books, posters and leaflets have been gathered from a wide range of organizations. To address the identified community training needs, three key training manuals and 11 posters to be used were identified and agreed upon by stakeholders. Community meetings were held to identify potential candidates to act as community mobilisers No progress was made on indicators as per Grantee M&E plan during this reporting quarter Initially, KOMEHO NAMIBIA proposed to train only 120 farmers from the twelve target communities but after being advised by Constituency Councillors, this decision was changed to avoid discrimination and exclusion of those willing to participate in the project. KOMEHO NAMIBIA is recommending to MCA-N to accommodate and train all interested farmers, but taking into consideration the initial target of 120 farmers to be trained. Desktop study on available training materials All stakeholders consulted urged KOMEHO NAMIBIA to make use of these existing materials for training message consistency, to save costs and to avoid duplication of efforts. The communities training needs segregated per target village are presented in Annex II. summary, the training needs or key topics identified by community members are: In Animal Health Management Dealing with emergencies i.e. assisting livestock during difficult birth Drought and floods preparedness and adaptation strategies General managerial practices (dehorning, castration, branding, animal handling techniques) Livestock marketing Rangeland management Record keeping Selection of breeding stock Supplementary feeding The review found three prominent manuals that cover all training topics or needs identified by the communities. These training manuals are: Training manual on Community Based Rangeland and Livestock Management by GOPA Training manual on Community Animal Health Agents of National Agricultural Support Services Programme (NASSP) by the MAWF Training manual on Livestock Marketing in Namibia by NNFU The review also found 11 complementary posters that cover training topics or needs identified by the communities. These posters published by Meat Board of Namibia are: Animal Health Management (Anthrax, Botulism, Black-quarter) Animal Health Management (Brucellosis, Mastitis, Foot Rot) Animal Health Management (Contagious Bovine Pleuropnemonia (CBPP) / Lung sickness) Animal Health Management - Deficiency of Minerals in Livestock (Lack of salt, Lack of phosphorus, Lack of iron, Lack of iodine) Animal Health Management - External Parasites & Skin Diseases (Mange, Ticks, Lice, Flies & Fleas, Ringworm) Animal Health Management (Foot and Mouth Disease / Ekondo Nelaka) LMEF Evaluation Design Final Report. March

37 Animal Health Management - Internal Parasites (Round Worms, Tapeworms, Liver Flukes) Animal Health Management (Pasteurellosis, Sweating Sickness, Anaplasmosis) Beef Cattle Marketing Namibian Beef Carcass cuts Selection of Beef Cattle LMEF Evaluation Design Final Report. March

38 Annex 4.2 MCAN/LMEF/2010/02 Project ID Project Name Implemented by Partners/stakeholders MCAN/LMEF/2010/02 Development of export opportunities for beef products from the Caprivi Region Meat Board of Namibia TAD Scientific Institute for Tropical Diseases at the Faculty of Veterinary Medicine at the University of Pretoria Istituto Caporale, Italy MEATCO Directorate of Veterinary Services Project period 15/03/ /12/2011 Grant awarded N$ 1,684,500 Location Objectives Caprivi Region Overall aim to increase the livelihood of cattle farmers in the NCAs through the creation of access to high value markets and improvement of livestock management. Of importance is that these efforts support an integrated approach to livestock production and wildlife conservation Specific aim is the development of a system allowing for the international export of beef products from the Caprivi. Another specific aim to be reached synergistically is optimised livestock production and marketing. The aims will be reached by realisation of the following objectives: Detailed research and development of protocols and standard procedures for the pilot phase Production and slaughter systems optimising the utilisation of livestock resources Improvement of livelihood of cattle producers International accreditation of the CBT concept as consequence of this project Methodology Activities will include a detailed market analysis, value chain description and analysis, the application of HACCP/CBT approaches and laboratory research. Main activities will be the improved management, increased slaughter and proof of the safety of the system by means of laboratory testing for FMD antibodies Project summary Cattle producers in the NCAs of Namibia are generally excluded from access to lucrative export markets due to two major factors: (i) the endemic occurrence of FMD restricts trade with countries representing valuable LMEF Evaluation Design Final Report. March

39 markets, and (ii) a combination of a rather traditional approach to livestock keeping and marketing, lack of knowledge on appropriate livestock husbandry and management practices as well rangeland of insufficient quantity and quality lead to slaughter animals of inferior quality. For the Caprivi region, however, FMD eradication is not realistic. Hence, an alternative approach is needed in order not to exclude cattle farmers from future market access. The approach which will be tested and evaluated in this project is an integration of the hazard analysis critical control point (HACCP) approach and the Commodity-Based Trade (CBT) concept to achieve effective food safety and animal disease risk management. In order to address not only the safety, but also the quality of the product, the project will involve cattle producers which commit to improved animal management practices and the delivery of quality animals to a local export abattoir. When it comes to marketing of cattle, the driving force is the immediate need for money rather than an in-advance planned marketing schedule. This is reflected in a very low off-take rate of 7% per year (compared to 25% in the South). Per year, two percent of the cattle go to the formal market (MEATCO export abattoirs), the rest of the marketed cattle go to the informal market. The MEATCO abattoir in Katima Mulilo has got a HACCP system in place and is certified for export to RSA and other regional markets. Daily slaughter capacity is 110 cattle; total throughput in 2010 was 7577 cattle. Phase 1 of the project is mainly consisting of theoretical research work which needs to be done to develop detailed value chain descriptions, market analysis and protocols relating to HACCP procedures, production protocols, etc. Another important component is the FMD virus testing done in an internationally accredited FMD. The 50 farmers who are being mentored own a total of 6,000 7,000 cattle and are distributed throughout all the constituencies of Caprivi The programme will eventually influence 12,000 households with 156,000 cattle across the entire NCAs QPR1 All subcontracts were signed between the grantee and the three subcontractors The audit at the Katima Mulilo abattoir and cutting plant has been carried out, in order to verify level of compliance with EU standards and International standards. The audit was aimed to assess also the food safety system implemented at the abattoir, particularly concerning HACCP principles implementation. Information has been collected on beef production in Caprivi region, in order to draft a flow diagram. This will facilitate the integration of HACCP principles in the animal health sector, with the view to adopt an integrated HACCP and CBT approach. LMEF Evaluation Design Final Report. March

40 Sampling protocols have been developed for the quarantined cattle and abattoir testing. An additional activity has been added upon suggestion of the University of Pretoria. The activity is called Natural Resource Base in the study area, which was seen as an important part of the research focussing on the integration of livestock production and nature conservation. Progress made on Indicators as per Grantee M&E plan Compact Year 2 (Oct-2010 to Sept-2011) Indicator Unit Quarterly Status (May 2011 end July 2011) Notes number of identified products number In progress 2 weeks delay in submission of market analysis report anticipated number of identified markets number In progress 2 weeks delay in submission of market analysis report anticipated Number of identified value chains number In progress 2 weeks delay in submission of value chain report anticipated Adapted Livestock Producer Forum (LPF) Mentorship Programme Incorporated suggestions following community consultation and workshops Signed work contract Identified value chains are integrated in the programme of the LPF Mentorship Programme Incorporated suggestions following community consultation and workshops which improve the practical implications of the programme Work contract & work programme agreed with internationally accredited FMD lab Will be done once value chain identification is completed Suggestions were noted during community visits and will be incorporated into the protocols The contract between TAD and the respective laboratory was signed none none Equipment for the collection and storage of samples will have to be organised and purchased by TAD within their research budget LMEF Evaluation Design Final Report. March

41 Annex 4.3 MCAN/LMEF/2010/03 Project ID MCA/LMEF 2010/03 Project Name Implemented by Partners/stakeholders Sero-epidemiological and Parasite Survey in the Northern Communal Areas in Namibia AGRA Professional Services Beneficiaries: communities located at or near the research sites, Directorate of Veterinary Services, Directorate of Engineering and Extension Services (DEES) and MCA/ Community Based Range Land Management (CBRLM). Project period 01/05/ /04/2013 Grant awarded N$ 3,952,080 Location Objectives NCAs Aim: Due to the virtual complete lack of baseline epidemiological and parasite infection data for the NCAs, this project proposes that such data be collected by systematic sampling of blood, faeces and other tissues in four regions of the NCA The specific objectives are: 1. To collect baseline epidemiological data regarding certain infectious diseases in cattle and goats in the NCAs; 2. To make the above data available to DVS; 3. Protocols and standard operating procedures for DVS staff for the investigation and diagnosis of abortion problems in goats and infectious fertility problems in cattle will be developed. In addition a user-friendly diagnostic tool kit with standard operating procedures is to be developed for DVS field staff and they will be trained in all of the above; 4. To make the data obtained from the serological and parasite survey also available to the site communities, where these data were obtained. Methodology The main activities include identification of research sites and communities, collection, processing and dispatch of samples, processing of data, publishing of data and results, developing a diagnostic tool kit and investigation protocols. In each of the selected regions between 50 and 60 bulls, 800 other cattle and between 400 and 500 goats in complete herds or flocks will be sampled, a total of 240 bulls, 4,000 other cattle and 2,400 goats. All sampling activities will be used to simultaneously train DVS staff in correct sampling and processing techniques as part of the development and implementation of investigation protocols as requested by the Chief Veterinary Officer. LMEF Evaluation Design Final Report. March

42 The prevalence of the various conditions and diseases will be plotted and recorded. The results will be published and made available to the Chief Veterinary Officer, MCA- Namibia and other stakeholders. Project summary This project only concerns the collection of baseline data and the development of investigation protocols. These can then be utilised at a later stage to control infectious diseases and parasite infection which will lead to improved fertility and production, which in turn should lead to increased income and reduced overstocking and rangeland degradation, since smaller numbers of livestock can be kept. The data will be made available to the Directorate of Veterinary Services (DVS) for the development and implementation of control and eradication programs for specific infectious diseases and to advise farmers on the appropriate use of anthelmintics (internal parasite remedies), as well as develop and implement human hygiene programs in co-operation with the relevant health authorities (toilets and regular de-worming) to eliminate human tapeworm infestation and concurrent bovine and porcine cysticercosis. The research data will automatically become part of the epidemiological database of DVS. The collection of baseline data will have no direct benefit to livestock owners, but will enable the development strategies to control and prevent venereal diseases, infectious causes of abortions and parasite infestation. Justification: Low calving rates in cattle Abortions in goats Bovine cysticercosis or bovine measles. condemned or retained Infected carcasses are Internal parasites of cattle sheep and goats There are no protocols and standard operating procedures in place for any of the above mentioned diseases for DVS field staff in the NCAs. The development of such protocols and implementation on a trial basis would solve this problem and also lead to results and epidemiological data that can be compared. Training is also required for DVS field staff in these protocols and a toolkit needs to be developed for use by DVS staff when investigating any of the above disease problems. QPR1 Activities mainly focused on field visits and community outreach meetings, stakeholder consultations planning meetings with DVS and Central Veterinary Laboratory (CVL), procurement of equipment and the first sampling activities. QPR2 LMEF Evaluation Design Final Report. March

43 Activities mainly focused on field visits and community outreach meetings sampling of animals, and training of Animal Health Technicians in the East Caprivi (including printed training handouts). Other main activities included compiling a diagnostic tool kit and procuring one sample kit as well as drafting disease investigation protocols and compiling a manual for Animal Health technicians. The fourth main activity involved collection, processing and submission to the CVL of blood samples, stool samples and peripheral blood smears of cattle and goats in East Caprivi. The last main activity was the first MCA Review workshop, which was held on the 2 nd of November in Windhoek. Project Risks: frequent changes at DVS and CVL management personnel. This can and has resulted in previous decisions being reversed at very short notice without any consideration for the negative effect on the implementation of the activities, resulting in unnecessary complications. In order to mitigate the effect the project leader will work a lot more closely with the lower management and state veterinarians and hold DVS and MAWF to written contracts and agreement if and when necessary. LMEF Evaluation Design Final Report. March

44 Annex 4.4 MCAN/LMEF/2010/04 Project ID Project Name Implemented by MCAN/LMEF/2010/04 Baseline Survey of Animal Nutrition in the Northern Communal Areas of Namibia AGRA Partners/stakeholders Project period 01/04/ /04/2014 Grant awarded N$ 8,809,378 Location Four main agro-ecological zones (Kaokoveld, Kalkveld, Ekuma floodplains and Kalahari sand plateau) Objectives Methodology Project Summary Four head of cattle and four goats at each one of the eight sites will be used to obtain blood serum samples, hair samples, milk samples and a faecal grab sample. Liver samples will be obtained from another eight animals. These five different types of animal tissue samples will then be analyzed for their nutrient content, primarily their content of proteins (nitrogen fraction), macro- and micro-minerals. Animal tissue sampling will be repeated systematically over three different seasons of the year. The Baseline Survey of Animal Nutrition attempts to identify and alleviate nutritional problems of livestock animals free-ranging in the northern communal areas of Namibia. It will supplement the Community-Based Rangeland and Livestock Management Project of the LMEF, which attempts to increase the quantity of grazing on offer by improved grazing management. Namibian experience has shown that, once the quantity of grazing is satisfactory, the nutritional value of the grazing becomes the next factor limiting animal production. Thus, the Baseline Survey samples soil, drinking water, forage plants and livestock animals directly to determine their nutrient status and identify nutritional deficiencies, excesses and other problems (Objective 1). These will then be addressed in a lick supplementation pilot trial (Objective 2), before the message will be passed on to the primary producer in the NCA in an extensive outreach effort (Objective 3). QPR1 The following activities were completed i. Activity 1.1: Pre-implementation planning ii. iii. Activity 1.2: Exploratory field visits Activity 1.3: Training of assistants in field methodology. LMEF Evaluation Design Final Report. March

45 iv. Deliverable 1: Inception report. The research protocol was finalized prior to the start of the field data collection phase. All research intervention sites have been finalized and the farmers/headmen responsible for the area identified and their cooperation obtained. QPR2 During the period under review (July-September 2011), the following Activities were engaged in: i. Activity 1.5: Design a database to capture all field and laboratory data. This Activity is an on-going process. ii. Activity 1.6: Data entry into the database and data clean-up. This Activity is an on-going process. iii. Activity 1.7: Submission of samples to the laboratory, which has to provide results within 3 months of sample submission (i.e. in November 2011) iv. Activity 1.8: Analysis of cold-dry season field data by laboratory v. Activity 1.10: Collect field samples in the hot-dry season. This Activity is still on-going and will only be completed by end November These Activities resulted in the following Deliverables being reported in this Quarterly Report: i. Deliverable 1.3: Inventory of cold-dry season samples and other field data collected. ii. Deliverable 1.4: Database in Excel format. This Deliverable is on-going and the database will be changed, enlarged and continually developed as more data comes in. In all probability, its format will keep on evolving until the last data has been collected and entered as it is not always possible to envisage the final format of a database when you enter the first pieces of data. QPR3 During the period under review (October December 2011), the following activities were engaged in: i. Activity 1.9: Update database with field data on soil, plant and animal nutrient status and laboratory analysis pertaining to the cold-dry season. ii. iii. iv. Activity 1.10: Collection of hot-dry season field samples was completed by end November Activity 1.11: Data entry into the database and data clean-up, associated with HD-season data. Activity 1.12: Submission of hot-dry season samples to the laboratory, which has to provide results within 3 months of sample submission (i.e. in March/April 2012) v. Activity 1.13: Analysis of hot-dry season field data by laboratory (see Appendix I; letter confirming receipt of samples) LMEF Evaluation Design Final Report. March

46 These activities resulted in the following deliverables being reported in this Quarterly Progress Report number 3: i. Deliverable 1.5: Laboratory analysis report pertaining to cold-dry season of 2011 (see Chapter 3). ii. Deliverable 1.6: Updated database with categorized field measurements and sample analysis of cold-dry season data. LMEF Evaluation Design Final Report. March

47 Annex 5 Field visit reports Annex 5.1 Field visit to Enghandja Annex 5.2 Visit to Meat Board, Windhoek Annex 5.3 Field visit to Omusati Region Annex 5.4 Field visit to Kankundi LMEF Evaluation Design Final Report. March

48 Annex 5.1 Field visit to Enghandja Project ID Project Name Implemented by Visited location MCAN/LMEF/2010/01 Unleashing the potential of livestock sector in Oshana and Ohangwena Regions Komeho Namibia Development Agency Enghandja Village, Oshikango Constituency, Ohangwena Region Visit date 17 February 2012 Activity attended People met Training on emergency procedures and animal nutrition Gabriel Hangara, Komeho Julius Ambond, Komeho, project manager Sarafina Hapulile, Komeho, project assistant Sergius Kanyangela, Komeho, project assistant Florence Msati, Komeho, home economics Wellem Shiponeni, DVS Eenhana Job Mvula, DVS Omafo Bernadinu Shekutamba, DEES Omafo Evelina Shuuluka, DEES Omafo Elikia Iyambo, DEES Omafo Victor Hakko, Ondangwa Farmers Market Description The training was one of the last trainings in the 6-months training programme provided to cattle keepers in the village of Enghandja. The training started as scheduled at 2 pm. Lunch was provided to all participants. The first training subject was on emergencies. The training was given by Job Mvula of DVS Omafo. He described a number of emergencies that may occur in cattle where the farmer can do no treatment himself, but needs to contact the veterinary services. He explained about the following cases: difficult birth, abortion, broken horns, fractures, poisoning, retained placenta and bloat. The second training was on animal nutrition, and especially on supplements. The training was given by Victor Hakko of Ondangwa Farmers Market. This a private enterprise, where animal supplements are sold. There was a good turnout of farmers. About 50 farmers attended the training, of whom the majority were women. The total number of households in the village is 198, meaning that about a quarter of the village was represented. Farmers seemed genuinely interested and asked a good number of relevant questions. LMEF Evaluation Design Final Report. March

49 Farmers were given handouts by the project, which were kept in files together with handouts given at earlier trainings. Observations The high turnout and active participation by farmers indicates that there is a demand for technical training. Apparently it is not common that farmers get an opportunity like this to discuss technical issues with technical experts from DVS or DEES. DVS and DEES, as well as private partners, may consider to make it common practice to have regular technical sessions with farmers at village level. Since the first training cycle of the Komeho project is about to end, it is important that lessons are learnt and documented. It would be good if the project staff makes an inventory of the strong and weak points of the training programme, based on questions asked to the participants: which training they liked best/least and why. It would even be good to ask villagers who did not participate why they didn t do so. LMEF Evaluation Design Final Report. March

50 LMEF Evaluation Design Final Report. March

51 Annex 5.2 Visit to Meat Board, Windhoek Project ID Project Name Implemented by Visited location MCAN/LMEF/2010/02 Development of export opportunities for beef products from the Caprivi Region Meat Board of Namibia (MBN) Windhoek office Visit date 21 February 2012 Activity attended People met Description Discussion with project manager Susanne Thalwitzer, Meat Board, project manager Because the abattoir in Caprivi has been closed (first for maintenance, then because of the outbreak of FMD) the project has also been interrupted and no activities take place in the field. Susanne explained about the general of the MBN (parastatal, statutory body) and its role in the NCAs. MBN is self-financed through levies on marketed/slaughtered livestock. Some of the income is used to support livestock keepers in the NCAs through projects. The LMEF and its objectives fits well with the objectives and approach of MBN. After the closure of the abattoir in Caprivi MBN requested, and got approved, postponement of the deliveries of phase 1 of the project. This was not yet reflected in any progress reports as it was apparently agreed between MBN and MCA-N that no QPRs are required as long as there is no progress. Susanne promised to forward relevant correspondence on the postponement with the evaluator. MBN has also requested MCA-N to approve the start of the second phase of the project which can run concurrently with the last stages of the first phase. The livestock market study, combined with the value chain analysis is now expected by end of March Susanne shared a DVD Beauty and the beef, which explains the background to the project. Observations Although it was agreed between MBN and MCA-N not to produce any more quarterly progress reports after QPR1, this seems awkward. Even if no deliverables were produced, some progress must have been made, for LMEF Evaluation Design Final Report. March

52 example administratively, in expanding the first phase project period and decisions on new dates for deliverables. Now there has been no further reporting since July 2011, now seven months ago. It is recommended to maintain a quarterly frequency for reporting, for transparent reporting of all progress, even if implementation is behind schedule. Susanne and the evaluator agreed to keep each other informed on relevant developments. The consultant will inform MBN early on upcoming evaluation visits, to allow for timely planning of field visits. LMEF Evaluation Design Final Report. March

53 Annex 5.3 Field visit to Omusati Region Project ID Project Name Implemented by Visited location MCAN/LMEF/2010/03 Sero-epidemiological and Parasite Survey in the Northern Communal Areas in Namibia AGRA-PSD Omusati Region Visit date 24 February 2012 Activity attended People met Description Collection of samples in the wet season Rainer Hassel, AGRA Professional Services, Project coordinator Rainer had been in the field with two teams of five people since 20 February. The teams were camping in the sampling areas, Rainer was staying overnight at Outapi, where I met him on Thursday night, 23 February. The next morning we left early, with two vehicles, to the area indicated for sampling. This was in Omusati Region, South of the road from Okahao to Opuwo. Because the road is under construction and it rained heavily the previous days and nights, progress along the road was very difficult. After the turn off to the village both vehicles got stuck and it took a long time to get them out of the mud. Various alternative tracks were tried thereafter, but with no success. Since time was passing rapidly it became less feasible for me to attend of the sampling and get back to the hotel in time. So the visit was then cancelled. Although it was unfortunate that I could not attend the sampling, I got quite of feel of the area and its conditions. Fortunately we had the opportunity to talk on Thursday night, where Rainer explained about AGRA and the project. Observations Rainer explained that the actual sampling is done by DVS staff. They are trained on the job by Rainer. This seems an important aspect of the project from a sustainability point of view. LMEF Evaluation Design Final Report. March

54 LMEF Evaluation Design Final Report. March

55 Annex 5.4 Field visit to Kankundi Project ID Project Name Implemented by Visited location MCAN/LMEF/2010/04 Baseline Survey of Animal Nutrition in the Northern Communal Areas of Namibia AGRA-PSD Kankundi, Kavango Region Visit date 27/29 February 2012 Activity attended Mapping of forage plants in selected grazing sites. Collection of forage samples. Collection of water samples. Collection of blood serum samples, hair samples, milk samples and a faecal grab sample of cattle. Collection of liver samples. People met Rainer Hassel, AGRA Professional Services, Project coordinator Cornelis van der Wal, AGRA Professional Services, Researcher Angelina Kanduvarisa,, AGRA Professional Services, Project Assistant Martha Sheepo, AGRA Professional Services, Project Assistant Gopa team, implementing CBRLM project Farmers providing animal samples Description During this field visit the consultant was accompanied by Indongo Indongo, MCA-N Manager: M&E. We met the AGRA-PSD team in Nkurenkuru on 27 February 2012 and had opportunities to get briefed on the project and procedures in the evening. The next morning there was a short coordination meeting with the Gopa team implementing the Community Based Range Land Management (CBRLM) project. The meeting took place in Nyege, on the way to Kankundi. The remainder of the day was spent on collection data on forage plants, by recording species on transect walks and collection of clippings. Data collection took place in Kankundi. On 29 February we observed (and assisted with) the collection of blood serum samples, hair samples and a faecal grab sample of four cattle of one of the participating farmers in Kankundi. Observations The collection of samples (from plants as well as animals) is an intensive work. It is all done by the team of four (or usually three) AGRA-PSD project staff. It was noted that there is little or no participation by government people or institutions. It is therefore unclear how the valuable collected data and LMEF Evaluation Design Final Report. March

56 resulting recommendations will be put to use in an institutionalised way. This may put the sustainability of the project at risk. It is acknowledged that AGRA-PSD has invited government on various occasions, but that there has been little or no response. Nevertheless this issue remains requiring attention. Perhaps involvement of local (district) authorities is a possibility. The significance of findings needs to be noted in relation to the number of samples and the variance (standard deviation) in the different data. LMEF Evaluation Design Final Report. March

57 LMEF Evaluation Design Final Report. March

58 LMEF Evaluation Design Final Report. March

59 Annex 6 Proposed alternative payment schedule Dates Activity Deliverable Due date Payment amount (%) as per ToR Payment amount (%) proposed Feb/Mar 2012 Evaluation design and planning Draft evaluation design report Comments by MCA-N 5 March March 2012 Final evaluation design report 20 March 2012 Information materials 3 April May/Jun st quarterly evaluation mission Quarterly progress report 20 June Aug/Sep nd quarterly evaluation mission Quarterly progress report 20 September Nov/Dec rd quarterly evaluation mission Quarterly progress report 3 December Feb/Mar th quarterly evaluation mission Quarterly progress report 20 March May/Jun th quarterly evaluation mission Quarterly progress report 20 June Aug/Sep th quarterly evaluation mission Quarterly progress report 20 September Nov/Dec th quarterly evaluation mission Quarterly progress report 3 December Feb/Mar th quarterly evaluation mission Quarterly progress report 20 March Apr-Jun 2014 Final evaluation grant cycles Draft evaluation report 30 May Stakeholders 30 June LMEF Evaluation Design Final Report. March

60 workshop and report Final evaluation report 14 July Information materials 21 July Jul-Sep 2014 Final comprehensive evaluation Draft evaluation report Stakeholders workshop and report 21 July August Final evaluation report 25 August Information materials 1 September Notes: The table is an adaptation of the Payment Schedule as attached to the Terms of Reference for the evaluation of the LMEF. It is suggested to combine the activities/deliverables for the different grant cycles, because the grant cycles will probably run concurrently. If it nevertheless transpires during implementation that different outputs are required for different grant cycles, the schedule will be adjusted accordingly. Because of the combination of reports for different grant cycles, the payment schedule also requires revision. In the table a revised payment schedule is proposed, where payment amounts fairly reflect the distribution of outputs over the project period. LMEF Evaluation Design Final Report. March