Monitoring Toolkit. Gardens

Transcription

1 EMERGENCY OPERATIONS AND REHABILITATION DIVISION (TCE) Monitoring Toolkit Gardens including Community, School, Backyard and Institutional garden projects September, 2008 by Andy Lyons, RIACSO Stanley Dlamini, Swaziland ECRU REGIONAL INTERAGENCY COORDINATION SUPPORT OFFICE (RIACSO) JOHANNESBURG, SOUTH AFRICA

2 THE FAO PROJECT MONITORING TOOLKIT SERIES This monitoring toolkit is one of a series that has been developed by the FAO Regional Interagency Coordination Support Office (RIACSO) in Other Monitoring Toolkits Developed by FAO/TCEO Johannesburg, South Africa. The goal of these Conservation Agriculture toolkits is to strengthen and harmonize monitoring Input Trade Fairs and evaluation of supported emergency activities Small Stock throughout the southern Africa region. To achieve Transboundary Animal Diseases this goal, these toolkits provide a core set of Small Scale Irrigation indicators and data collection templates that enable common data collection and the aggregation of results across multiple project sites within countries and the region. The audience for this monitoring toolkit is anyone who is involved in implementing garden projects. This includes FAO staff at country and regional levels, as well as implementing partners at specific project sites, government extension officers, NGO staff, and consultants. This is the second version of the gardens monitoring toolkit. The first version was developed in October This version reflects additional feedback from regional reviews and a season of pilot testing in Swaziland. The main changes in this version are the inclusion of supplemental indicators and data collection templates for specific types of garden projects, and some rearranging of the sections. These toolkits are a work in progress. To be successful they must address regional cross-cutting information needs on project effectiveness, while also accommodating a diverse set of working conditions, operational resources, and information requirements of numerous national and local users. This is not an easy task, hence feedback from users is essential. Please channel all feedback through your local FAO Emergency Coordination Unit (ECU). Comments should be sent to the Regional Information Officer at RIACSO. How to use this Toolkit 1. Review the core output and outcome indicators (page 3) 2. Review the sample data collection forms (page 23) 3. Meet with project staff and implementing partners and reach consensus on: a. Indicators and data collection instruments (Annex II, page 23) b. A workplan for data collection, processing, and analysis (page 15) 4. Implement the workplan. 5. Review and present results back to stakeholders 6. Revise workplan as needed. Development of this toolkit has been made possible by the generous support of The Government of the Republic of South Africa under OSRO/RAF/510/SAF. ii

3 TABLE OF CONTENTS An Introduction to Garden Projects...1 Monitoring Garden Projects...1 Categorizing Garden Projects...2 Gardening Core Indicators...3 Data Collection Workplan...6 Pre-activity planning...8 Selection of beneficiaries...8 Baseline household survey...8 Training...10 Planting, production, harvesting, and sales...10 End-of-season review...12 Terminal evaluation...12 Post-project evaluation...15 Making a M&E Workplan...15 References...17 Annex I. FAO s Regional Monitoring Strategy for southern Africa...19 Basic vs. Preferred Standards for Monitoring...20 Measuring Targeting, Household Vulnerability and HIV/AIDS Impact...21 Annex II. Sample Data Collection Forms...23 Adapting the forms...23 Garden Beneficiary Selection Worksheet...24 Garden Diary Section 1: Planting & Harvesting...25 Garden Diary Section 2: Profit and Loss Statement...26 Garden Diary Section 3: Rainfall (Format 1)...27 Garden Diary Section 3: Rainfall (Format 2)...28 Garden Diary Section 4: Tracking Market Prices...29 Production and Sales Summary...30 Instructor's Training Log...31 Garden Seasonal Summary...32 (for all gardens)...32 Household Garden Survey...35 School Garden Survey...39 Institutional Garden Survey...41 TABLE OF TABLES Table 1. Classification of gardens...2 Table 2. Garden core indicators...4 Table 3. Data collection tasks for each phase of implementation...7 iii

4 iv

5 AN INTRODUCTION TO GARDEN PROJECTS Gardens have been one of the mainstays of food security projects for many years, and for good reason. Gardening is a core livelihood strategy for almost all social strata in nearly all parts of Africa, so it is relatively easy to get participation for garden projects in both rural and urban settings. Gardening is also an extremely versatile development activity; projects can be designed to achieve a variety of goals from better nutrition, increased incomes, HIV/AIDS mitigation, to market development. From an implementation standpoint, the logistics of garden projects are somewhat easier than field crops because they tend to be located near settlements and often at a central location such as a school or water point. The relatively low requirements for land and inputs also make gardening a viable livelihood strategy for the poorest households, making these projects attractive to government and donors who want to help this hard-to-reach population. Successful garden projects must be tailored to fit the local agronomic conditions, dietary preferences, cultural traditions, and market context (FAO 2001). Experience has also taught us that traditional garden projects create the greatest impact when combined with one or more complementary activities, for example small livestock production, savings schemes, health and nutrition education, market development, or small-scale irrigation. For additional information on garden projects, see the References section (page 17). What is a garden? Gardens are not easily defined because there is as much variation in the characteristics of gardens as there is in the diverse contexts in which they re found. Even among experts there is no consensus on where the line falls between gardens and field cultivation. Most gardens are smaller and cultivated more intensely than field crops, and have a more diverse crop mixture. Gardens are also more likely to be irrigated, fertilized, fenced, closer to homes, and cultivated year round. However there isn t a single universal standard that can be applied across the board, as exceptions can be found to all of the above criteria 1. Fortunately, for the purposes of monitoring the term garden is just a label or category for a project, and we don t need to worry too much about trying to precisely measure it. The local understanding of a garden should be followed rather than trying to adopt an external or complicated set of criteria. The bottom line: if the project document says it s a garden, then it s a garden. MONITORING GARDEN PROJECTS To strengthen the monitoring of garden projects, we need to: 1. Develop a system for classifying the different types of garden projects 2. Define core indicators 3. Develop criteria for identifying food insecure households and households impacted by AIDS 4. Design a plan for the logistics or nuts and bolts of data collection and analysis 1 see also FAO (2001): Information Sheet 1: Definition and Concept of Home Gardens in Africa [ 1

6 Categorizing Garden Projects From a monitoring standpoint, gardens are not the easiest to monitor due to the great diversity in contexts and project designs. Among the garden projects supported by FAO in southern Africa, there are nutrition gardens as part of home-based care programmes, school gardens, community gardens, drip kit gardens, seed production gardens, hospital gardens, urban gardens, and gardens at Junior Farmer Field and Life Schools (JFFLS). While all of these garden projects share some common characteristics, such as vegetable production, most also have specific objectives or emphases that are tailored to the local context (Figure 1). The general and specific goals of a garden project should be defined in the project document. For example a community garden project might have a heavy emphasis on demonstrating the benefits of a new technology such as treadle pumps, whereas a backyard garden project might place a strong emphasis on improving household nutrition levels. Figure 1. Cross-cutting and project-specific objectives of garden projects school gardens household gardens cross-cutting objectives community gardens institutional gardens project specific objectives We need a way to categorize garden projects so that monitoring tools and protocols can be tailored appropriately, and results disaggregated when needed. For the purposes of this toolkit, gardens will be classified following the scheme used by Machakaire and Hobane (2005), shown in Table 1. Table 1. Classification of gardens Type of Garden Characteristics community garden a large joint garden where multiple individuals keep separate plots but share equipment, infrastructure, training, and maintenance responsibilities. communal garden a garden managed by a group of people who share both responsibilities and production, either equally or according to some formula agreed upon by the group household garden a garden maintained by a single household, also known as a backyard or kitchen garden institutional garden a garden managed primarily for nutrition or income by an institution, such as a hospital, neighborhood care point, orphanage, church, or prison school garden a garden based at a school often with educational objectives in addition to nutrition In addition to defining categories for the different types of gardens, to design data collection we also need to think about the structure of garden projects. This will help us define indicators and decide what needs to be measured and how. Figure 2 below illustrates a generic hierarchical structure that suits many garden projects. 2

7 Figure 2. Elements of garden projects Level Description Measurable characteristics country level the ECU is the main coordinating body at the country level number of project sites project site A geographic area (e.g., district) where garden projects are supported, usually by a single implementing partner number of gardens in the site gardens one or more beneficiaries An area, usually fenced, where gardening takes place. Management can be by a single individual, a community group, or an institution such as a school or hospital. Individuals who belong to a household and participate and benefit from a garden activity size of garden area under irrigation Variables measured at the number of garden level beneficiaries production and income crop diversification cultivation methods & technology production training provided vulnerability level household demography nutritional status time & labour gardening skills income Variables measured at the individual level Monitoring as a Tool for Participant Education Including beneficiaries in monitoring is a good practice in general, and is particularly important for garden projects. Garden projects require a high level of labour input, and if the project isn t perceived to be working in the eyes of the participants then it will be very difficult to sustain. Furthermore monitoring plays an important role in the learning process which is an explicit goal of most garden projects. Many of the suggested tools in this toolkit therefore are built upon a strategy of using monitoring not only for assessment but also as a tool for training. Gardening Core Indicators During an M&E workshop in February 2006, FAO Emergency Coordinators from southern Africa identified a set of core indicators for garden projects (Lyons, 2006). These are presented in Table 2 below. 3

8 The difference between output and outcome indicators Monitoring is often divided into looking at the process of conducting an activity, and its ultimate impact. Process is measured by describing the immediate outputs, such as the number of inputs distributed or people trained. Impact is evaluated by defining and measuring the medium and longterm outcomes of an activity. In order to claim that an activity made a difference, we need evidence of both the process and the final impact. Hence most monitoring systems simultaneously keep track of both by defining separate indicators for outputs and outcomes. Table 2. Garden core indicators Indicator Output Indicators Basic 1. Number of gardens established, by type, size, and age 2. Number crops per garden, disaggregated by type and size of garden 3. Number of beneficiaries, disaggregated by gender and level of vulnerability 4. Number of gardens using promoted technologies, disaggregated by type of technology 5. Number of beneficiaries trained, disaggregated by type of training and gender School 6. Number of students who benefit from the garden, disaggregated by type of benefit (instruction, school feeding, take vegetables home, etc.) 7. Number of hours of instruction time per week, disaggregated by subject Outcome Indicators Basic 8. Quantity of production disaggregated by crop 7. Quality of dietary intake / meals served (Food Consumption Score) 9. Number of months of production per year Interpretation This indicator will tell us the number of gardens supported, how many are new, and distribution of their sizes. This indicator will capture patterns of crop diversification in the supported gardens, and whether certain garden types or sizes are more diverse than others. This indicator captures the overall scope of garden projects in terms of beneficiaries. This information can also be presented in map form. Many garden projects are designed to demonstrate the effects of promoted technologies, such as biocontrol plants or live fencing. This indicator will capture the types and extent of new technologies tested. Training is a key element of most garden projects, this indicator will capture both the types of training offered as well as number of beneficiaries Students can benefit from school gardens in a variety of ways. This indicator will simultaneously highlight the breadth and depth of impact. This indicator reflects the degree to which the garden is used for educational purposes, and the variety of topics integrated into garden activities Production for consumption or income is one of the primary outcomes of most garden projects. The Food Consumption Score is an index of the consumption of major food groups in the past week, weighted by their nutritional content. It is a good outcome indicator for access to food and the ability to transform food into a diet. This indicator captures one of the main goals of garden projects which is to smooth out food production throughout the year, particularly during the hungry season 4

9 Preferred Indicator Interpretation 10. Number of households who ate Although this indicator doesn t capture the type or or sold produce from supported degree of impact, this is a core indicator that cuts gardens, disaggregated by level of across most activities supported by FAO vulnerability 11. Cropping intensity Cropping intensity is the ratio between the total amount of area cultivated summed over all growing cycles in a year, divided by the physical area prepared for cultivation. It is a measure of intensity of the land and captures whether a garden is being fully utilized. 12. Income generated from garden sales 13. Percentage of beneficiary households who practice garden skills for a second year with little or no direct support Income is an important asset for livelihood security and one of the desired outcomes of many household and community garden projects. This is one of the long-term indicators that reflects the level of sustainability of an activity. What about measuring nutritional status? Most people involved in garden projects hope to ultimately see an improvement in the nutritional status of participants and their families. However, in most cases nutritional status is not necessarily a good indicator for a garden project because garden projects by themselves only address some of the factors that affect nutritional status. Garden projects should improve availability, access, and diversity of food, but improved nutritional status also depends on good health and hygiene, maternal and child care, skills in food storage, and the ability to prepare well-balanced nutritious meals. If a garden project has these additional health and nutrition components (e.g., through joint programming with government or UNICEF), then an indictor for nutritional status would make sense. As an alternative, dietary intake is a reasonably good outcome indicator that captures both availability and the ability to transform crops into a diet. Dietary intake can be measured using the Food Consumption Score which is relatively simple to measure and captures both the diversity of a diet as well as its nutritional content. Therefore this indicator gives us a viable means for tracking how we may be contributing to changes in nutritional status before and after the project. 5

10 Data Collection Workplan At a glance: Basic level data collection Pre-activity planning: conduct a baseline assessment of the physical characteristics of a sample of supported garden sites (does not apply to new gardens) Early in the season: conduct a baseline household survey on a sample of individual participants (backyard and community gardens only) Training: trainers maintain a simple training log, recording the numbers of people trained and topics covered Planting, harvesting, and sales: garden owners/managers keep a diary of planting, harvesting, and sales activity End of season: garden managers and extension officers compile a summary of the season, including training provided, crops planted, production, and sales End of project: an end-of-project beneficiary survey is repeated using the same sample as the baseline The most challenging component of many M&E systems is data collection. A data collection workplan is used to plan the amount of time, human resources, transport, and other resources needed to collect data. A good data collection workplan recognizes the tradeoffs between the information needs, available resources for fieldwork, and the project context. The most important characteristic of a data collection workplan is that it is viable, even if that means collecting less data than you would ideally like. Figure 3. Data Collection Workplan info needs & indicators staff & operational resources project context data collection workplan A data collection workplan takes into account the information needs, operational resources, and project context to come up with a plan for data collection, processing, and analysis that is viable. Table 3 below describes the main steps in implementing a garden project and the accompanying data collection tasks at each step of the process. The sections that follow describe the data collection tasks in greater detail. This table can serve as a guide for a data collection workplan, but is 6

11 incomplete because it doesn't specify individual personnel who will be assigned for each task, geographic areas, transport requirements, amount of time needed, budget requirements, etc. Table 3. Data collection tasks for each phase of implementation Implementation Phase Data Collection Tasks Basic Preferred (in addition to basic) pre-activity planning & garden baseline assessment (page 8) selection of beneficiaries (page 8) baseline household assessment (page 8) training (page 10) planting & production (page 10) harvesting and sales (page 10) end-of-season review (page 12) terminal evaluation (page 12) post-project evaluation (page 15) discussions with stakeholders conduct a baseline assessment of the existing garden sites (page 32) with sample size of 20% (min 20, max 30) (excluding new gardens) garden beneficiary selection worksheet (page 24) administer baseline household survey (page 35) random selection of beneficiaries sample size 5% (min 30, max 200) training log maintained (page 31) garden owner/manager records key events in garden diary (page 25) same sample as baseline garden manager fills in monthly production and sales summary (page 30) same sample as baseline garden manager fills in seasonal summary sheet (page 32) administer end-of-project household survey with same sample as baseline (page 35) stakeholder interviews focus group discussions focus group discussions with potential participants baseline sample size for improved garden projects: 40% (min 20, max 60), including ¼ non-supported gardens 2% (maximum 50) of the Garden Beneficiary Selection Worksheets randomly selected for entry into the monitoring database for analysis of the selection process sample size 10% (min 30, max 400) 25-30% of sampled households should be non-beneficiaries GPS coordinates recorded additional knowledge assessments and evaluations GPS coordinates for each garden recorded additional focus group discussions separate focus groups for men/women external consultant contracted for evaluation revisit some of the sampled households for semi-structured interviews stakeholder interviews focus group discussions repeat as needed 7

12 Pre-activity planning During the pre-activity planning phase, the details of the project design are finalized, sites identified, and roles and responsibilities of partners defined. For improved garden projects (as opposed to new gardens), this is also an opportunity to collect some baseline data on the existing garden sites using the Garden Seasonal Summary (page 32). The focus of this assessment is on the physical characteristics of the garden size, infrastructure, water source, etc. If production and sales data for the previous season are available, these can be entered as well. These data will be used as a baseline comparison for physical improvements to the gardens. All Garden Seasonal Summary Sheets should be entered into the monitoring database. While it may not be possible to visit all of the existing gardens for a baseline assessment, the Basic monitoring standard calls for 20% of project sites to be visited (maximum 20), while the Preferred standard encourages at least 40% of the existing gardens to be assessed (maximum 50). These sites should be randomly selected so that they are representative of all gardens in the project area. For small garden projects, it may be feasible to make a baseline assessment of all the supported gardens. In addition, the Preferred level of monitoring encourages baseline measurements for some nonsupported gardens. These control group gardens should be similar to the project sites as much as possible, with the exception of level of external support. Garden projects supported by another programme would generally not be a good choice for the control group, because we are not usually trying to compare projects against each other. The Preferred monitoring standard also calls for some focus group discussions with potential beneficiaries to learn more about gardening practices, skills, markets, water sources, inputs availability, etc. Such discussions normally take place at the beginning of a project anyway, a focus group discussion is merely a more structured format for focusing and recording the discussion. Selection of beneficiaries The selection of beneficiaries is a big process which is frequently community-based. These factors put limits on the amount of data that can be feasibly collected. The information required at this stage is therefore not detailed information on each and every beneficiary, but general information on the process and criteria by which beneficiaries are selected. The Garden Beneficiary Selection Worksheet (page 24) is designed to help the groups doing the selection to be systematic about the criteria they use, but is not intended to be collected and entered into the monitoring database at the Basic level of monitoring. However the Preferred standard calls for a randomly selected sample of beneficiary selection worksheets (2% with a maximum of 50) to be entered so that a profile of the targeting criteria used can be generated. Baseline household survey The Household Garden Survey (page 35) is the first of two household-level surveys used in backyard and community gardens to collect more detailed information on the household and their previous experiences with gardening. Data from the baseline survey is also used to determine the level of vulnerability of households, which will say something about the targeting methods, and the percentage of participants that come from households affected by HIV/AIDS. The baseline survey should be conducted near the beginning of the garden project, for example during the selection of beneficiaries or soon after the planting activities begin. The same households that are selected for the baseline survey will be revisited during the end-of-project survey, hence it is very important that the households for the baseline survey are selected randomly so that they 8

13 represent the entire population of participants. A good way to randomly select households for the baseline survey is to: 1. decide the total number of participants to be surveyed, based on the needs and resources available for fieldwork, data entry, etc. 2. decide which area(s) will be visited, making sure that at least some households are included from each agro-ecological region, market zone, urban proximity, etc. 3. decide how many participants will be visited in each of the sampled areas (either divide the total desired sample size by the number of areas evenly, or make the sample size in each area proportionate to the total number of participants in each area) 4. take the participant list from each of the selected areas, and select the every n th name on the list, where: total number of beneficiaries in the area n = desired number of households for surveying The Basic standard of monitoring calls for 5% of participants to be selected for the baseline survey, with a minimum of 30 and a maximum of 200 for the entire project. If the Preferred standard is used, the total sample size should be 10% of the total number of beneficiaries, with a minimum of 30 and a maximum of 400, of which ¼ to ⅓ should be non-participant households for comparison purposes. Non-participant households should also be selected as randomly as possible, from the same areas as the participant households, and be involved in other gardening activities if possible. The Preferred standard also recommends recording GPS coordinates of sampled households in order to facilitate relocating the same household during the end-of-project household survey and visualizing the spatial patterns in vulnerability and project outcomes. Our office staff is small and our implementing partners don t have the capacity to do survey work. What should we do? In the real world, operational constraints and institutional dynamics can make it difficult to survey an even moderate sample like 5%, particularly when a project has a late start or field staff have no experience or incentives to do survey work. In these cases, you should still survey as many households as resources permit, and concentrate on making the exercise a learning experience for project staff and partners. Although the small sample size may make it difficult to make definite conclusions about the project as a whole, some interesting lessons can be learned even from small surveys. If nothing else, you are guaranteed to learn some lessons on implementing surveys, such as the amount of time and training needed for enumerators, changes needed to the survey forms, human resources needed for the data entry process, etc. This can help you plan better for the following season. In addition to downsizing the sample, other options may also exist for conducting surveys with limited resources. Many organizations seek to share the costs of data collection by partnering with other organizations. For example the Swaziland ECU uses university students as enumerators when they come for attachment. In Zimbabwe, FAO coordinated 27 NGOs to implement a survey to collectively measure the impact of a number of similar input provision schemes. 9

14 Training Most garden projects include a set of training activities in areas such as production skills, food storage and preparation, health and nutrition, or life skills. The Basic level of monitoring requests training staff to maintain a simple log book of training activities (see Instructor's Training Log, page 31). This template is simple enough to be used by all trainers, including community-based trainers. At the end of each growing season, training logs should be collected and summarized and the total number of trainings, participants, and contact hours recorded on the Garden Seasonal Summary (page 32). The wide array of training topics makes it difficult to provide a universal assessment tool for the many types of training, however the Preferred monitoring standard calls for use of training assessments appropriate for the context. Training assessments can focus on measuring changes in awareness, knowledge, values, or behavior. A variety of assessment methods are used in training evaluations, including interviews, observation of behavior, garden inspections, and focus group discussions. We do a lot of training in our garden project, so why isn t there an outcome indicator for the impact of training? Many garden projects include a training component to improve participants skills in areas such as production, nutrition, food processing, irrigation maintenance, or marketing. The number of topics and people trained is one of the core output indicators for garden projects (Table 2, page 4), and a template for recording the outputs of training is included in Annex II (page 31). However there are no core outcome indicators for training, for two reasons. First, measuring the impact of training properly is very intensive and beyond the capacity of many implementing partners. You have to give knowledge assessments and observe behavior over a period of time that often exceeds the duration of the project. Second, it is very difficult to aggregate information on training impact from different projects and sites, especially when the topics of training are so varied. This does not mean that it isn t important to assess the impact of training, because without assessments of training we really have no way of knowing whether the training is relevant to the participants or if the skills are really sinking in. However this type of focused evaluation is better suited for a terminal evaluation or post-project assessment. Some sample questions for a training assessment are provided in the section on terminal evaluations (page 12). In addition, projects with a heavy training emphasis are encouraged to define additional indicators to measure specific training outcomes for specific projects. Planting, production, harvesting, and sales Monitoring planting and production in gardens is difficult, because unlike field crops cultivation and harvesting take place almost continuously. Hence for record keeping to be effective, it must originate at the garden level, preferably updated on a daily basis. Garden owners/managers can keep track of important events such as planting, harvesting, and sales by maintaining a garden diary using a template such as the one shown in Annex II (page 25). The garden manager is the individual with the overall responsibility for a garden. In an institutional garden, this is usually a staff member of the organization. In a backyard garden, this is an individual 10

15 in the household. A community garden may or may not have a manager from the community. If a garden manager is absent, an extension officer can work with the group to identify someone who can take responsibility for record keeping, or individuals within the garden can maintain diaries for their own plots. Experience has demonstrated that community-level monitoring is more likely to be effective if the information collected is relevant to local needs. Maintaining garden records is useful for many purposes other than monitoring. Keeping a garden diary can be used for various training purposes and demonstrating the effectiveness of various production methods. Record keeping however might be a new skill that will require initial training and extension support. Introducing garden-level record keeping may be particularly challenging in backyard garden projects, because vulnerable households tend to lack formal education as well. When garden diaries don t seem viable or the needed resources for training and extension are lacking, the following strategies might be helpful: community-based extension workers, lead gardeners, or local training staff can provide support to local garden owners/managers in record keeping garden-level record keeping systems can be pilot tested at a sample of garden sites NGO or government extension workers can take charge of record keeping, for example through biweekly or monthly visits To help compile garden-level production and sales data, it is also helpful to have intermediate summaries that can then be aggregated into a final seasonal summary. The Production and Sales Summary sheet (page 30) is a template for compiling daily records of production and sales into a monthly summary. Training might also be needed for this form, including how the information can be used to improve planning and management. Best Practices in Implementing a Survey: Training and Field Supervision As much as we try to ensure that a survey form is clear and easy to use, there has never been a survey that didn't cause some confusion in the field. Hence an important component of survey work is providing training for enumerators. This doesn't have to be extensive, however training is essential to ensure that all enumerators understand the questions, translate them into the local language in a similar manner, record answers similarly, measure the size of fields using the same method, etc. It s also helpful to pilot-test the survey beforehand to catch any mistakes or confusion in the way the questions are written. A second best practice for survey work is field supervision. Particularly in the early stages of data collection, it is important for survey forms to be reviewed by a supervisor as they come in, so any problems or confusion can be quickly identified, diagnosed, and corrected before they cause any data to be thrown out. 11

16 End-of-season review The end of every garden cycle warrants a review regardless of whether project support is coming to an end or will continue for another season. The three-page Garden Seasonal Summary (page 32) is designed to assist an end-of-season assessment by summarizing the physical features of the garden as well as training, production, and sales activity. The seasonal summary can be filled out by garden owners/managers, with assistance from extension staff as needed. The seasonal summary sheet relies on the Garden Diary and/or Production and Sales Summary for production and sales data, and training log(s) for the training summary. If production or training data are not available, these sections should be skipped and the rest of the summary sheet filled out. All seasonal summary sheets should be collected and entered into the monitoring database as several of the core indicators are derived from this data. The Preferred standard also recommends conducting some focus group discussions with participants at the end of each season to explore issues that were missed or too complex to capture during the season. Issues such as the impact of training, gender issues, relationship between the stakeholders, and planning for the next season are examples of possible topics for discussion. The other half of the story: Interpreting indicators Indicators are useful in telling us what happened, but that's only one-half of the story. The other half is why the observed results occurred. To enable the reader to interpret our indicators, we must describe the context during the reporting period. For example production and income might be low not because participation or training was poor but because the fencing material was delivered late. Our description of the results should describe the factors which affected the success of the activity both positively and negatively. Terminal evaluation Terminal evaluations are often a contractual obligation, but should really be seen as an opportunity to share results with partners, review experiences, document lessons learned, and make recommendations for future garden projects. If funds are available, an outside consultant Preferred can be helpful by bringing a wider set of evaluation skills and a broader perspective on garden projects. However even when funds are tight, an internal evaluation is highly worthwhile for packaging, interpreting, and communicating the results of the monitoring system. The elements of a final evaluation often include: analysis and interpretation of project monitoring data interviews with major stakeholders focus group discussions with beneficiaries assessment of the broader context 12

17 An important source of information for a terminal evaluation of a backyard or community garden project is the Post-Harvest section of the Household Garden Survey (page 37). This section of the household survey repeats many of the same questions that were asked on the baseline, allowing a comparison of practices. The Preferred monitoring standard calls for some non-participants to be included in the end-of-project survey as a source of comparison data. We haven t got all of the monitoring data from our implementing partners, and the terminal evaluation report is due next week! What should we do? Missing or delayed data is a fact of life in project monitoring, especially when you rely on implementing partners to collect and/or enter data for you. When you re up against a deadline and some data hasn t been collected and/or entered into the monitoring database, you can still produce summaries with what you have. Although more data usually produces more accurate estimates of total impact, preliminary estimates can still be generated in order to meet a reporting deadline. Delayed data should also trigger a lesson learned for future monitoring. Was the data collection workplan too ambitious? Were the expectations for implementing partners clearly articulated and incorporated into the Letter of Agreement? Were there technical problems with the fieldwork or data entry? By diagnosing the problem you can improve the monitoring system for future reviews. Most terminal evaluations also include focus group discussions and interviews with stakeholders including community leaders, partners from government, other NGOs, and donors. The Preferred monitoring standard calls for separate focus groups for men and women to ensure that gender issues come out clearly and without bias. Separate discussion groups may also be needed if there are other sub-groups among the participants, such as youth or individuals living with HIV/AIDS. Questions that should be addressed in a terminal evaluation are usually outlined in a Terms of Reference, and should be based on the issues that emerged during the course of the season. It is generally preferable to articulate 3-5 key issues for a project evaluation to focus on, rather than ask every question in the book and risk getting an evaluation that is extremely general or based on scanty evidence. See also FAO (1998) for guidelines on planning a terminal evaluation. 13

18 Sample Evaluation Questions for a Terminal Evaluation of a Garden Project 2 Some typical issues addressed in terminal evaluations include: Describe the patterns in vulnerability of the beneficiaries, including impact from HIV/AIDS. Summarize the achievements of the garden project (output and outcome indicators). Describe any observed changes in dietary intake of the beneficiaries. What training was provided, and what are the observable outcomes of this training? How did beneficiaries perceive the garden project? How did implementing partners and other stakeholders perceive the garden project? Describe any barriers to implementation. Describe any gender issues that emerged during the project. Discuss any gender patterns in the core indicators. Discuss the roles of women in all phases of the project (planning, implementation, evaluation), and whether the strategies used by the project to include women were effective. How did households affected by HIV/AIDS benefit from the project? What strategies were used to mainstream the needs of HIV/AIDS impacted households into project design? Were they effective? If technology trials were part of the project design, discuss the results of the trial, the project s strategy for information sharing, uptake, etc. Describe any project-specific objectives (e.g., educational achievements in school gardens, health outreach in hospital gardens) and discuss observed outcomes. What impacts did the garden project have on livelihoods? What will be needed to protect any gains in livelihood, and what additional assistance could be needed to maintain the progress made? Describe any significant changes that took place in the broader context, such as climatic patterns, organizational or political changes, inflation, etc., and how these changes might have affected the success of the garden project Describe any unanticipated benefits or spin-offs that emerged from the garden project, such as additional investment attracted, self-initiated group activities, new relationships between the beneficiaries and extension staff from government and private companies, expansion of technologies demonstrated, etc. Describe any unexpected problems created by the garden project, such as intra-community or intra-household conflicts, elite capture, etc. Describe issues concerning sustainability of the garden project Recommendations for future garden projects 2 see also Machakaire and Hobane (2005) 14

19 Post-project evaluation Output and outcome indicators do a reasonably good job in tracking the short and medium term impacts of gardens, however other anticipated results will only emerge a year or more after the project is over. Post-project evaluations are rare in emergency programmes because there is usually no requirement nor funding to conduct them. Nevertheless, the Preferred monitoring standard recommends a post-project evaluation because there is no other way to determine whether the promoted technologies and approaches for gardening are sustainable on their own or depend on continued donor subsidies. One strategy for conducting a post-project evaluation is to make it a joint exercise with other organizations that have also been promoting gardens, and to include evaluations of past gardens in the M&E workplan of current garden projects. A post-project evaluation is similar in approach to a terminal evaluation, with particular emphasis on changes in the broader context, unanticipated outcomes, creation of dependency, and sustainability of project impact. The best way to prepare for a post-project evaluation is to ensure that the terminal evaluation is well-documented, and data are archived in an appropriate format for re-analysis. MAKING A M&E WORKPLAN Indicators and data collection templates are necessary, but won't collect any data by themselves or produce any analyses. For that you need a workplan which operationalizes the details of monitoring. Each country office needs to take the lead on developing its own workplan for garden M&E as part of its overall M&E work plan in line with local resources, information needs, and operational constraints. A monitoring workplan specifies: what data is going to be collected, when and by who (i.e., data collection workplan, page 6) how, when, and where paper forms will be entered or captured in a database or spreadsheet who is going to do the analysis and write-up estimated costs and time requirements associated with the above tasks This toolkit is not a workplan, but the tools and recommendations included provide many of the core elements of a M&E workplan. To produce a monitoring workplan, the following additional steps are recommended: 15

20 Steps in Developing an Monitoring Workplan 1. Read through this toolkit to get an overview of what is needed and ideas for your specific monitoring system. 2. In addition to the information needs which have already been identified, make a list of the most important national and local level information needs and questions that you would like to address through monitoring. 3. Look at the list of Basic and Preferred indicators on page 3, and decide which ones will be adopted for your country. 4. Think about the specific factors that make households vulnerable to food insecurity in the local context, and come up with a vulnerability index ranging from 1 to 3, based upon the responses to questions in the baseline household survey (see discussion on page 21). 5. Think about the indicators of HIV/AIDS impact, and how to determine whether a household is classified as impacted by AIDS based on the responses to questions in the baseline household survey (page 35). 6. Discuss the indicators and monitoring needs with implementing partners, outlining information needs and options for data collection. 7. Look at the sample data collection forms in Annex II, and based on the chosen indicators decide which ones will be used and whether they need to be modified. 8. Modify the data collection forms as needed. Contact RIACSO and other users of the garden monitoring toolkit if changes to the forms or database are needed. 9. Develop a workplan which outlines all of the needed M&E tasks, including the sampling methodology, field work, data entry, analysis, and writing. The workplan should specify the dates and staff involved in each task. 10. Integrate monitoring responsibilities into job descriptions, individual workplans, and Letters of Agreement. 16

21 REFERENCES Bilinsky, P., and Swindale, A Months of inadequate household food provisioning (MIHFP) for measurement of household food access: Indicator guide. Food and Nutrition Technical Assistance Project (FANTA). Washington, DC. [ FAO Evaluation Mission Brief. FAO Evaluation Service, Rome. [ FAO Improving nutrition through home gardening: A training package for preparing field workers in Africa. Food and Agriculture Organization of the United Nations, Rome. [ FAO School gardens concept note: Improving child nutrition and education through the promotion of school garden programmes. FAO interdepartmental working group, Rome. [ FAO Setting up and running a school garden. A manual for teachers, parents and communities. FAO, Rome. [ FAO A garden for everyone self-tutorial manual. FAO Regional Office for Latin America and the Caribbean. [ Landon-Lane, C Livelihoods grow in gardens: Diversifying rural incomes through home gardens. FAO Diversification Booklet 2. FAO Agricultural Support Systems Division, Rome. [ftp://ftp.fao.org/docrep/fao/006/y5112e/y5112e00.pdf] Lyons, A monitoring and evaluation framework for FAO emergency and rehabilitation activities in southern Africa. FAO RIACSO, Johannesburg, South Africa Machakaire, V., and Hobane, A.P.P Review of garden based production activities for food security in Zimbabwe. FAO Zimbabwe Working Group on Gardens. Harare. Marsh, R Building on traditional gardening to improve household food security. Food, Nutrition and Agriculture. No. 22, pp [ftp://ftp.fao.org/docrep/fao/x0051t/x0051t02.pdf] Mitchell, R. and Hanstad, T Small homegarden plots and sustainable livelihoods for the poor. LSP Working Paper 11. FAO Livelihood Support Programme, Rome. [ Swindale, A., and Bilinsky, P Household dietary diversity score (HDDS) for measurement of household food access: Indicator guide. Food and Nutrition Technical Assistance Project (FANTA). Washington, DC. [ 17

22 18

23 ANNEX I. FAO S REGIONAL MONITORING STRATEGY FOR SOUTHERN AFRICA Monitoring versus Evaluation monitoring: the process of collecting information in a structured manner, based on a specific set of goals and objectives evaluation: the process of summarizing and analyzing information to answer specific questions about a project This toolkit is one of a series developed by FAO Emergency Operations Division in southern Africa to strengthen and harmonize project monitoring and evaluation throughout the region. Project monitoring in general serves three purposes: accountability monitoring allows funders, partners, and beneficiaries to see whether a project is being managed responsibly, is on course, and achieving the desired impacts. implementation monitoring facilitates implementation of an activity. For example, project managers need to know the problems faced by farmers raising small stock in order to plan training activities and procure appropriate inputs. improving project design monitoring allows project staff to make improvements in both the design and implementation of projects, for example targeting, the sequencing and timing of tasks, design of training, complementary services, etc. FAO TCEO and other humanitarian relief agencies are trying to strengthen monitoring for all of the above reasons. To strengthen activity monitoring in seven countries in southern Africa, FAO/TCEO is using a three-pronged approach (Lyons, 2006): a conceptual framework outlining a regional M&E system based on the information needs and operational resources of FAO and its partners a set of core outcome indicators which cut across multiple activities and project sites, in order to show aggregated impact at national and regional levels a series of monitoring toolkits like this one providing guidelines and tools for monitoring different types of activities Why all this new fuss over monitoring? What's wrong with the way we've been reporting? Monitoring in emergency interventions is frequently a challenge due to time constraints, limited staff resources, and the short timeframe of most projects. M&E is therefore often delayed until the project is nearly over, or focuses only on the immediate outputs of an activity. A recent review of project terminal reports has shown that most project reporting has not met the basic information needs of project planners and funding agencies, who have a strong interest in project impact, lessons learned, and the linkages between activities. Without evidence to demonstrate that activities are producing results, the credibility of the entire programme is questionable. This toolkit represents one mechanism to fill in some of these gaps by providing guidelines and templates for a more systematic approach toward data collection and analysis. 19

24 Basic vs. Preferred Standards for Monitoring One of the challenges of trying to harmonize monitoring in multiple project sites across multiple countries is the tremendous variability in the operational contexts and resources for monitoring. To accommodate the varying levels of staff and funding, and not over-burden offices that are poorly resourced, the monitoring guidelines in this toolkit are classified as BASIC and PREFERRED. The BASIC level of monitoring covers all of the key information needs, including the core outcome indicators, using a minimum amount of time and staff resources. The PREFERRED standards also capture all of the core information needs, but go slightly beyond to get a better understanding of an activity s impact relative to the baseline conditions, a more thorough understanding of cause and effect, and an assessment of the longer-term impact and sustainability issues. All FAO ECUs and their implementing partners are expected to implement a monitoring system which at a minimum meets the BASIC monitoring standards, and as many of the PREFERRED standards as resources permit. Some ECUs may also wish to borrow from the PREFERRED standards to meet reporting obligations under agreements from other funding agencies or joint-programming MOUs. Specific Measurable Attainable Relevant Time-bound Good indicators are SMART A good indicator measures one and only one characteristic or quality, clearly and unambiguously. A common mistake is to try to put too many characteristics into one indicator. For example, if we are told the indicator quantity of diversified crop production has dropped, we wouldn't know if the indicator fell because production was low, or production was normal but it just wasn t diversified. It is important that an indicator be measurable in a consistent manner in different places and at different times, so that results can be compared over time and across sites. If the indicator is very subjective or not well-defined (e.g., the 'effectiveness' of a training program), then it will be difficult to interpret or compare results. It is important that an indicator be measured easily and reliably. If the measurement is very complicated, requires a lot of resources, or is dependent on many factors falling into place, there is a high risk it will not be measured at all. Indicators are supposed to tell us whether we are reaching our goals and objectives, so a good indicator must clearly relate to the condition being measured. We need to know the time period of the measurements to be able to interpret an indicator. The time interval and frequency of measurement should be explicit in both the indicator definition and reporting. 20

25 General Differences Between Basic and Preferred Monitoring Standards BASIC data collection is mostly incorporated into routine operational tasks, such as beneficiary selection and training, thus minimizing separate standalone tasks such as household surveys sample sizes for surveys are kept as small as possible comparison data can be gathered at the same time as impact assessment, for example from memory recall or non-supported production from the same household, instead of a control group only direct beneficiaries are monitored PREFERRED in addition to the collection of data during normal operational tasks, additional focused data collection activities are encouraged larger sample sizes, including some nonbeneficiaries, are used when picking samples for more in-depth data collection (e.g., surveys) participants in focus group discussions should be stratified (e.g., by gender and/or age) to encourage non-biased expression of viewpoints a post-project assessment is encouraged to determine whether results are sustainable Measuring Targeting, Household Vulnerability and HIV/AIDS Impact Like many other food security interventions, one of the primary target groups for backyard and community garden projects are households which are vulnerable to food insecurity or nutrition deficiencies, but are still capable of gardening (i.e., vulnerable but viable). There are numerous factors which can cause households to become food insecure, but one factor which is of particular importance in southern Africa is the HIV/AIDS pandemic. Many of the household level indicators for gardens require results to be disaggregated based on the level of vulnerability in general, and HIV/AIDS in particular, to show whether vulnerable households are being reached and if so whether the intervention is producing benefits appropriate to their needs. This poses three challenges for monitoring gardening projects, 1) how do we identify appropriate households for participation, 2) how do we measure vulnerability to food insecurity, and 3) how do we determine whether a household has been impacted by HIV/AIDS? Participation in a school or community garden is often linked to geographic proximity, and/or current enrolment or membership in a group. However there are many methods for selecting beneficiaries for backyard garden projects. Some targeting methods involve household-level surveys followed by quantitative analysis to determine thresholds of vulnerability. Other methods draw on local knowledge through community-based discussions to identify households which are vulnerable but still have a capacity for raising small animals. Other methods involve self-selection mechanisms (e.g., requiring in-kind contributions of work) to weed out all but the targeted households. Sometimes methods are mixed, for example combining community-based targeting with an objective validation process that randomly selects names from a communitygenerated list and validates whether they meet the stated criteria. All methods have their strengths and weaknesses, and no single method is appropriate for all contexts. In many emergency situations, community-based targeting methods approaches are used due to constraints of time and data. It may not be very practical to record detailed information for every beneficiary selected for a garden project, however it is feasible to ask the groups involved in targeting to at least record the criteria they use in selecting beneficiaries. This has generally not been done in the past, leaving the targeting process a "black box" open to criticism. The Garden Beneficiary Selection Worksheet (page 24) has been designed to make the targeting process more systematic and record the criteria used in targeting. This worksheet is designed to be used at the community 21

26 level and is not meant to be entered into the monitoring database. The PREFERRED monitoring standard however recommends entering a sample of randomly selected worksheets for analysis of the targeting process. Secondly, we need to validate the effectiveness of our targeting method and determine whether targeted households are benefiting from the activity. While it isn't possible to assess the vulnerability of all participants, if we randomly select households for the household survey, our results should be approximately correct for all participants. Measuring the vulnerability of dozens of households is certainly easier than trying to measure hundreds, but this still leaves us with a big methodological challenge: how does one measure vulnerability to food insecurity? The short answer is that it varies from place to place, depending on the type of farming system, soils, access to employment and social services, and so on. The characteristics of a vulnerable household living on rocky soils in the highlands of Lesotho will be quite different than the characteristics of a vulnerable family in the Zambezi valley. The amount of land needed for food production, dependence on draft power, labour requirements, accessibility to fertilizer, opportunities for alternative employment, and many other factors will vary enormously from country to country and even within countries. For any one context, multiple criteria will probably be needed to measure vulnerability to food insecurity. The methodology and results of the national VAC survey may be of some assistance in developing an index of vulnerability. The same is true for determining whether a household has been impacted by AIDS. In some countries, there are village-based AIDS support groups and one can ask direct questions to assess whether a household has been affected by AIDS. In other places, the stigma of AIDS is still quite strong and indirect methods or proxies must be used to avoid offending people or worsening the stigmatization. Each country office will therefore need to select criteria for determining how to measure household level vulnerability to food insecurity. The end result should be a vulnerability index that ranges from 1 to 3, where 1 = low vulnerability, 2 = moderately vulnerable, and 3 = highly vulnerable. The vulnerability score should be based on the responses to a series of simple yes/no questions on the Household Garden Survey (page 35). For example, the ECU in Malawi may decide that a highly vulnerable household is one that has only one working-age adult with five or more children, or has less than one ha of land. The Zambia ECU may decide that a highly vulnerable household in Southern Province is one that has no cattle or draught power, regardless of the family size or land holdings. Similarly each household visited during the baseline and subsequent surveys must be assigned a value of yes or no as to whether the household has been impacted by AIDS. This could be based on whether an adult member of the household has died, someone in the household is chronically ill, or the presence of orphans and vulnerable children (OVCs). Like vulnerability, it will be up to each national ECU to determine how to use the responses on the baseline household survey to identify households impacted by AIDS, with RIACSO providing assistance when needed. Each Emergency Coordination Unit needs to decide how to interpret the responses on the baseline household survey to determine the thresholds for high, moderate, and low levels of vulnerability appropriate for the local livelihood system, and how to determine whether a household has been impacted by AIDS. 22

27 ANNEX II. SAMPLE DATA COLLECTION FORMS The sample data collection forms in this annex include: Applies To Data Processing Basic Preferred 1. Garden Beneficiary Selection Worksheet backyard none sample entered into M&E database 2. Garden Diary backyard, none none community 3. Production and Sales Summary backyard, community none sample entered into M&E database 4. Instructor's Training Log all gardens with a training component none sample entered into M&E database (excluding schools) 5. Garden Seasonal Summary all gardens entered into M&E database entered into M&E database 6. Household Garden Survey backyard, community entered into M&E database entered into M&E database 7. School Garden Survey school entered into M&E entered into M&E database 8. Institutional Garden Survey institutional entered into M&E database Adapting the forms database entered into M&E database These sample data collection templates incorporate all the existing monitoring forms that were submitted for review, but have been designed to be as generic as possible. Some additional adaptation will undoubtedly still be needed to suit the local context. (To modify the forms, soft copies in MS Word format can be obtained from the Regional Agronomist at RIACSO). Minor design changes that won't affect data entry can be made fairly easily, such as the addition of a logo, signature lines, labels, explanatory text, etc. Adding new fields, columns or sections is also possible, however this could also require modification of the data processing system if the additional information needs to be captured. If new fields, columns, or questions are needed, it is recommended that you consult with the RIACSO office and other ECUs using the garden M&E database to see if other users would also like to adopt the new fields. Another type of adaptation needed are the labels for the geographic references. Most countries have administrative areas called provinces and districts, but in some countries administrative areas are known by other names. These labels can be modified on the paper versions of the data collection forms as needed. We already have a monitoring system for our garden project, why do we need to start using these new forms? The short answer is you don't, provided that your current monitoring tools can produce the required numeric summaries for the core outcome indicators outlined in Table 2 (page 4). The data collection forms in this toolkit are provided as sample templates that can be adopted when nothing comparable is currently available. These templates have been designed to be as generic as possible, but can be adapted at the country level to capture additional information as needed. 23

28 Garden Beneficiary Selection Worksheet Instructions The purpose of this form is to record how people are chosen to be a beneficiary for a garden project. This form can be used by any person or group involved in the selection of beneficiaries, including community groups, extension officers, or project staff. To use the form, put a check or 'x' in each column that was used to select the person. Column headings should be changed to match the actual criteria used. Name of beneficiary m/f Location Reason(s) for selection as a beneficiary (check all that apply) Elderly head HH Widow head HH Orphan head HH Recent death in HH No one employed Many children Poor harvest last yr No one to send money No small livestock Sickness Insufficient land No draft power Active in a group or programme Other (specify) Explanation or comments Selection by: Date: 24

29 Garden Diary Section 1: Planting & Harvesting Instructions: Record in your notebook the dates when you planted and harvested vegetables / fruit trees / crops from the garden. Use a separate page for each item you produce. Vegetable / Crop: Vegetable / Crop: Date Planted / Harvested Quantity Problems / Comments Date Planted / Harvested Quantity Problems / Comments

30 Garden Diary Section 2: Profit and Loss Statement Instructions: Record in your notebook the income and costs from your garden. Use a separate page for each month. At the end of each month, calculate the total amount of profit or loss made. Profit and Loss Statement Profit and Loss Statement Month: Month: COSTS Date Expense Amount COSTS Date Expense Amount Total Costs: INCOME Date Source Amount Total Costs: INCOME Date Source Amount Total Income: Net Profit (income costs): Total Income: Net Profit (income costs): 26

31 Garden Diary Section 3: Rainfall (Format 1) Instructions: record in your notebook the number of days it rained each month. You can either use tick-marks or write numbers. January: February: March: April: May: June: July: August: September: October: November: December:

32 Garden Diary Section 3: Rainfall (Format 2) Instructions: record in your notebook the days it rained each month by making a mark in the appropriate box. Month Jan Feb Mar Apr May Jun Aug Sep Oct Nov Dec 28

33 Garden Diary Section 4: Tracking Market Prices Instructions: record in your notebook the prices of different vegetables in the market, in order to know when is the best time to plant and sell. Select only those vegetables that you have an interest in. Market Price Market Price Vegetable Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

34 Production and Sales Summary Instructions. This form is used to summarize the production and sales from a single garden for a period of time (e.g., a month). The information on this summary sheet can come from a Garden Diary or other record keeping system. If the amount harvested or sold is not known, write unknown in the quantity column. This form can be copied into a notebook or photocopied. Garden location: GPS Coordinates: (if known) S: E: Reporting period: From: To: baseline Supported garden: yes no (circle one) Crop or Product Harvested Sold Quantity Quantity Unit Price Comments Prepared by: Name Position Date 30

35 Instructor's Training Log Instructions. This template is designed for a trainer or extension officer to record all training activities that are provided for participants in a garden project. This form can either be photocopied or copied into a notebook. Additional columns can be added as needed. At the end of the season, the information from the training diary should be copied onto the Garden Seasonal Summary Sheet. Garden name or location: District: Trainer: Training Activities Date(s) Training Topic(s) Instructor Males Females Time (hrs) Comments 31

36 Garden Seasonal Summary (for all gardens) Instructions. This three-page seasonal summary sheet should be filled out by garden member(s) or manager at the beginning of the project (baseline) and end of each gardening season. If a garden does not have a manager, an extension officer can fill in the information. There should be one summary sheet for each garden. A copy of the Seasonal Summary Sheet should be forwarded to the office for entry into the database. Garden name / location: District: GPS Coordinates: (if known) S: E: 1. Size of garden: m 2 / ha 2. Type of garden: household / backyard community with individual plots community with group beds school garden Junior Farmer Field and Life School NCP hospital orphanage prison other: Reporting period: From: To: baseline mid-project end of project Supported garden: yes Project: 3. Number of people who grow things in the garden: females males no 4. Age of garden: this is the first season 1-2 years 3-4 years 5 years or more 5. Number of months this garden was in production in the past 12 months: 6. Number of growing cycles (seasons) this garden was used in the past 12 months: 7. Vegetables, crops and tree products grown this period: Item Produced (including vegetables, trees, & crops) Planted Harvest (tick if harvested) (tick if planted) good fair poor 32

37 Water supply & irrigation 8. Area under irrigation: m 2 % of garden 9. Area rainfed only: m 2 % of garden 10. Irrigation methods used: Water Source(s): dam borehole well tap stream open pits tank other: Water Lifting Devices none (gravity fed) bucket elephant pump rope and washer treadle/pedal pump diesel/petrol pump Water Spreading watering can / bucket farrows sprinklers hoses drip kit other: 11. Reliability of water source: water was never a problem water level got low water source dried up 12. Comments on water supply: continue on back if needed Problems encountered 13. Please describes problems encountered pest or insect damage plant diseases weeds livestock damage storm damage 14. Comments on problems encountered: theft lack of tools water supply irr. equipment not available irr. equipment broken inputs not available (specify which): inputs not affordable (specify which): fuel not available fuel not affordable other: continue on back if needed Gardening methods demonstrated 15. Which methods were used or demonstrated in the garden? use of plants for biocontrol live fencing mulching composting kraal manure micro-dosing fertilizer agro-forestry trench gardening crop rotation keyhole gardening other: other: Physical Infrastructure Improvements 16. Please comment on any improvements to fixed infrastructure made this season to the garden site during the reporting period (e.g., fencing, wells, tap, toilet, etc.) continue on back if needed 33

38 Summary of Training Activities Held at Garden Training Topic Month Comments 17. Comments on training activities: continue on back if needed Recommendations 18. Please write any other comments and recommendations below: continue on back if needed Prepared by: Name Position Date 34

39 Household Garden Survey (for community and backyard gardens) Introduction. This survey is designed to learn about how people practice gardening so we can improve gardening projects. This is not an exam, and there are no right or wrong answers. Your responses will be used to prepare a report, but neither your, nor any other names, will be mentioned in any report and all the responses will be mixed together so there will be no way to identify that you gave this information. We appreciate your participation in this survey, but participation is completely voluntary and optional. Do you have time to participate? Name: m / f GPS coordinates: S: Project: beneficiary non-beneficiary E: [District]: [Village]: PART I. BASELINE VISIT Date of visit: Monitoring officer: Homestead Details 1. Are you the head of household? yes no 2. People living in homestead Children (0-17) Adults (18-59) Elderly (60+) for the last three months: M F M F M F 3. Is the head of the homestead: elderly widow orphan chronically ill none of these 4. Number of orphans living in homestead (aged 0-17): orphan: a child 17 years old or less with at least one parent deceased chronically ill: sick and unable to work for a total of 3 months over the last 12 months 5. How do you prepare your fields? by hand animal power tractor 6. Please a) list, and then b) rank according to importance, your household sources of income in the last six months. Income Source Rank * * Rank: 1=most important, 2=next important, etc. 7. Have any of the adult members of the household been sick and unable to work for a total of 3 months over the last 12 months? yes no 8. Have any of the non-elderly adult members of the household died in the 12 months? yes no 9. Is anyone in this household currently receiving home-based care, ARVs, or active yes no in an HIV support group? 35

40 BASELINE VISIT (cont'd) 10. List all forms of food or cash assistance received in the last six months, including school feeding programmes and home based care: Experience with Gardening 11. Do you currently have a household garden? yes no If yes: 12. Which specific vegetables, crops and other items do you grow? 13. In the past 12 months, in which months did you harvest from your garden? Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec (circle all that apply) 14. Were you involved with a garden project last season? no yes, specify: Participation in the Garden Project (non-beneficiaries can skip this section) The following questions all refer to the project garden (backyard or community) 15. What is the distance to the garden? minutes km 16. How many hours per week do you spend in the garden? hours 17. Do you have your own individual plot or garden within the project garden? yes no 18. How big is your garden plot: m Which vegetables and crops will you / have you plant? 20. What are your expectations of the garden project? 21. What training topics do you feel you need, if any? 22. Would you be interested in keeping a garden diary where you record things like planting, harvesting, and sales? yes no Dietary Intake 23. Over the last seven days, on how many days did you consume the following foods? maize, maize porridge: other cereal (e.g. rice, sorghum, millet): cassava: potatoes, sweet potatoes: sugar or sugar products: beans and peas: groundnuts and cashew nuts: vegetables/ relish /leaves: bread, pasta: fruits: beef, goat, or other red meat: poultry: pork: eggs: fish: oils/fats/butter: milk / yogurt / other dairy fortified blended food (e.g. CSB, HEPS): Form reviewed by: Approved by: 36

41 Household Survey (cont'd) Beneficiary: PART II. POST-HARVEST / END OF PROJECT VISIT Date of visit: Reporting period (season): Monitoring officer: Garden Experiences This Season 24. How many gardens did you use for growing this season? 25. Which vegetables did you harvest this season? 26. Which other products did you harvest this season (e.g., fruit, seed, herbs, livestock)? 27. For each of the items you produced in your garden, please describe how you used it (sale, consumption, gift). Also please describe any problems experienced (e.g., production, storage, marketing) Vegetable / Crop Harvested Used How 1 Problems 2 1 used how: 1=consumed, 2=sold, 3=given away, 4=stored, 5=other (specify) multiple answers OK 2 problem(s): 1=poor germination, 2=weeds, 3=insufficient watering, 4=no fertilizer, 5=too much rain, 6=harvest problems (specify), 7=storage problems (specify), 8=marketing problems (specify), 8=other (specify) multiple answers OK 28. If yes, in the past 12 months, in which months did you harvest vegetables from your garden? Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec (circle all that apply) Participation in the Garden Project (non-beneficiaries can skip this section) 29. The project garden was: the only garden in the household primary garden of multiple gardens a secondary garden 30. Did you receive any training as part of the garden project? yes no 31. If yes, please fill specify the topics covered: Topic Training provided by Month Comments 32. Which inputs, tools, and other items did you receive from the project? 37

42 33. Did you keep a diary or other record keeping system of your harvest and sales? yes no 34. If yes, please summarize the total amounts harvested and sold: Vegetable / Crop Amount Harvested * Amount Sold * Income Generated * include the appropriate units (heads, kg, buckets, etc.) 35. Please a) list, and then b) rank according to importance, your household sources of income in the last six months. Income Source Rank * * Rank: 1=most important, 2=next important, etc. 36. If you earned any income from your garden in the last 30 days, what did you do with the money? check all that apply buy food clothes pay off debts school fees give away buy seed or fertilizer household items other: 37. Describe any other benefits from the project: 38. Do you have any recommendations for future garden projects? Dietary Intake 39. Over the last seven days, on how many days did you consume the following foods? maize, maize porridge: fruits: other cereal (e.g. rice, sorghum, millet): beef, goat, or other red meat: cassava: poultry: potatoes, sweet potatoes: pork: sugar or sugar products: eggs: beans and peas: fish: groundnuts and cashew nuts: oils/fats/butter: vegetables/ relish /leaves: milk / yogurt / other dairy bread, pasta: fortified blended food (e.g. CSB, HEPS): 38

43 School Garden Survey School: GPS coordinates: S: Monitoring officer: E: Date of visit: [District]: Supported garden: yes no Project: 1. Number of students in school: Grade Total number of students Total num of OVCs Work in Garden Eat from Garden Take Veggies Home female male female male female male female male female male 2. Do students work in the garden? yes no 3. If yes, for how many hours a week and is the work part of a class? Grade % who learn in garden Agric Science Hours per Week Home Economics Other: Other:

44 4. Do you have a feeding programming in this school? yes no If yes, then: 5. How many meals per day? 6. In the last seven days, how many days did you serve: maize, maize porridge: other cereal (e.g. rice, sorghum, millet): cassava: potatoes, sweet potatoes: sugar or sugar products: beans and peas: groundnuts and cashew nuts: vegetables/ relish /leaves: bread, pasta: fruits: beef, goat, or other red meat: poultry: pork: eggs: fish: oils/fats/butter: milk / yogurt / other dairy fortified blended food (e.g. CSB, HEPS): 7. Are any of the items grown in the garden sold? yes no If yes, what is the money used for? 8. Do students get to take items grown in the garden home? yes no If yes, which students? 40

45 Institutional Garden Survey (including hospital, orphanages, Neighborhood Care Points, etc.) Site: GPS coordinates: S: Monitoring officer: E: Date of visit: [District]: Supported garden: yes no Project: 1. Number of people who eat from this garden: Age Range male female 0-5 yrs 6-10 yrs yrs yrs 20+ yrs 2. How many meals per day are prepared from this garden? 3. In the last seven days, how many days did you serve: maize, maize porridge: other cereal (e.g. rice, sorghum, millet): cassava: potatoes, sweet potatoes: sugar or sugar products: beans and peas: groundnuts and cashew nuts: vegetables/ relish /leaves: bread, pasta: fruits: beef, goat, or other red meat: poultry: pork: eggs: fish: oils/fats/butter: milk / yogurt / other dairy fortified blended food (e.g. CSB, HEPS): 4. Where do you get technical support? 5. List any inputs received from the project: Item Quantity Nutritional Rehabilitation Units (garden for patient families at a hospital) 6. How long has the patient been hospitalized? days 7. How many times has the patient been admitted in the past year? 8. Distance from home to hospital? NCP Garden (Neighbor Care Point) 9. Distance to nearest NCP? km 41