RAIN PROJECT: IMPACT EVALUATION REPORT

Transcription

1 ealigning griculture to mprove utrition RAIN PROJECT: IMPACT EVALUATION REPORT An impact evaluation report prepared by the International Food Policy Research Institute of the Realigning to Improve Nutrition (RAIN) project in Zambia May 2016 Jody Harris, Phuong Hong Nguyen, John Maluccio, Adam Rosenberg, Lan Tran Mai, Wahid Quabili, Rahul Rawat

2 Acknowledgements A large community-based study like this could not be completed without the coordinated efforts of many people. The International Food Policy Research Institute (IFPRI) team thanks all those who facilitated every step of this process: Concern Worldwide staff in Dublin, particularly the Strategy Advocacy and Learning (SAL) advisors and the Zambia liaisons within the International Programmes Department, for their support and commitment to the RAIN project and its evaluation. Concern Worldwide staff in Zambia for their constant support for our work. In particular, Gudrun Stallkamp who provided critical overall support to this evaluation, and who provided inputs in selecting sites for the evaluation, commented on several pieces of this survey including the evaluation design, ethics approval application, questionnaire revisions, and for connecting the evaluation team with other key personnel at Concern Worldwide; this survey could not have been completed with her support Mumbwa Child Development Agency (MCDA) for their support in implementing the project Palm Associates for facilitation of administrative and logistical support to the survey fieldwork. The Government of Zambia health and agriculture officials in Mumbwa District and Central Province who provided support to this survey and to the RAIN project The National Food and Nutrition Commission for loaning the survey team anthropometric equipment, and for invaluable support to anthropometric training of enumerators The entire survey team from Palm Associates from both survey rounds, including the research assistants, field supervisors, fieldworkers, and data entry and management team Finally, to the women and children that belong to the households who participated in the surveys, we thank you for your role in making this a success. Your contributions of time and information, and your dreams for a more healthy future for your children are at the very heart of this project. The RAIN project was funded by Irish Aid and the Kerry Group with additional support from the Bank of Ireland. Funding for the evaluation was provided by Concern Worldwide, through grants received from Irish Aid, Kerry Group, and PATH through support provided by the UK Government s Department for International Development. Additional support for the evaluation was provided by the CGIAR research program on for Nutrition and Health, led by IFPRI. The views expressed herein are those of the authors and in no way can be taken to reflect the official opinions of the funding organisations.

3 Acronyms BMI CSA CSO CWW DHS DPT ENA FANTA HAZ HDDS HHS IFPRI IYCF NFNC NGO OPV RAIN SEA WAZ WHO WHZ Body Mass Index Census Supervisory Area Central Statistics Office Concern Worldwide Demographic and Health Survey Diphtheria/Polio/Tetanus Essential Nutrition Actions Food and Nutrition Technical Assistance Height-for-age z-score Household Dietary Diversity Score Household Hunger Scale International Food Policy Research Institute Infant and Young Child Feeding National Food and Nutrition Commission Non-governmental Organization Oral Polio vaccine Realigning to Improve Nutrition Supervisory Enumeration Area Weight-for-age z-score World Health Organization Weight-for-height z-score

4 Table of contents EXECUTIVE SUMMARY IX 1 INTRODUCTION OVERVIEW OF THE NUTRITION SITUATION IN ZAMBIA DESCRIPTION OF THE RAIN PROJECT EVALUATION OBJECTIVES STRUCTURE OF THE REPORT 23 2 METHODS EVALUATION DESIGN RATIONALE FOR AGE RANGE TO DETECT IMPACTS ON STUNTING RANDOMIZATION PROCESS SAMPLE SIZE ESTIMATE SAMPLING METHODOLOGY SURVEY INSTRUMENTS CONCEPTUAL BASIS FOR THE IMPACT EVALUATION QUESTIONNAIRE HOUSEHOLD QUESTIONNAIRE ANTHROPOMETRIC MEASUREMENT DATA COLLECTION AND MANAGEMENT TRAINING OF PERSONNEL SURVEY TEAM COMPOSITION DATA MANAGEMENT SURVEY INFORMATION DATA ANALYSIS EXPOSURE TO THE PROGRAM IMPACT ESTIMATES ON MAIN AND SECONDARY OUTCOMES DECOMPOSITION ANALYSIS ETHICAL APPROVAL 37 3 RESULTS: BASELINE CHARACTERISTICS KEY BASELINE INDICATORS 39 4 RESULTS: RAIN INTERVENTION EXPOSURE KEY PROGRAM EXPOSURE INDICATORS COMMUNITY EXPOSURE TO NON-RAIN PROGRAMS 45

5 5 RESULTS: IMPACT OF RAIN INTERVENTIONS ON ANTHROPOMETRIC OUTCOMES MAIN IMPACT ANALYSIS OF RAIN INTERVENTIONS ON ANTHROPOMETRIC OUTCOMES INTENT-TO-TREAT OUTCOMES FOR ANTHROPOMETRIC OUTCOMES AMONG CHILDREN MONTHS OF AGE INTENT-TO-TREAT OUTCOMES FOR ANTHROPOMETRIC OUTCOMES AMONG CHILDREN MONTHS OF AGE PLAUSIBILITY ANALYSIS 1: HIGH POTENTIAL FOR EXPOSURE AGE GROUPS ( MONTHS) PLAUSIBILITY ANALYSIS 2: DOSE-RESPONSE RELATIONSHIP BETWEEN PROGRAM EXPOSURE CHILD NUTRITION PLAUSIBILITY ANALYSIS 3: CHANGES IN THE UNDERLYING DETERMINANTS OF CHILD GROWTH AND NUTRITION CHILD CHARACTERISTICS MATERNAL CHARACTERISTICS: DEMOGRAPHIC, HEALTH SEEKING BEHAVIOR, AND NUTRITIONAL STATUS HOUSEHOLD CHARACTERISTICS: FOOD SECURITY, DIETARY DIVERSITY, SOCIOECONOMIC STATUS, AND ACCESS TO SERVICES 72 6 RESULTS: IMPACT OF RAIN INTERVENTIONS ON IYCF PRACTICES MAIN IMPACT ANALYSIS OF RAIN INTERVENTIONS ON IYCF OUTCOMES PLAUSIBILITY ANALYSIS: DOSE-RESPONSE RELATIONSHIP BETWEEN PROGRAM EXPOSURES AND IYCF PRACTICES 86 7 RESULTS: IMPACT OF RAIN INTERVENTIONS ON KNOWLEDGE AMONG CAREGIVERS IMPACT RAIN PACKAGE OF INTERVENTION ON NUTRITION AND HYGIENE KNOWLEDGE 89 8 RESULTS: IMPACTS OF RAIN INTERVENTIONS ON THE WOMEN S EMPOWERMENT WOMEN S SOCIAL EMPOWERMENT WOMEN S ECONOMIC EMPOWERMENT WOMEN S EMPOWERMENT IN AGRICULTURE AT ENDLINE RESULTS: IMPACT OF RAIN INTERVENTIONS ON PRODUCTION OF NUTRIENT-RICH FOODS FOOD PRODUCTION RESULTS: DECOMPOSITION ANALYSIS OF THE DETERMINANTS OF CHANGES IN CHILD GROWTH OUTCOMES OVER TIME DETERMINANTS OF CHILD GROWTH DISCUSSION 134 REFERENCES 139

6 12 APPENDICES 140 APPENDIX 1: DETAILED SAMPLE SIZE CALCULATIONS 140 APPENDIX 2: VARIABLES INCLUDED IN THE DECOMPOSITION ANALYSIS 145 APPENDIX 3: DETAIL ON THE RAIN PROJECT 147 APPENDIX 4: COMPARING RAIN AND DHS 150 APPENDIX 5: EXPLAINING THE IMPACT OF RAIN: PROGRAM IMPACT PATHWAYS 151 List of tables TABLE SAMPLE SIZES TABLE SAMPLE SIZES USED FOR ANALYSES TABLE 2.4.1: ANALYSIS METHODS- USE AND INTERPRETATION TABLE BASELINE CORE IMPACT INDICATORS BY PROGRAM GROUP TABLE SELECTED BASELINE MATERNAL UNDERLYING FACTORS BY PROGRAM GROUP TABLE SELECTED BASELINE HOUSEHOLD UNDERLYING FACTORS BY PROGRAM GROUP TABLE PROGRAM EXPOSURE: PARTICIPATION AND DELIVERY TABLE PROGRAM PARTICIPATION TABLE PROGRAM PARTICIPATION (AMONG MOTHERS WITH CHILDREN <2 YEARS OLD) TABLE SMF AND CHV INTERACTION TABLE SMF AND CHV INTERACTION (AMONG MOTHERS WITH CHILDREN <2 YEARS OLD) TABLE EXPOSURE TO BROADER RAIN PROJECT EVENTS AND MATERIALS TABLE EXPOSURE TO BROADER RAIN PROJECT EVENTS AND MATERIALS (AMONG MOTHERS WITH CHILDREN <2 YEARS OLD) TABLE NGO PRESENCE, BY PROGRAM GROUP TABLE SOCIAL CAPITAL CLUBS SUPPORTED BY NGOS, BY PROGRAM GROUP TABLE INCOME GENERATING ACTIVITIES SUPPORTED BY NGOS, BY PROGRAM GROUP TABLE TRAINING ACTIVITIES SUPPORTED BY NGOS, BY PROGRAM GROUP TABLE ANTHROPOMETRIC INDICATORS AMONG CHILDREN MONTHS OF AGE, BY PROGRAM GROUP AND SURVEY ROUND. 49 TABLE ANTHROPOMETRIC INDICATORS AMONG CHILDREN MONTHS OF AGE BY PROGRAM GROUP AND SURVEY ROUND TABLE ANTHROPOMETRIC INDICATORS AMONG CHILDREN MONTHS OF AGE BY PROGRAM GROUP AND SURVEY ROUND.. 54 TABLE ANTHROPOMETRIC INDICATORS AMONG BENEFICIARIES WITH CHILDREN MONTHS, BY PROGRAM GROUP AND SURVEY ROUND TABLE ANTHROPOMETRIC INDICATORS AMONG BENEFICERIES WITH CHILDREN MONTHS OF AGE BY PROGRAM GROUP AND SURVEY ROUND TABLE ASSOCIATION BETWEEN PROGRAM EXPOSURE WITH CHILD STUNTING AMONG CHILDREN M TABLE CHILD IMMUNIZATION AND SUPPLEMENTATION STATUS AMONG CHILDREN MONTHS, BY PROGRAM GROUP AND SURVEY ROUND TABLE CHILD IMMUNIZATION AND SUPPLEMENTATION STATUS AMONG CHILDREN MONTHS, BY PROGRAM GROUP AND SURVEY ROUND TABLE CHILD MORBIDITY, BY PROGRAM GROUP AND SURVEY ROUND TABLE SELECTED BASELINE MATERNAL UNDERLYING FACTORS, BY PROGRAM GROUP AND SURVEY ROUND TABLE TIME ALLOCATION, BY PROGRAM GROUP TABLE TIME ALLOCATION AMONG BENEFICIARIES, BY PROGRAM GROUP TABLE USE OF PRENATAL CARE, BY PROGRAM GROUP AND SURVEY ROUND TABLE MATERNAL DIETARY DIVERSITY, BY PROGRAM GROUP AND SURVEY ROUND (< 24 MO) TABLE MATERNAL DIETARY DIVERSITY AMONG BENEFICIARIES, BY PROGRAM GROUP AND SURVEY ROUND TABLE WOMEN S NUTRITIONAL STATUS, BY PROGRAM GROUP AND SURVEY ROUND... 71

7 TABLE WOMEN S NUTRITIONAL STATUS AMONG BENEFICIARIES, BY PROGRAM GROUP AND SURVEY ROUND TABLE HOUSEHOLD PERCEPTIONS OF FOOD INSECURITY, BY PROGRAM GROUP AND SURVEY ROUND TABLE HOUSEHOLD PERCEPTIONS OF FOOD INSECURITY AMONG BENEFICIARIES, BY PROGRAM GROUP AND SURVEY ROUND TABLE HOUSEHOLD DIETARY DIVERSITY, BY PROGRAM GROUP AND SURVEY ROUND TABLE HOUSEHOLD DIETARY DIVERSITY AMONG BENEFICIARIES, BY PROGRAM GROUP AND SURVEY ROUND TABLE HOUSEHOLD ASSET OWNERSHIP, BY PROGRAM GROUP AND SURVEY ROUND TABLE HOUSEHOLD ACCESS TO SERVICES, BY PROGRAM GROUP AND SURVEY ROUND TABLE HOUSEHOLD SOCIOECONOMIC STATUS, BY PROGRAM GROUP AND SURVEY ROUND TABLE WHO RECOMMENDED IYCF INDICATORS, BY PROGRAM GROUP AND SURVEY ROUND TABLE DIFFERENCE IN DIFFERENCE OF WHO RECOMMENDED IYCF INDICATORS, BY PROGRAM GROUP AND SURVEY ROUND TABLE PRE-LACTEAL FEEDING AMONG CHILDREN 0-23 MONTHS OLD, BY PROGRAM GROUP AND SURVEY ROUND TABLE FOOD GROUPS CONSUMED IN THE PAST 24 HOURS AMONG CHILDREN 6-23 MONTHS OLD, BY PROGRAM GROUP AND SURVEY ROUND TABLE TIMELINESS OF INTRODUCTION OF COMPLEMENTARY FOODS AMONG CHILDREN 6-23 MONTHS OLD, BY PROGRAM GROUP AND SURVEY ROUND TABLE REPORTED MEAL FREQUENCY AMONG CHILDREN 6-23 MONTHS OLD, BY PROGRAM GROUP AND SURVEY ROUND TABLE WHO RECOMMENDED IYCF INDICATORS AMONG BENEFICIARIES (THOSE ARE THE MEMBER OF A RAIN S WOMEN S GROUP) BY PROGRAM GROUP AND SURVEY ROUND TABLE DIFFERENCE IN DIFFERENCE OF WHO RECOMMENDED IYCF INDICATORS AMONG BENEFICIARIES (THOSE ARE THE MEMBER OF A RAIN S WOMEN S GROUP) TABLE KNOWLEDGE ABOUT BF AMONG MOTHERS OF CHILDREN MONTHS OF AGE, BY PROGRAM AREA TABLE KNOWLEDGE ABOUT BF AMONG MOTHERS WHO ARE MEMBERS OF RAIN GROUP WITH CHILDREN MONTHS OF AGE, BY PROGRAM AND SURVEY ROUND TABLE REPORTED KNOWLEDGE ON TIMELINESS OF INTRODUCTION OF COMPLEMENTARY FOODS AMONG MOTHERS WITH CHILDREN MONTHS, BY PROGRAM GROUP AND SURVEY ROUND TABLE REPORTED KNOWLEDGE ON FEEDING DURING ILLNESS AMONG MOTHERS WITH CHILDREN MONTHS, BY PROGRAM GROUP AND SURVEY ROUND TABLE REPORTED EXPOSURE TO INFORMATION, BY PROGRAM GROUP TABLE REPORTED KNOWLEDGE OF HYGIENE PRACTICES, BY PROGRAM GROUP TABLE RELATIONSHIP WITH SPOUSES, BY PROGRAM GROUP AND SURVEY ROUND TABLE PERCEPTION OF EQUALITY, BY PROGRAM GROUP AND SURVEY ROUND TABLE DECISION MAKING POWER, BY PROGRAM GROUP AND SURVEY ROUND TABLE WOMEN S SOCIAL CAPITAL, BY PROGRAM GROUP AND SURVEY ROUND TABLE FINANCIAL EMPOWERMENT, BY PROGRAM GROUP AND SURVEY ROUND TABLE ACCESS TO ASSETS AND ABILITY TO SELL ASSETS, BY PROGRAM GROUP AND SURVEY ROUND TABLE PURCHASING DECISIONS, BY PROGRAM GROUP AND SURVEY ROUND TABLE SUMMARY KEY DOMAINS OF WOMEN EMPOWERMENT (ALL WOMEN), BY PROGRAM GROUP AND SURVEY ROUND TABLE SUMMARY KEY DOMAINS OF WOMEN EMPOWERMENT (AMONG RAIN BENEFICERIES), BY PROGRAM GROUP AND SURVEY ROUND TABLE WOMEN'S EMPOWERMENT IN AGRICULTURE, BY PROGRAM GROUP AND SURVEY ROUND TABLE WOMEN'S EMPOWERMENT IN AGRICULTURE AMONG RAIN BENEFICIARIES, BY PROGRAM GROUP AND SURVEY ROUND TABLE NUMBER OF FIELD CROPS CULTIVATED, BY PROGRAM GROUP AND SURVEY ROUND TABLE NUMBER OF VEGETABLES OR FRUITS CULTIVATED, BY STUDY ARM AND SURVEY ROUND TABLE REARING ANIMALS AND PRODUCTION OF ANIMAL SOURCE FOODS, BY PROGRAM GROUP AND SURVEY ROUND TABLE PRODUCTION OF SEVEN DIFFERENT FOOD GROUPS, BY PROGRAM GROUP AND SURVEY ROUND TABLE DIFFERENCE IN DIFFERENCE OF FOOD PRODUCTION INDICATORS, BY PROGRAM GROUP AND SURVEY ROUND TABLE DIFFERENCE IN DIFFERENCE OF FOOD PRODUCTION INDICATORS IN HOUSEHOLDS PARTICIPATING IN RAIN, BY PROGRAM GROUP AND SURVEY ROUND TABLE CHANGES IN THE MEANS OR PERCENT OF OUTCOMES BETWEEN 2011 AND 2015 IN ZAMBIA

8 TABLE CHANGES IN THE MEANS OR PERCENT OF KEY VARIABLES IN ZAMBIA (SAMPLES OF CHILDREN MONTHS) TABLE HEIGHT-FOR-AGE Z REGRESSIONS BY ROUND WITH TEST FOR COEFFICIENT DIFFERENCES OVER TIME AMONG CHILDREN MONTHS TABLE STUNTING REGRESSIONS BY ROUND WITH TEST FOR COEFFICIENT DIFFERENCES OVER TIME AMONG CHILDREN MONTHS IN ZAMBIA TABLE WEIGHT-FOR-HIGHT Z REGRESSIONS BY ROUND WITH TEST FOR COEFFICIENT DIFFERENCES OVER TIME AMONG CHILDREN MONTHS TABLE WASTING REGRESSIONS BY ROUND WITH TEST FOR COEFFICIENT DIFFERENCES OVER TIME AMONG CHILDREN MONTHS TABLE DECOMPOSING SOURCES OF HAZ AND STUNTING CHANGE BY SURVEY ROUND AMONG CHILDREN MONTHS IN ZAMBIA TABLE DECOMPOSING SOURCES OF WHZ AND WASTING CHANGE BY SURVEY ROUND AMONG CHILDREN MONTHS IN ZAMBIA TABLE DECOMPOSING SOURCES OF HAZ AND STUNTING CHANGE BY SURVEY ROUND AMONG CHILDREN MONTHS IN ZAMBIA TABLE DECOMPOSING SOURCES OF WHZ AND WASTING CHANGE BY SURVEY ROUND AMONG CHILDREN MONTHS IN ZAMBIA TABLE DEFINITIONS OF VARIABLES USING IN THE ANALYSES TABLE 1 PROCESS EVALUATION SAMPLING List of figures FIGURE 2.1. RAIN EVALUATION DESIGN FIGURE 2.2. MAP OF RANDOMIZED INTERVENTION AREAS (CONTROL NOT SHOWN) FIGURE 5.1. PREVALENCE OF STUNTING, UNDERWEIGHT AND WASTING AMONG CHILDREN MONTHS BY PROGRAM GROUP AND SURVEY ROUND FIGURE 5.2. PREVALENCE OF STUNTING, UNDERWEIGHT AND WASTING AMONG CHILDREN MONTHS BY PROGRAM GROUP AND SURVEY ROUND FIGURE 5.3. PREVALENCE OF STUNTING, UNDERWEIGHT AND WASTING AMONG CHILDREN MONTHS BY PROGRAM GROUP AND SURVEY ROUND FIGURE 5.4. PREVALENCE OF STUNTING, UNDERWEIGHT AND WASTING AMONG BENEFICERIES WITH CHILDREN MONTHS BY PROGRAM GROUP AND SURVEY ROUND FIGURE 10.1 HAZ BY CHILD AGE, RAIN PROGRAM AREAS 2011 AND FIGURE 10.2 WHZ BY CHILD AGE, RAIN PROGRAM AREAS 2011 AND FIGURE 10.3 TRENDS IN STUNTING AND WASTING PREVALENCE, BY CHILD AGE AND SURVEY TIME IN ZAMBIA FIGURE 10.4 DISTRIBUTIONS OF CHILD HAZ SCORES AMONG CHILDREN MONTHS IN ZAMBIA BY SURVEY ROUND FIGURE 10.5 DISTRIBUTIONS OF CHILD WHZ SCORES AMONG CHILDREN MONTHS IN ZAMBIA BY SURVEY ROUND FIGURE 12.1 COMPARISON OF STUNTING AND WASTING BETWEEN RAIN AND DHS FIGURE 12.2: RAIN PROGRAM IMPACT PATHWAY (PIP) ACHIEVEMENT

9 Executive summary RAIN project model Almost half of children in Zambia are stunted, and reduction of chronic malnutrition is prioritized in nutrition policies and programs in the country. The Realigning to Improve Nutrition (RAIN) project was a partnership between Concern Worldwide (CWW), Mumbwa Child Development Agency (MCDA) and the International Food Policy Research Institute (IFPRI), aiming to design, implement and evaluate a model of multi-sectoral integration of interventions to reduce the prevalence of chronic malnutrition in Mumbwa district, in the Central Province of Zambia. A key component of the RAIN project was to document evidence of both impact and process for application in other contexts and at scale, through a rigorous evaluation design. Project partners over the course of the 5-year project have included the Mumbwa District Child Development Agency (MCDA), Women for Change (WFC), the Zambian Ministry of and Livestock (MAL), the Zambian Ministry of Health (MoH), and the Zambian Ministry of Community Development Maternal and Child Health (MCDMCH). The project targeted children during the critical period from conception through 24 months of age, roughly equivalent to the first 1,000 days of life, through integrated agriculture, nutrition and health community based interventions. The overall approach focused on addressing the multi-sectoral causes of malnutrition and on learning how to effectively tackle the challenges of inter-sectoral collaboration. The RAIN project comprised of a district-level agriculture intervention to increase year round availability of, and access to, nutrient rich foods at the household level, in some areas accompanied by promotion of optimal health, nutrition, and care seeking behavior through the delivery of social behavior change communication. Most program elements were delivered through local women s groups created by the program, led by a female Smallholder Model Farmer (SMF) nominated by her group to receive agricultural training and inputs and pass these on to the group during monthly meetings. In the nutrition and health areas, groups were also linked to an existing Community Health Volunteer (CHV) who received additional training in nutrition topics to pass on to the group. In addition, some communitywide gender sensitization and information activities were undertaken. The RAIN project and its impact evaluation were designed to help address a critical gap in the evidence base regarding the degree to which agricultural interventions, either alone or when combined with nutrition and health interventions, can improve child nutrition, and ultimately reduce the prevalence of

10 stunting in young children. It aimed to establish proof of concept for an intervention model that can then be replicated and scaled up within Zambia and beyond. RAIN evaluation Objectives The primary evaluation objective of this evaluation was to: 1) Assess the impact of the two different RAIN intervention packages on stunting, among children months of age. In addition to the primary evaluation objective, the secondary evaluation objectives were to assess the impact of the RAIN package of interventions on: 2) Core WHO infant and young child feeding (IYCF) indicators among children 0-23 months of age 3) Health and nutrition knowledge among caregivers of young children 4) Different domains of women s empowerment 5) Agricultural production, and in particular, the availability of, and access to, a year-round supply of diverse and nutritious foods Evaluation Design A fully randomized evaluation design to evaluate the impact of RAIN was determined to be not feasible, primarily for practical project implementation reasons, and therefore a hybrid design was adopted that combines a cluster randomized probability design comparing the two RAIN intervention packages, with a plausibility design that compares the RAIN intervention arms to a non-randomized control group. Randomization was carried out at the level of the census supervisory area (CSA), a sampling unit used by the Central Statistics Office of Zambia. This design yielded 3 different study arms: 1. The only (Ag-only) group, which included agricultural interventions implemented by Concern Worldwide and its partners 2. The -Nutrition (Ag-Nutrition) group, which included both agriculture and nutrition/ health interventions implemented by Concern Worldwide and its partners 3. The Control group, which had access to standard government agriculture and health services, and where Concern Worldwide carried out no implementation activities x

11 Sample size estimates were powered to detect a reduction in the prevalence of stunting (the primary impact indicator) of 8 percentage points, and a 0.2 Z-score difference in the mean height-for-age Z-score (HAZ) between any two study groups. The study was also powered to detect appreciable impacts in the secondary impact indicators. At baseline, we sampled approximately 1000 households per arm with a child aged months of age, for a total sample size of 3044 households. At endline, we oversampled the two RAIN interventions arms, by approximately 20% (for an approximately sample size of 1200 households per arm), to account for potential limited intervention exposure at the household level, for a total sample size of 3536 households. Repeated cross-sectional baseline (2011) and endline (2015) surveys were conducted in the same communities over time, at the same time of the year (July-August). Data collection included a household questionnaire and anthropometric measurements, collecting information on nutrition outcomes and determinants of malnutrition at the child, maternal and household level. Evaluation Analyses Three broad sets of analyses were conducted: 1) Estimation of impact of the RAIN intervention arms (Ag-only, and Ag-Nutrition), compared to the control, and to each other, on the primary impact indicator i.e. stunting. This included: a. Estimation of main impact of RAIN interventions on prevalence of stunting, and mean HAZ scores, using difference-in-difference (DID) estimates b. A series of additional analyses to assess the plausibility of identified impacts. These additional analyses included the following: Plausibility analysis 1: Analysis of the change in stunting prevalence among children in high potential-for-impact age group i.e. among children months of age Plausibility analysis 2: Dose-response analysis between program exposure and child growth outcomes, thereby creating an internal comparison group to test program effects with greater degree of confidence. Plausibility analysis 3: Analysis of change in determinants of stunting over time. 2) Estimation of impact of RAIN interventions on secondary outcomes: Infant and young child feeding practices; maternal nutrition and health knowledge; women s empowerment; and agricultural production. xi

12 3) Decomposition analysis to examine various social, behavioral and economic factors as potential drivers of change in linear growth and stunting over time. All major impact analyses were conducted using: 1) intent-to-treat analysis, where all sampled households are included in the analysis, regardless of whether they were actually exposed to the RAIN project; as well as by 2) per-protocol analysis, where analysis is restricted to those households confirmed to be RAIN project beneficiaries in the two intervention arms. Results Baseline randomization At baseline, there were no statistically significant differences among the three study arms on anthropometric outcomes (HAZ, WHZ, stunting and wasting). Most IYCF indicators were comparable across the three study groups; the exceptions were for early initiation of breastfeeding and meal frequency, which were lower in the Ag-Nutrition group. Additionally, maternal and household characteristics, were also comparable among the three groups at baseline, with no major statistically significant difference between groups. These results (presented in detail in the RAIN baseline report) indicate that randomization at baseline was successful. Household exposure to RAIN interventions Key to interpreting the impact results is understanding intended exposure to the project interventions. This comprises participation (whether and to what extent a household was participating in RAIN project components) and delivery (RAIN implementation being received as planned). Overall participation was 31 percent in the Ag-only group, and 34 percent in the Ag-nutrition group. Of this third of all eligible households participating, the intensity of program delivery varied, with approximately 50% of all households receiving medium or high levels of program delivery. There was no difference in reported participation when restricting the exposure analyses to households with a child < 2 years of age, when sampled at endline. In terms of intensity of delivery, SMF attendance at RAIN groups was high (approximately 90%), but CHV attendance was low (38-45%); group members therefore had more opportunity to interact with trainers from the agriculture side than from the health side. The additional home visits, aimed at providing one-to-one support for gardening and IYCF counselling and support, were more limited still; SMF home visits were not happening as often as planned (45%, and 53% of households were visited by an SMF in 2015, in the Ag-Nutrition and Ag-only arms, respectively), and CHV visits even less often (13% in the Ag - Nutrition arm). In terms of spillover between study arms, leakage xii

13 of CHV participation to the Ag-only group was fairly common as some clinics where RAIN-trained CHVs were based were sited on the border between project areas, but leakage of the project to control areas was not common, with very few women in this area reporting participation. The overall pattern and magnitude of these results did not differ between all households sampled, and households with a child < 2 years of age, when sampled at endline. There was no difference between intervention and control areas in presence of other non-governmental organization (NGO)-supported projects or activities. We can therefore conclude that the control group was functioning as a valid comparator to the intervention arms. We do not have data on the presence of government services in these areas, although there is no reason to believe that there was any difference in government services across study groups. PROGRAM IMPACTS: I. Nutritional status Over time, between baseline and endline, the prevalence of stunting decreased significantly in all three study groups in the impact evaluation age range of children months of age. However, there was a differential decline in stunting in the three study groups, in favor of the control group, with a significantly greater decrease, compared to the two RAIN interventions groups. DID impact estimates suggest a negative impact of the RAIN intervention groups on stunting, ranging from +7pp for the Agonly group, to +9pp for the Ag-Nutrition group, compared to the control group, respectively. Similar negative impacts on mean HAZ scores were seen for the Ag-only group (-0.35 HAZ, compared to the control group). Due to the aging of children throughout the duration of the RAIN project, there was greater potential for certain age groups of children to be exposed to the RAIN interventions for longer than others. Caregivers of children months of age at the time of the endline survey had the opportunity to have been enrolled in the program for the entire 1000 days period between conception and age two. Therefore, we conducted impact analyses in this age group of children. Similar declines were observed in stunting, over time, for all three study group, but no differential reductions in the prevalence of stunting were observed. This suggests a null impact of the RAIN project interventions on stunting, among those with greatest potential for exposure during the 1000 day window of opportunity. xiii

14 Since fewer households than expected participated in the RAIN project, we undertook per-protocol analyses, specifically restricting the impact analyses to households who reported a member having joined a RAIN women s group.we observe similar negative impacts on stunting. Levels of wasting increased significantly over time, for all three study groups, in pure intent to treat analyses among: 1) children months of age at endline, 2) children with greater potential for exposure (24-47 months of age, at endline), and in per-protocol analyses for: 3) children months of age at endline, and 4) children with greatest potential for exposure. In DID analyses, there was a consistent positive impact on the prevalence of wasting, in the Ag-only arm, compared to the control arm, of about -4 pp. This suggests an overall protective effect on wasting, of the Ag-only study arm. Maternal level underlying determinants of child nutrition We assessed caregiver demographics, health seeking behavior, time use, and nutritional status as underlying determinants of child nutritional status. Overall, maternal characteristics were similar across program groups, at both baseline, and endline. At endline, women in the Ag-Nutrition study group had a higher prevalence of >4 prenatal visits and also had higher number of prenatal visits compared to control group. However, these differences were present at baseline as well. There were significant improvements in maternal dietary diversity (measured out of a total of 7 food groups) within study arms over time, in both intent-to-treat, and per-protocol analysis, but there were no differential changes over time, in favor of any study group, in impact analyses. Maternal Body Mass Index (BMI) increased over time in both the Ag + Nutrition and the Ag-only study groups, but these increases were not differential. Overall, in DID impact analyses, there was an increase of approximately 10pp in the proportion of overweight women in the Ag-only arm, compared to the control arm. There was a commensurate decrease in the proportion of women classified as of normal weight, in the Ag-only arm. In per-protocol analysis, there was a marginally significant positive program impact on maternal BMI of 0.57 kg/m 2 in the Ag-only arm, compared to the control arm. Household level underlying determinants of child nutrition We assessed household food security, dietary diversity, socio-economic status, and access to services as underlying determinants of child nutrition. Overall, respondents perception of their household food security (using the household hunger scale) decreased significantly over time for all groups, in both the full sample, and among confirmed RAIN beneficiaries. In impact analyses, there was a significant decrease in the prevalence of little to no hunger in the Ag-Nutrition group, compared to the control group, and a significant increase in the level of moderate hunger in this group. This was the case in xiv

15 both intent-to-treat analysis, and among confirmed RAIN beneficiaries. There was a significant positive program impact on the level of severe hunger ; DID impact estimates indicate reduction of between 4-7 pp in the Ag-only group, compared to the control, in both intent-to-treat, and per-protocol analyses. At the same time, the Ag-Nutrition group had significant positive impacts on household dietary diversity, with an increase of about 1 food group, based on a 12 food group scale, in both intent-to-treat, and perprotocol analyses. Overall, there were no significant differences in socio-economic status or access to services between groups over time. II. Infant and young child feeding The second evaluation objective was to assess the impact of the RAIN package of interventions on core WHO infant and young child feeding (IYCF) indicators among children 0-23 months of age. At endline, all breastfeeding-related IYCF indicators were high across all three study groups, but complementary feeding practices were sub-optimal ranging from approximately 25-30% for the minimum acceptable diet, to 60% for the minimum meal frequency. Several of these IYCF indicators improved over time, within groups, such as early initiation of breastfeeding (ranging from a pp increase over time), or complementary feeding indicators (with increases ranging from 6 to 12 pp for different indicators, in the RAIN intervention groups), but these increases were not differential in favor of any group. Of note, consumption of iron rich foods decreased over time (ranging from 13-15pp), in all groups. Overall, there was no attributable program impact on improving IYCF practices in both intent-to-treat and per protocol analyses. The only impact we found was the consumption of legumes/nuts which was higher in both intervention arms compared to control. III. Caregiver Health and Nutrition Knowledge The third evaluation objective was to assess the impact of the RAIN package of intervention on health and nutrition knowledge among caregivers. Overall, IYCF knowledge increased over time, except for breastfeeding based on child s demand and continue breastfeeing if mother is ill. The overall improvements in breastfeeding knowledge over time was lower in the Ag-only group, when compared to the control group. As with complementary feeding, the knowledge of timely introduction of complemetary food was significant higher in the Ag-Nutrition arm, compared to the control arm; this xv

16 was most notable for animal source foods (flesh foods and eggs, where there was an approximate 20pp greater knowledge regarding the appropriateness of feeding these foods to children 6-8 months of age). Hygiene knowledge was significantly different among study arms at endline for many aspects relating to hand washing; protecting children from worms; and making drinking water safer, but the direction of difference was not consistently favorable to a single study arm, or to the RAIN intervention arms. IV. Women s Empowerment The fourth evaluation objective was to assess the impact of the RAIN package of intervention on women s empowerment. We assessed impacts on eight social and economic domains of women s empowerment, plus empowerment in agriculture. There were clear impacts of the RAIN interventions on different domains of women s empowerment. In DID impact analyses, we observe significant program impacts in the Ag-only group, compared to the control group, on social capital, asset access, financial empowerment, perception of equality. The Ag Nutrition group had significant program impacts, when compared to the control group on social capital only. Overall, the direction of impacts was similar in intent-to-treat, and per-protocol analyses, with slightly higher impacts in per-protocol analyses. There was a clear shift over time in women s involvement in decision making in agriculture. This shift occurred across all study groups for different aspects of decision making. The change was greater in the RAIN intervention groups compared to the control group, suggesting a clear impact of the RAIN interventions on improving women s empowerment in agriculture in both intent-to-treat, and perprotocol analysis. V. Production The fifth evaluation objective was to assess the impact of the RAIN agriculture package of interventions on the availability of and access to a year-round supply of diverse and micronutrient-rich plant and animal source foods at household level. Overall the RAIN interventions had a consistent significant attributable impact on several different dimensions of agricultural production and consequent availability during the year of nutritious foods. Both the Ag-Nutrition and the Ag-only arms, had greater increases over time, compared to the control group, on the total number of foods produced, the total number of agricultural activities engaged in by the households, and the number of months producing Vitamin A rich foods, and dairy. Consistently, program effect sizes were approximately two-fold larger in per-protocol analyses. xvi

17 Drivers of nutrition change Finally, we conducted decomposition analysis to explain factors that may have contributed to the large reductions in stunting over time, and increase in wasting. Our analysis however explain only a small proportion of these changes. The model explains 8.4% and 6.3% of the actual change in HAZ scores and stunting prevalence, respectively, in our key age group (24-59 months) during this period. Among the sources of predicted change, receipt of nutrition counseling and reductions in child morbidity stand out as the largest factors, explaining the largest proportion of the predicted change in HAZ scores. Unexpectedly, household hunger and agriculture production variables did not predict reductions in stunting prevalence or increases in HAZ scores. The model only explains 1% actual change for WHZ and 13% actual change for wasting. For children aged 0-59 months, the models explain 4-6% of actual change for HAZ/stunting and around 16% for WHZ and wasting. Similar analyses with several rounds of the Demographic and Health Survey (DHS) data from Zambia showed similar results. Discussion This report presents findings from a large, complex, multi-year, inter-sectoral project that combined agriculture and nutrition interventions to impact child nutrition. The RAIN project is one of a handful of projects that includes a rigorous randomized design to examine impact, and responds directly to recent calls for stronger evaluation designs of agriculture and nutrition interventions to strengthen the evidence base on these links. Overall, the RAIN project had mixed impacts. The project had: 1) consistently positive impacts on agricultural production, 2) impacts on different domains of women s social and economic empowerment, as well as women s empowerment in agriculture, 3) impacts on household food security as measured by household dietary diversity, and 4) a potential protective effect on child wasting. In general, where there were significant program impacts, the magnitude of these impacts was larger in per-protocol analyses, among confirmed RAIN beneficiaries. There were however no discernable impacts on reducing the prevalence of stunting, on improving IYCF practices among young children, or on improving caregiver health and nutrition knowledge. There appears to be little to no additional benefit of the Ag-Nutrition arm, compared to the Ag-only intervention arm for the impacts achieved. This is further evidenced by greater exposure to the agricultural intervention components of the RAIN project, compared to the nutrition intervention components. xvii

18 The low coverage achieved by the program is one possible factor contributing to the lack of consistent impacts across project objectives. The larger effect sizes in per-protocol analyses support this hypothesis. As noted previously, only a third of eligible households surveyed had participated in the project through joining a women s group the main point of entry into the RAIN project. In addition, home visits by SMFs and CHVs, a critical component of the intervention delivery by the program, did not materialize as envisaged, losing a key one-to-one element that would bolster the project. Across both the women s groups and home visits, it was clear that the agriculture frontline workers (SMFs) were more active than the health-side workers (CHVs), which plays out in the improved agriculture outcomes compared to nutrition and health outcomes in our evaluation. Only percent of households had both high participation and high levels of program delivery; around half of households were categoriezed as receiving medium or high level of delivery. The process evaluation (conducted in 2013 & 2014) suggested that the SMF, a position created specifically for the RAIN project, were initially more incentivized to act for RAIN through RAIN s provision of incentives (agricultural inputs) than the CHV positions that already existed in the community and which did not receive similar incentives until after 2014, though both groups received additional training. In addition, CHVs serviced the entire community, whereas SMFs were working specifically with RAIN groups; also SMFs are a newly created positions, and the information they provide was new, whilst the CHVs were existing positions so women should have been exposed to many of the messages before. However, the project strengthened the knowledge of the CHVs and increased the number of CHVs in the area. It is clear from the evaluation that by and large, the Ag-only group fared better than the Ag-Nutrition group, suggesting no additional value of nutrition interventions in this project, implemented primarily by CHVs. As noted in the results section, perprotocol analysis (whereby analysis was restricted to those individuals that were confirmed RAIN beneficiaries) did not alter the main findings, though almost two-thirds of the sample was lost in this process. As such it is hard to make definitive claims about lack of impact being solely driven by low participation. The impact evaluation design attempts to attribute changes in key impact indicators of interest to the RAIN interventions. As such, it is important to note a limitation of the design is its inability to accurately account for elements beyond the RAIN project interventions. For several indicators of interest, we observe changes over time in both the RAIN intervention groups, as well as the control group. While we are able to document the lack of leakage of formal program delivery of RAIN interventions to the control group, we are unable to document informal leakage of secondary intervention components (knowledge, agricultural inputs etc. provided by non-project staff) among peers or relatives across program study xviii

19 arms. Additionally, we are unable to adequately account for general improvements in government health services across the district, regardless of RAIN study group. There are clear improvements over time, between baseline and endline, in access and use of government-run health services across all study groups, as documented in this report. This is evident in increased access and use of prenatal care, as well as receipt of health and nutrition counseling services at government clinics. As such, it is plausible that this increase in access and use directly impacts health and nutrition knowledge, and IYCF practices, components of prenatal counseling and under 5 clinics at government health centers. This increase may be sufficiently large to prevent detection and attribution of RAIN interventions, over and beyond general increases and secular trends. There is limited evidence from other evaluations of similar programs. An evaluation of Helen Keller International s (HKI) Enhanced Homestead Food Production (EHFP) intervention on Burkina Faso (one of only a very few rigorous impact evaluations of a similar agriculture-nutrition programs designed to improve child nutrition) demonstrated results that were similar for many, but not all, of the outcomes examined for the RAIN project (Olney, Pedehombga et al. 2015). The evaluation of the EHFP program in Burkina Faso found 1) positive impacts on wasting among children, 2) small positive impacts on anemia, 3) small positive impacts on maternal dietary diversity, 4) positive impacts on maternal underweight, and 5) positive impacts on several dimensions of women s empowerment. Of note, the EHFP program had no impact on reducing the level of stunting, and no impact on food security. Overall, the results from the RAIN evaluation contribute to alleviating some of the dearth of evidence from rigorous impact evaluations of integrated agriculture and nutrition programs. There are clear and important maternal and household level benefits of this program, which may be achieved in other, similar programs, and lessons learnt can be used to to scale-up the existing project. The clear and consistent impact of the RAIN intervention on agriculture production, and on women s empowerment, two core objectives of the program, are noteworthy and consistent with the limited evidence to date from similar interventions. xix

20 1 Introduction 1.1 Overview of the nutrition situation in Zambia Zambia ranks low on human development metrics, national income per capita, Gini coefficient, and other indices and measures of poverty and inequality. With a young, rural and very sparsely spread population, with low life expectancy and poor gender equity (Central Statistical Office, Ministry of Health et al. 2009; UNDP 2010), and fairly low agricultural productivity largely due to poverty and poor infrastructure, the Government sees little financial revenue. Production of maize is heavily promoted in Government policy and programs, and is the predominant cash and subsistence crop, with food security in Zambia generally equated to maize security (Smale and Jayne 2009). The Global Hunger Index (K. von Grebmer, J. Bernstein et al. 2015) ranks Zambia as having an extremely alarming hunger situation, highlighting major deficits in nutrition and child survival. Zambia suffers from a high rate of under-five mortality at 75 per 1000 live births, though this is a substantial improvement since the last Demographic and Health Survey (DHS) and more than halved over the past 15 years (Central Statistical Office, Ministry of Health et al. 2014). An estimated 13 percent of adults years old are HIV-positive (Central Statistical Office, Ministry of Health et al. 2014). Fiftyeight percent of children are fully vaccinated at 12 months, and only around half of all children who suffer from diarrhoea, fever or pneumonia received appropriate treatment (Central Statistical Office, Ministry of Health et al. 2014). Use of improved drinking-water sources is low (85% urban, 49% rural), as is access to improved sanitation (56% urban; 34% rural) (UNICEF 2015). Vitamin A supplementation is relatively high nationally (80 90 percent for children under one year), but pockets of poor coverage persist. Despite endemic malaria, only 40 percent of under-fives sleep under insecticide-treated nets. Primary health care is free in Zambia for pregnant women and children under five, although barriers to access still exist; fertility rate (6.2 per woman) is very high (Central Statistical Office, Ministry of Health et al. 2014). Prevalence of undernutrition (stunting) in Zambia had increased since 1990, remaining stable at around 45 percent from 2000 to 2011 (WFP and FAO 2010); the most recent DHS survey shows a reduction in stunting to 40% nationally (Central Statistical Office, Ministry of Health et al. 2014). Six percent of children in Zambia are wasted and 15 percent are underweight; about 10 percent of women have a low body mass index indicating maternal underweight, and 23 percent are overweight (Central Statistical 20

21 Office, Ministry of Health et al. 2014). Infant and young child feeding practices are variable, with 73 percent of children under six months exclusively breastfed, but only 11 percent fed appropriately for their age (Central Statistical Office, Ministry of Health et al. 2014). In Central Province, in which Mumbwa district is located, 42.5 percent of children below five years of age are stunted, 15.3 percent are underweight, and 4.6 percent are wasted (Central Statistical Office, Ministry of Health et al. 2014). Chronic malnutrition is prioritized in the National Food and Nutrition Strategic Plan (2011), and the Sixth National Development Plan explicitly mentions nutrition as an essential cross-cutting issue for achieving the country s socio-economic development. The Government of Zambia is an early riser country within the global Scaling Up Nutrition (SUN) movement and is a Feed the Future focal country. Mumbwa district, where the RAIN project is being implemented, is a rural district in Central Province in the middle of Zambia, around a two hour drive from the capital Lusaka, with a good trunk road connection but little in the way of local roads, transport, or energy infrastructure. Mumbwa is classed by the Famine Early Warning System as the Central Maize-Cotton Zone, with maize and cotton growing widespread; this area is not prone to drought as rainfall is normally adequate and the area has moderate access to the market, though this varies between Wards (ZVAC 2004). Stunting in Central Province was slightly higher than the national average at the last survey, at 42.5%, and wasting slightly lower, at 4.6% (Central Statistical Office, Ministry of Health et al. 2014). 1.2 Description of the RAIN Project The Realigning to Improve Nutrition (RAIN) project was a partnership between Concern Worldwide (CWW) and the International Food Policy Research Institute (IFPRI), aiming to design, implement and evaluate a model of multi-sectoral integration to improve stunting rates in Mumbwa district, Zambia, and to document evidence of both impact and process for application in other contexts and at scale. The project targeted children during the critical period from conception through 23 months of age, roughly equivalent to the first 1,000 days of life, through integrated agriculture, nutrition and health interventions. The overall approach focused on addressing the multi-sectoral causes of malnutrition and on learning how to effectively address the challenges of multi-sectoral collaboration; the RAIN project was rooted in literature suggesting that integrated actions by several sectors can provide more effective and sustainable processes through which improved nutrition outcomes can be achieved. 21

22 The RAIN project comprised of a district-level agriculture intervention to increase year round availability of, and access to, nutrient rich foods at the household level, in some areas accompanied by promotion of optimal health, nutrition, and care seeking behavior through the delivery of social behavior change communication. Most program elements were delivered through local women s groups created by the program, led by a female Smallholder Model Farmer (SMF) nominated by her group to receive agricultural training and inputs and pass these on to the group during monthly meetings. In the nutrition and health areas groups were also linked to an existing Community Health Volunteer (CHV) who received additional training in nutrition topics to pass on to the group. In addition, some communitywide social marketing and information activities were undertaken. A critical characteristic of the RAIN model was hoped to be its ability to be brought to scale and contribute to the achievement of development goals on hunger and poverty. The RAIN project enrolled 4,437 beneficiaries over the four years of implementation, in staged enrolment, with no phase-out strategy for women once they were recruited. Project partners over the lifetime of the project included the Mumbwa District Child Development Agency (MCDA); Women for Change (WFC); the Zambian Ministry of and Livestock (MAL), and the Zambian Ministry of Health (MoH). For more information see Appendix 3: Detail on the RAIN project. The RAIN project was designed to help address a critical gap in the evidence base regarding the degree to which agricultural interventions, either alone or when combined with nutrition and health activities, can reduce the prevalence of stunting in young children. It aimed to establish proof of concept for an intervention model that could then be replicated and scaled up within Zambia and beyond. The project was designed to detect and attribute impact to this RAIN intervention model, and was implemented, monitored and evaluated using a design involving two different intervention groups vs. a comparison group: Intervention Group One was supported to participate both in agriculture and nutrition/ health activities, while Intervention Group Two was supported to participate in agricultural activities only, and the results compared to a control group receiving standard Government services. In line with Concern s core commitment to eradicate extreme poverty, the RAIN project ensured that extremely poor and vulnerable households were included in the project. 1.3 Evaluation Objectives The objectives of this impact evaluation are to assess the impact of two different packages of RAIN interventions delivered through community-based agriculture and health platforms. 22

23 The primary objective of this evaluation is: 1. To assess the impact of the two different RAIN intervention packages on nutrition outcomes among children 24 months and older In addition to the primary evaluation objective, there are several secondary objectives. These include: 2. To assess the impact of the RAIN package of interventions on core WHO infant and young child feeding (IYCF) indicators among children 0-23 months of age 3. To assess the impact of the RAIN package of intervention on health and nutrition knowledge among caregivers 4. To assess the impact of the RAIN package of intervention on women s empowerment 5. To assess the impact of the RAIN agriculture package of interventions on the availability of and access to a year-round supply of diverse and micronutrient-rich plant and animal source foods at household level 1.4 Structure of the Report This report is structured as follows. Chapter 2 presents the evaluation design, sampling methodology, the main components of the survey questionnaire, and the logistics of fieldwork. Chapter 3 describes sample characteristics at baseline. Chapter 4 presents data on exposure to the different components of the RAIN intervention. Chapter 5 presents findings on the core anthropometric impact indicators by program group and survey round, including descriptive statistics and difference-in-difference analysis to look at changes between groups over time. This chapter also presents three different plausibility analyses to dig deeper into the findings, looking at age groups most likely to have benefited from the program; at confirmed beneficiaries of the program; and at changes in the underlying determinants of nutrition that might explain the results. Chapter 6 presents findings on the impact of RAIN on infant and young child feeding (IYCF) practices by program group and survey round, as well as plausibility analysis looking at confirmed beneficiaries of the program. Chapter 7 presents findings on the impact of RAIN on nutrition-related knowledge among caregivers. Chapter 8 presents findings on the impact of RAIN on women s empowerment, and chapter 9 on the impact of RAIN on access to diverse foods. Chapter 10 then presents a decomposition analysis to examine various social, behavioral and economic factors as potential drivers of child growth over time, before chapters 11 and 12 discuss the findings in light of 23

24 process evaluation data and existing literature, and provide some conclusions. The Appendix provides more detailed results for some chapters, where necessary. 2 Methods 2.1 Evaluation Design To evaluate the impact of RAIN activities, two different evaluation designs were considered, both employing 2 different intervention groups (community based agriculture interventions alone; and agriculture + health/nutrition interventions) and a control group, but differing in how these groups are assigned. A fully randomized cluster evaluation design to evaluate the impact of RAIN was determined to be infeasible primarily for practical project implementation reasons, and therefore a hybrid design was adopted that combines a cluster randomized probability design of the RAIN interventions, with a plausibility design that compares RAIN interventions to a control group (Figure 2.1.1). This cluster randomized design yields 3 different study arms: 1. interventions only 2. AND nutrition/health interventions 3. Comparison group with no RAIN project interventions Repeated cross-sectional surveys were administered in the same communities and at the same time of the year in project years 1 and 5 (baseline 2011 and endline 2015, respectively). This design will allow us to 1) document changes in key impact and process indicators over 4 years of program implementation in the two RAIN project intervention areas and the control area; and 2) determine the impact of agriculture and health and nutrition interventions relative to agricultural interventions alone and relative to the control group. 24

25 Figure 2.1. RAIN Evaluation Design Rationale for Age Range to Detect Impacts on Stunting The assessment of the impact of nutritional interventions on child anthropometry (as well as other outcomes) should consider the age at which assessments should be made to detect the greatest difference between intervention and comparison areas. Evidence suggests that (1) the longer children are exposed to early nutrition inputs like improved IYCF practices and nutrition supplementation in the under-two age period, the greater the impact will be; and (2) the earlier children are exposed, the greater the impact will be (Martorell 1995). The logic of the RAIN project targeting this vulnerable age period is that the investments in the first two years of life are progressive and cumulative. Therefore, our ability to detect a significant impact on anthropometric status, particularly, will be greatest among those children who were exposed to RAIN interventions in the entire period preceding the age of months, which signifies the peak age of growth faltering in Zambia. At that point in time, the differences between RAIN intervention and control children will be much larger than in the preceding period because RAIN interventions, if received early enough and long enough, should protect children and prevent the deterioration of their height-for-age 25

26 measurements. Measuring the anthropometric impacts of early nutrition interventions in a younger age group (i.e., <18 months of age) will likely underestimate the total impact of the inputs. For the RAIN project, this means that the age at which impact on anthropometry should be assessed is dependent on (1) the child s age at the onset of exposure to RAIN interventions, (2) the total duration of exposure to the RAIN interventions within the 0-23 month target age-focus for the RAIN project, and (3) the age of peak growth faltering. We apply these principles with the impact evaluation design to define the exact age group on whom to assess impact on anthropometry outcomes, below Randomization Process During an early exercise with district government officials, two control wards were randomly chosen from a set of three ward pairs; the two other ward pairs were selected to receive RAIN project interventions. Within the 4 intervention wards there were a total of 29 survey areas previously defined for the Zambian census (areas known as CSAs). From an evaluation perspective, it was desirable to randomly assign these 29 CSAs to the two different RAIN intervention groups. From an operational point of view, however, it was desirable to keep as few as possible smaller units. This would reduce the risk of spill-over effects and would support staff working on the project to select and maintain properly the specific areas that are assigned to them over a period of several years. Considering both statistical and operational aspects, six CSAs were merged into three. This resulted in a total of 26 CSAs that would be available for randomisation. A map was prepared that showed the location of all 26 areas and their boundaries within the four intervention wards, and a process of drawing lots without replacement was used during a community meeting in Mumbwa with representation from the District MOH, MACO, MOLFD, MCDA and Concern Worldwide. The resulting randomization is shown in Figure

27 Figure 2.2. Map of randomized intervention areas (control not shown) and nutrition only Sample Size Estimate For both Baseline and Endline, all sample size calculations were carried out using STATA 11 software. Two different sample size estimates were obtained. The first was to detect changes in stunting, and the second was to detect changes in height-for-age Z scores. At baseline, sample size of 1000 children aged months per study arm, for a total of 3000 children, was determined to be sufficient to allow us to detect, at endline, a minimum difference in any two groups of a: I. 8 percentage point difference in the prevalence of stunting II. 0.2 Z-score difference in the mean height-for-age Z-score (HAZ), assuming a standard deviation of 1.3. This estimate was based on an estimated baseline prevalence of stunting of 53 percent, with a mean HAZ of -1.8 as reported by the 2009 Demographic and Health Survey (Central Statistical Office, Ministry of Health et al. 2009), a power of 80, a one-sided test, a minimum of 13 sampling clusters per study arm, and considers a relatively high level of clustering (ρ=0.01) of undernutrition within each of the 3 study groups (see Appendices 27

28 Appendix 1: Detailed sample size calculations). At baseline, we sampled approximately 1000 households with a child aged months of age, for a total sample size of 3044 households. At endline, we oversampled the two RAIN interventions arms, by approximately 20% (for an approximately sample size of 1200 households per arm), to account for potential limited intervention exposure at the household level, for a total sample size of 3536 households. 1. Table Sample sizes + Nutrition only Control All Baseline Endline Sampling Methodology Sampling of children in these surveys was specific to households that had at least one child aged months of age, the age range for detecting impacts on stunting i.e. the primary RAIN project impact indicator. However, to capture impacts of the RAIN project on key IYCF indicators, children between 0-23 months of age were also sampled. Logistical considerations prevented us from sampling children 0-23 months and months of age from unique households; sampling unique household would substantially increase our sample size which was not feasible. Therefore, preference was given to households that had both a child months of age as well as a child 0-23 months of age. Where multiple children within these age ranges were present in a household, the youngest child within the household was selected. A listing exercise was undertaken in the 6 wards to determine eligible households i.e. households with the presence of at least one child aged months of age. A child aged months old was identified as the INDEX child. A child aged 0-23 months old was identified as the NON-INDEX child. All households were categorized into 3 possible categories: 1. Households with an INDEX child only 2. Households with both an INDEX and a NON-INDEX child 3. Households with a NON-INDEX child only 1 Based on an assumption of 80% coverage of eligible women, as estimated in Concern s 2013 coverage survey 28

29 Because the CSA is the unit of randomization, it is therefore the unit of most interest from an evaluation perspective as it is the unit by which treatments are applied. However for sampling purposes the CSA covers a very large geographic area and a large number of households and is therefore not practical as a unit for sampling. For sampling purposes the unit that is most practical is the Supervisory Enumeration Area (SEA), which is a clearly identified geographic area as per the Government Central Statistics Office (CSO). The majority of CSAs have 3 SEAs within them, though within the 6 wards from which the baseline survey sample was drawn, there was a range of 2-5 SEAs per CSA, as per the 2010 CSO sampling frame. The sample size calculations required that a total of 69 households be sampled in the endline survey per unit of randomization-the CSA. The majority of CSAs had 3 SEAs, and within these SEAs 23 households were sampled. In CSAs that had 4 or 5 SEAs, 3 SEAs were randomly sampled and 23 households were sampled from each SEA. In CSAs that had only 2 SEAs, 35 households were sampled from the first SEA, and 34 households from the second. Therefore, based on this sampling procedure, a total of 69 households were sampled per CSA as required by the sample size calculations. Within each SEA, households were sampled based on the household listing exercise conducted at the start of field work, using a systematic random sampling procedure that used a random number table. A total of 3044 households were sampled for this baseline survey. Only 34 households that were selected as part of the systematic random sampling procedure refused to be interviewed. 2.2 Survey Instruments Several instruments were created for the collection of field data in this survey, based on a conceptual framework designed to capture the various factors known to affect child nutrition as described in the baseline report (Harris, Quabili et al. 2011). The conceptual framework and survey instruments are described below. Data was collected through a multi-module household questionnaire, and anthropometric measurements of children and caregivers Conceptual basis for the impact evaluation questionnaire The UNICEF conceptual framework (UNICEF 1990) formed the basis for the questionnaires, which were designed to capture information on all levels of influence that contribute to child undernutrition. The framework identifies the causes as immediate, underlying and basic, with each level of factors having influence on the other. The framework identifies inadequate dietary intake and diseases as the immediate causes of undernutrition, which are in turn influenced by a range of underlying factors at the 29

30 household and family level. These are mainly insufficient or lack of access to food, clean water and sanitation, health services, inadequate maternal and child care, and maternal undernutrition. The immediate causes are embedded within the larger societal factors or basic causes which range from human and social capital, women status in the society, political and social atmosphere, etc. In addition to the above key impact indicators, because the household is the unit of analysis, and the focus of our data collection, we captured information from sampled households on the impact of the RAIN project on key agriculture, health, and nutrition indicators as outlined in the log frame (e.g. household food security, dietary diversity, and IYCF practices). In addition, information was collected on other factors that influence the uptake, adoption and impact of RAIN interventions, such as socioeconomic status; local food production; household market access and food purchasing behaviour; household parental characteristics; maternal knowledge of essential nutrition actions (ENA); exposure to other agriculture, health, and nutrition interventions; exposure to media; and household gender relationships. At endline, we also included questions on women's time use, and exposure to the RAIN intervention Household questionnaire The household questionnaire was written in English and translated simultaneously by data collectors into one of the several local languages; responses were noted in English. It was administered to the mother of the index child chosen for the survey (or the child s primary caregiver, if this was not the child s biological mother; the word caregiver is used for the rest of this report, apart from those instances where questions relate to the biological mother of a child only, such as with pre- and postnatal care). The household questionnaire was based on previous nutrition evaluation questionnaires used by IFPRI, which were grounded in the questionnaire model used for Demographic and Health Surveys and incorporated several validated instruments to measure different determinants of nutrition. The questionnaire was adapted substantially for the purposes of the RAIN evaluation, particularly in terms of local foods, agricultural techniques, and indicators of socio-economic status. Feedback was sought from the implementing NGO, and the questionnaire was piloted at baseline and revised into the final version Anthropometric measurement Anthropometric measurements (height and weight) were taken for eligible children in each household (less than 5 years of age) and their caregivers. Anthropometric data was collected using a standardized 30

31 method (WHO Multicentre Growth Reference Study Group 2006) and assessments made by trained and standardized field staff. Weight of the children was measured using electronic weighing scales precise to 100 g. Locally manufactured collapsible length/height boards, which were precise to 1 mm, were used to measure recumbent length of the children < 24 months of age and standing height of children 24 months. Quality control in the field was overseen by supervisors checking measurements daily against tables listing outlier values, and feedback given to enumerators. 2.3 Data collection and management Training of personnel IFPRI hired an experienced and well-qualified survey firm, Palm Associates in Lusaka, Zambia, to conduct the RAIN baseline and endline surveys. The firm has experience with numerous large-scale surveys in rural Zambia for clients such as the World Bank, and works closely with the Zambia Central Statistics Office (CSO). The senior administration of the firm worked with the IFPRI team in all planning and training activities, and was also closely engaged in field supervision of the survey along with his management staff. The training of enumerators took place in Lusaka for 10 days in June The training was led by the IFPRI team along with senior researchers and trainers from Palm Associates. Several specialists were invited to provide training in specific sections, including 1) training on the use of census maps and household listing techniques by a specialist from the Central Statistical Office (CSO); 2) training on comportment and field logistics by senior Palm Associates administration; and 3) training on anthropometric measurement by a specialist consultant from the Zambia National Food and Nutrition Commission (NFNC). Both enumerators and data entry staff, and team supervisors, attended the training, in order to standardize the teams and ensure familiarity with the questionnaire and procedures; supervisors also underwent further training related to their specific duties in the field, including administration of a community questionnaire, quality control of questionnaires, and sampling methods. The training for the household survey component was undertaken using lecture, role-play, and practice methods. The training course consisted of instructions regarding interviewing techniques and field procedures, a detailed review of modules and questions included on the questionnaires, mock interviews between participants in the classroom, and practice interviews. 31

32 The team members were re-trained and fully standardized in anthropometric assessment for the RAIN endline survey, whether or not individuals had previously undertaken anthropometric work. Team members were trained by an anthropometry specialist from the National Food and Nutrition Commission using Food and Nutrition Technical Assistance ( FANTA) guidelines, including explanation of the theory of anthropometric measurement and its application in the RAIN project; use of the equipment, including both demonstration and practice; and recording of anthropometric data accurately.. Standardization for anthropometry was done on young children within the age range for the RAIN survey (recruited from the local community, and attending the training with their caregivers), following the methods recommended in FANTA guidelines Survey Team Composition The endline survey was conducted by twelve field teams, double that at baseline, to reduce field time. The team members for the baseline survey were recruited and trained in Lusaka, and the teams then traveled to Mumbwa District to carry out the fieldwork. Selection criteria included 1) education and training background as well as experience in survey work; 2) understanding and fluency in the local language(s); and 3) time commitment. Each of the teams comprised two enumerators, whose primary purpose was to undertake household interviews; two anthropometrists, who were specially trained to undertake anthropometric measurements for women and children; one driver, whose job was to ensure that the teams got to the right places, in time, and with the correct equipment; and one supervisor, whose responsibility was to coordinate the team members and to oversee the logistical arrangements and survey quality. The two anthropometrists also conducted the field interviews whenever they were not taking anthropometric measurements. Each team was also assigned a 4WD vehicle to facilitate mobility Data Management The data management process involved designing data entry programs for the household questionnaire, training data entry personnel, supervision of data entry, and developing error identification and data cleaning syntaxes. Data entry screens were developed in CS Pro 4.1, and all data cleaning was done in STATA 11. In both screen development and data cleaning tasks, the Palm Associates IT specialist collaborated very closely with IFPRI. Data was double-entered by two data entry operatives, and the data entry screens were specially designed to prompt the verifier whenever discrepancies arose between the first and second entries. As the data entry was taking place, the DES also performed 32

33 frequent consistency checks. Cleaned raw data files were submitted to IFPRI where further data cleaning and variable generation was carried out. Among the more critical steps involved at this stage of data cleaning was ensuring the validity and utility of the household roster which is a key module in the survey, against which all other modules are matched. Additionally, standard data cleaning procedures were followed during the process of variable generation and tabulation of survey results which included range and extreme value checks, skip pattern checks, and basic verification of question responses that should be consistent across survey modules Survey information Endline data collection was carried out over a period of 4 weeks between July and August On average each interview took between 2 to 2.5 hours, with longer interview times occurring at the start of the data collection period. There was little resistance from the community or individual households to the data collection or the burden it placed on respondents. This was evident by the high response rate among households that were approached to be interviewed. Sample sizes at baseline and endline are shown in Error! Reference source not found.; an over-sample of 20% at endline allowed greater power for sub-analyses of households containing program beneficiaries. 2.4 Data analysis Data analysis was carried out according to the impact evaluation statistical analysis plan that was developed in 2010, and updated in 2015, and shared with Concern. The main impact analyses were conducted to estimate the impact of the RAIN interventions on anthropometric outcomes and determining factors. Further analyses were carried out to strengthen evidence of impact attributable to the program by using a dose-response analysis that assessed outcomes based on exposure and intensity of exposure to RAIN interventions, adjusting for characteristics that may have an effect on outcomes, and by tracing impacts through the theory of change. Finally, analyses were also conducted to identify factors that may have contributed to changes in child growth over the evaluation period. Three broad sets of analyses were conducted: 1) Estimation of impact of the RAIN intervention arms (Ag-only, and Ag-Nutrition), compared to the control, and to each other, on the primary impact indicator i.e. stunting. This included: 33

34 a. Estimation of main impact of RAIN interventions on stunting, and HAZ scores, using difference-in-difference estimates b. A series of additional analyses to assess the plausibility of identified impacts. These additional analyses included the following: Plausibility analysis 1: Analysis of the change in stunting prevalence among children in high potential-for-impact age group i.e. among children months of age Plausibility analysis 2: Dose-response analysis between program exposure and child growth outcomes, thereby creating an internal comparison groups to test program effects with greater degree of confidence. Plausibility analysis 3: Analysis of change in determinants of stunting over time. 2) Estimation of impact of RAIN interventions on secondary outcomes: Infant and young child feeding practices; maternal nutrition and health knowledge; women s empowerment; and food production. 3) Decomposition analysis to examine various social, behavioral and economic factors as potential drivers of change in linear growth and stunting over time. All major analyses were conducted using intent-to-treat analyses, as well as among those households confirmed to be RAIN project beneficiaries (per protocol analyses) Table Sample sizes used for analyses + Nutrition only Control Intention to treat analyses Baseline Endline Per-protocol analyses Baseline Endline Below we provide an overview of the statistical analysis methods used in these analyses, the aim of each method, and some basics of interpretation of the outputs produced (Table 2.4.2). 34

35 Table 2.4.2: Analysis methods- use and interpretation Statistical analysis method Aim Interpretation Descriptive statistics: Percentages and means Joint ANOVA F test, adjusted for clustering effects Student T test Difference in difference (DIFF commands) controlled for clustering effect, child age and gender (for anthropometry and IYCF outcomes) Multivarate regression analyses and linear decomposition analyses To describe the sample at baseline and at endline. To compare the balance among three groups at baseline or endline. To compare the difference between baseline and endline within each study aim To test for statistical differences between study arms over the two rounds of surveys. To assess whether particular factors are determinants of change over time. Percentages or means (with standard deviations) are shown for each study arm at different time points, sometimes disaggregated by age group. Statistical difference between study arms at one point in time is shown using asterisks. If the study arms are significantly different to each other, asterisks will be shown in the first column of the baseline or endline results. Statistical difference within each study arm over time is shown using asterisks. The numbers refer to the percentage point difference between baseline and endline within each study arm. Percentage point differences between study arms are shown (either positive increases or negative decreases over time), along with asterisks denoting whether differences between arms are significant over time. Percentages are shown that denote the share of predicted change explained by each variable, as an indication of the importance of each variable as a determinant of the outcome. 35

36 2.4.1 Exposure to the program Key to interpreting the impact results is exposure to the intervention. We constructed an exposure to the program scale that comprises participation (whether and how much a household was participating in RAIN project components) and delivery (RAIN implementation being received as planned). Using data from the exposure module of the questionnaire, exposure scores were calculated in the following way: A Participation score was created based on 1) being a member of a RAIN women s group (possible scores of 0 or 1); the number of group meetings in the first six months of 2015 (a score of 1 is assigned for 1-3 group meetings, a score of 2 for 4-6 meetings, and a score of 3 for 7 meetings); and other RAIN events attended (possible scores of 0 or 1). The total scores were summed, and divided by tertiles to create low, medium and high participation categories. An Ag Delivery score was created by summing performance across two major areas: whether SMFs attended group meetings (possible scores of 0 and 1); and the number of home visits conducted by SMFs (a score of 1 for 1 visit, a score of 2 for 2 visits, and a score 3 for 3 visits). The total delivery score was then used to create three categories: no delivery, medium delivery (score 1) or high delivery (score 2). An Ag- Nutrition Delivery Score was measured by CHV meeting attendance (a possible score of 0 or 1); and CHV home visits (a score of 1 for 1 visit, a score of 2 for 2 visits, and a score 3 for 3 visits). These measures were in addition to all of the measures described above for the Ag only delivery score, and then divided into low, medium and high intensity categories as with the Ag-only group Impact estimates on main and secondary outcomes The 2011 and 2015 household datasets were merged into a single dataset with a time period identifier. The difference-in-differences (DID) (or double-difference) method was applied to assess the difference between the change in the outcomes for intervention (RAIN Ag-only or Ag-nutrition interventions) and control groups in the baseline and endline, accounting for geographic clustering effect for all outcomes, and additional control for child age and gender for anthropometry and IYCF outcomes. The impact estimates yielded (1) percentage point changes binary outcomes; and (2) change in mean scores if they are continuous outcomes. We first conducted analyses for all sample as intent to treat analyses, then restricted analyses in the subsample who exposed to RAIN program, or among those with high potential of impact (in case of child anthropometry) 36

37 For the analyses noted above, regression models were run using the diff and xtreg commands in Stata 13. Both approaches to the regression modeling provided estimates of impact that capture changes over time and between groups. Results are reported at P<0.05, P<0.01, and P<0.001 significance levels Decomposition analysis As with analyses that examine household exposure to RAIN interventions, a key component of strengthening the plausibility of RAIN impact results is to examine changes in underlying determinants over time. Decomposition analysis was conducted to examine various social, behavioral and economic factors as potential drivers in child growth over time, particularly HAZ, stunting, WHZ and wasting that changed over time. These analyses draw upon previous work on explaining the drivers of change in anthropometric outcomes over time in other countries (Headey 2014; Headey, Hoddinott et al. 2014). Using the two rounds of impact evaluation surveys (baseline and endline) and focusing on variables that were consistently measured in both rounds, we begun by looking at the determinants for child growth among children months of age, as it is the main impact evaluation group. Further, we looked at the determinants for child growth among all children less than 5 years of age. Two sets of independent variables were included in the analysis. First, the time-variant intermediate determinants which can help both in understanding which factors explain nutritional differences across children as well as in decomposing changes across time. These include household factors, maternal factors, health service access, sanitation, child morbidity. The second set was control variables that are essentially fixed over time such as maternal height, child and maternal age, child gender, location fixed effects, and RAIN intervention (see Appendices Appendix 1: Detailed sample size calculations). Basic regression analysis was used to test a model of nutrition outcomes against underlying determinants. Then, the Oaxaca-Blinder type decomposition analysis (if several independent variables changed over time) or simple linear decomposition (if parameters of interest are stable over time) was used to determine which factors contributed to change in nutrition outcomes and by how much. 2.5 Ethical approval Ethical permission for the RAIN project, including baseline and endline surveys, was sought and approved in 2010 from the Humanities and Social Sciences Research Ethics Committee of the University of Zambia for the project titled Impact Evaluation of the Realigning to Improve Nutrition 37

38 Project in Mumbwa District, Central Province. Written authorization was also received from the Office of the Permanent Secretary, Ministry of Health, Government Republic of Zambia. Written informed consent was obtained from all survey respondents. A contact sheet was left with each household as well as a copy of informed consent sheet which was also stored by the survey team. 38

39 3 Results: Baseline characteristics KEY RESULTS: At baseline, there were no statistically significant differences among the three study arms on anthropometric outcomes (HAZ, WHZ, stunting and wasting). Most IYCF indicators were comparable across the three study groups; the exceptions were for early initiation of breastfeeding and meal frequency, which were lower in the Ag-Nutrition group. Additionally, maternal and household characteristics, were also comparable among the three groups at baseline, with no major statistically significant difference between groups. These results (presented in detail in the RAIN baseline report) indicate that randomization at baseline was successful. 3.1 Key baseline indicators Overall, the three study arms at baseline were comparable. There were no significant differences between study arms on HAZ, WHZ, stunting or wasting (Table 3.1.1). There were few significant differences on IYCF indicators, eg. Early initiation of breastfeeding and minimum meal frequency were significantly different within the ag-nutrition group over time, but no other results were (Table 3.1.1). Maternal (Table 3.1.2) and household (Table 3.1.3) characteristics were also similar among three groups. Table Baseline core impact indicators by program group Age group (months) Ag- Nutrition only Control N Nutritional status of children HAZ Prevalence of stunting WHZ Prevalence of wasting Core IYCF indicators Early initiation of breastfeeding ** Exclusive breastfeeding among children< 6 months Continued breastfeeding at 1 year ( months) Introduction of solid, semi-solid food, or soft food Minimum diet diversity ( 4 food groups) Minimum meal frequency * Minimum acceptable diet Consumption of iron-rich food * P<0.05, ** P<0.01, *** P< Minimum is defined as 2 times for breastfed infants 6-8 months; 3 times for breastfed children months; 4 times for non-breastfed children months. Meals include both meals and snacks and frequency is based on mother s report. 2 Acceptable diet is defined as who had at least the minimum dietary diversity and the minimum meal frequency during the previous day. 3 Iron-rich or iron-fortified foods include flesh foods, commercially fortified foods especially designed for infants and young children that contain iron, or foods fortified in the home with a micronutrient powder containing iron. 39

40 Table Selected baseline maternal underlying factors by program group + Nutrition only Control N= 978 N= 1,025 N= 1,041 Age (years) (mean± SD) (8.35) (8.48) (9.14) Year of schooling (mean± SD) 6.48 (2.49) 6.34 (2.60) 6.47 (2.40) Education Percent Percent Percent Never attend school Primary school Middle school High school or higher Occupation Others Body Mass Index (kg/m 2 ) Underweight (< 18.5) Normal ( ) Overweight (>=25) Table Selected baseline household underlying factors by program group + Nutrition only Control N= 978 N= 1,025 N= 1,041 Household hunger category (HHS) Little to no hunger (score 0-1) Moderate hunger (score 2-3) No hunger (score 4-6) Household dietary diversity category (HDDS) 0-4 food groups 10.64*** food groups food groups food groups Household assets Ownership of house (%) 95.08*** Total number of durable goods (mean) 8.39 (6.25) 8.06 (6.47) 8.49 (6.80) Total number of productive assets (mean) 8.58 (32.97) 8.49 (34.83) 7.40 (4.70) Household socioeconomic status First quintile Second quintile Third quintile Fourth quintile Fifth quintile

41 4 Results: RAIN intervention exposure KEY RESULTS: Overall participation was 31 percent in the Ag-only group, and 34 percent in the Ag-nutrition group. Of this third of all eligible households participating, 50% received medium or high levels of program delivery. There is no difference in reported exposure when restricting the exposure analyses to households with a child < 2 years of age, when sampled at endline. In terms of intensity of delivery, SMF participation in RAIN groups was high (approximately 90%), but CHV participation was low (38-45%); group members therefore had more opportunity to interact with trainers from the agriculture side than from the health side. The additional home visits, aimed at providing one-to-one support for gardening and IYCF counselling and support, were more limited still; SMF home visits were not happening as often as planned (45, and 53% of households were visited by an SMF in 2015, in the Ag- Nutrition and Ag only arms, respectively), and CHV visits even less often (13% in the Ag-Nutrition arm). In terms of spillover between study arms, leakage of CHV participation to the ag-only group was fairly common as some clinics were sited on the border between project areas, but leakage of the project to control areas was not common, with very few women in these areas reporting participation. The overall pattern and magnitude of these results did not differ between all households sampled, and households with a child < 2 years of age, when sampled at endline. 4.1 Key program exposure indicators Key to interpreting the impact results is understanding exposure to the intervention; this comprises participation (whether and how much a household was participating in RAIN project components) and delivery (RAIN implementation being received as planned). Table shows participation (column) and delivery (row) for both intervention groups. Overall participation was 31 percent in the Ag-only group, and 34 percent in the Ag-nutrition group. Of this third of all eligible households participating, around 50% received medium or high levels of program delivery.table shows the main exposure results in terms of the participation of eligible women in the RAIN project 2, and Table shows this data for caregivers of children under 2 years. Most respondents in both groups had heard of the RAIN project in the two intervention areas, and around a third of households in these areas had a member who was identified as currently belonging to a RAIN group. Around a quarter had ever attended a RAIN group meeting, and a very small proportion had attended in the past six months 3. These low participation rates are borne out in other data (not shown) which finds that of 39 percent of households reporting any agricultural training in the past year, 62 percent was provided by RAIN. Date of joining was 2 This survey sampled only households where there was a child under age 5, therefore respondents to the survey would have been eligible to join RAIN. We ask about current participation of any woman in the household in RAIN. 3 The survey was undertaken between July-Aug 2015, and respondents were asked about meetings since January

42 split evenly over the four rounds of recruitment between 2011 and 2014 in the ag-nutrition areas, but tailed off slightly in the final years in the ag-only area. Very few women in the control area reported participating in RAIN, suggesting low program leakage between the intervention and control areas, but 16 percent had heard of the project. Another aspect of exposure is contact with the SMFs (agriculture) and CHVs (health) who are the frontline workers on the RAIN project. Table and Table show that SMFs were present at the last RAIN group in a majority of cases, but CHVs were present at fewer than half of those meetings in the Ag-Nutrition areas, and were present at almost 40 percent of group meetings even in the agricultureonly areas, suggesting large program leakage between the two intervention arms. Home/garden visits by SMFs were reported by around half of households in both intervention groups (fewer in Ag only), for around 25 minutes per visit. CHV visits were much less common: only around 10% of households reported having been visited in 2015, again for minutes. As well as group meetings and home visits, RAIN also organized other means to get messages to the broader population in the intervention areas. Table and Table show that around 60 percent of respondents reported having heard RAIN radio broadcasts; around 30 percent had seen RAIN posters; and around 20 percent had participated in other events such as gender days. Table Program exposure: Participation and delivery group (n= 1,244) and Nutrition group (n=1212) Non participation N=859 (69.1%) Participation in Ag (%) N=385 (30.9%) Low Medium High Non participation N=797 (65.8%) Participation in Ag-Nutrition N= 415 (34.2%) Low Medium High Delivery in Ag (%) No (score ) Medium Delivery in Ag- Nutrition No (score 0) Medium High High

43 Table Program participation Ag-Nutrition (n=1212) (n=1244) Control (n=1080) Have you heard of the RAIN project, run by Concern Worldwide? 95.63*** Is any member of this household a member of a RAIN women s group? 35.89*** Are you a member of a RAIN women s group? 34.24*** Are you an SMF (Model farmer) for the RAIN project? 6.68*** Have you ever attended a RAIN group meeting? 26.57*** Number of RAIN group meetings attended since 1 January 1, When did you join this group Responses are among the entire sample, unless indicated otherwise: 1 Among people who attended a RAIN group meeting 2 Among people who is a not a SMF for the RAIN project Table Program participation (among mothers with children <2 years old) Ag-Nutrition (n= 635) (n= 568) Control (n= 727) Have you heard of the RAIN project, run by Concern Worldwide? 95.12*** Is any member of this household a member of a RAIN women s group? 33.23*** Are you a member of a RAIN women s group? 31.34*** Are you an SMF (Model farmer) for the RAIN project? 7.09*** Have you ever attended a RAIN group meeting? 22.83*** Number of RAIN group meetings attended since 1 January 1, When did you join this group Responses are among the entire sample, unless indicated otherwise: 1 Among people who attended a RAIN group meeting; 2 Among people who is not a SMF for the RAIN project 43

44 Table SMF and CHV interaction Ag-Nutrition (n= 322) (n= 273) Percent Percent Percent of groups with RAIN SMF present last meeting Percent of groups with RAIN CHV present last meeting Percent of households visited by SMF in Percent of households visited by CHV in Mean (SD) Mean (SD) Duration of SMF visits (n=295) (14.38) (16.72) Duration of CHV visits (n=270) (17.28) (32.28) Note: figures are calculated among people who attended a RAIN group meeting Table SMF and CHV interaction (among mothers with children <2 years old) Ag-Nutrition (n= 145 ) (n= 131) Percent Percent Percent of groups with RAIN SMF present last meeting Percent of groups with RAIN CHV present last meeting Percent of households visited by SMF in Percent of households visited by CHV in Mean (SD) Mean (SD) Duration of SMF visits (n=295) (16.87) (18.22) Duration of CHV visits (n=270) (22.10) (41.15) Note: figures are calculated among people who attended a RAIN group meeting Table Exposure to broader RAIN project events and materials Ag-Nutrition (n= 1212) (n= 1244) Percent of households participating in other RAIN events Percent of households who have seen RAIN posters Percent of households who have heard RAIN radio messages Percent of households with access to solar drier Table Exposure to broader RAIN project events and materials (among mothers with children <2 years old) Ag-Nutrition (n= 635 ) (n= 568 ) Percent of households participating in other RAIN events Percent of households who have seen RAIN posters Percent of households who have heard RAIN radio messages Percent of households with access to solar drier

45 4.2 Community exposure to non-rain programs KEY RESULTS: There was no difference between intervention and control areas in presence of other NGO-supported projects or activities. Using the community-level data, we can see whether NGO programs other than RAIN were working in the intervention and control areas. Overall, there were no significant differences across study areas in NGO presence (Table 4.2.1), social capital clubs (Table 4.2.2) or income generating activities (Table 4.2.3) supported by NGOs, or training activities supported by NGOs (Table 4.2.4), other than the presence or absence of RAIN. We do not have data on the presence of government services in these areas, though there was reported spillover of nutrition/health information between intervention arms from RAINtrained CHVs working in the health facilities which are visited by people from the Ag-only areas as well, as some health facilities were on the border between Ag-nutrition and Ag-only areas; this is unlikely to have affected the control areas, which are geographically distant from the intervention areas. We can therefore conclude that the control group appears to function as a valid comparator to the intervention arms. Table NGO presence, by program group + Nutrition only Control N= 41 N= 40 N= 41 AGDC Alliance Cargile CEF Conservation farming unit Child fund Climate for change Concern Worldwide/RAIN Dunavant Farmer group/ AGRO club Women for change Women s club

46 Table Social capital clubs supported by NGOs, by program group + Nutrition only Control N= 41 N= 40 N= 41 Farmers /Agricultural Club Fishery Club Bee Keeper s Club Women s Club Credit Club Peer Educators Table Income generating activities supported by NGOs, by program group + Nutrition only Control N= 41 N= 40 N= 41 Agricultural production Agricultural marketing Agricultural processing Business development (general) Tailoring Carpentry Black smithing Kniting/weaving Crafts Table Training activities supported by NGOs, by program group + Nutrition only Control N= 978 N= 1,025 N= 1,041 Financial Management Training Business / Enterprise Training Natural Resource Management Training Tourism Training Health training Agricultural training Nutrition training

47 5 Results: Impact of RAIN interventions on anthropometric outcomes Children s weight and height measurements were used to derive statistical scores by comparing each child s anthropometric measurements to the 2006 WHO child growth standards reference for his/her age and sex (WHO 2010). The three indicators created were height-for-age Z-score (HAZ), weight-for-age Z-score (WAZ) and weight-for-height Z-score (WHZ). Stunting was defined as HAZ < -2 Z-scores; underweight was defined as WAZ < -2 Z-scores; and wasting was defined as WHZ < -2 Z-scores. Results on mean z-scores and proportions malnourished by the different indicators were generated separately for RAIN intervention and control areas. 5.1 Main impact analysis of RAIN interventions on anthropometric outcomes KEY RESULTS: Over time, between baseline and endline, the prevalence of stunting decreased significantly in all three study groups in the impact evaluation age range of children months of age. However, there was a differential decline in stunting in the three study groups, in favor of the control group, with a significantly greater decrease, compared to the two RAIN interventions groups. There was no difference in stunting declines between the two intervention groups. DID impact estimates in this age group, suggest a negative impact of the RAIN intervention groups on stunting, ranging from +7pp for the Ag-only group, to +9pp for the Ag-Nutrition group, compared to the control group, respectively. Similar negative impacts on mean HAZ scores were seen for the Ag-only group (-0.35 HAZ, compared to the control group). Levels of wasting increased significantly over time, for all three study groups, in pure intent to treat analyses among: 1) children months of age at endline, 2) children with greater potential for exposure (24-47 months of age, at endline), 3) children months of age at endline, in perprotocol analysis, and 4) children with greatest potential for exposure, in per protocol analysis. In DID analyses, there was a consistent positive impact on the prevalence of wasting, in the Ag-only arm, compared to the control arm, of about -4 pp. This suggests an overall protective effect on wasting, of the Ag-only study arm. 47

48 % Intent-to-treat outcomes for anthropometric outcomes among children months of age Overall, levels of stunting reduced significantly between 2011 and 2015 in all three study arms, in the main impact evaluation group (24-59 months) (Figure 5.1). However, the magnitude of stunting reduction was smaller in intervention groups (reductions of 12.3 pp for Ag-Nutrition and 13.7 for Agonly) compared to the control group (22 pp) (Table 5.1.1). The difference-in-difference (DID) impact estimates for stunting show a significant differential change over time, with lower impact estimates of Ag-Nutrition group and the Ag-only groups compared to control group (Impact estimate of +9.0 and +7.4 pp, compared to the control group, respectvely). No differential reduction in stunting was observed between the two intervention group (Ag-Nutrition vs Ag only). There was also a statistically significant increase in mean HAZ for all three study groups over time, with a greater increase in the control group compared to Ag only group (DID HAZ). In contrast to stunting, levels of wasting increased significantly over time for all three groups (Table 5.1.1). The DID estimations for wasting among children months of age show a significant differential change of approximately -4 pp in favor of the two intervention groups, compared to the control group. Figure 5.1. Prevalence of stunting, underweight and wasting among children months by program group and survey round Baseline Endline Ag-Nutrition Control Ag-Nutrition Control Ag-Nutrition Control Stunting Underweight Wasting 48

49 Table Anthropometric indicators among children months of age, by program group and survey round Impact indicators Baseline Endline Ag-Nutrition Control Ag-Nutrition Control (n= 963) (n= 1,014) (n= 1,026) (n= 1212) (n=1,244) (n=1,080 ) Ag- Nutrition T2-T1 T2-T1 Control T2-T1 Ag- Nutrition vs control Difference in difference vs control Agnutrition vs HAZ, mean (SD) ** *** 0.51*** 0.87*** * 0.14 (1.37) (1.39) (1.52) (1.82) (1.71) (1.65) WAZ, mean (SD) ** *** 0.30*** 0.19*** (1.03) (1.00) (1.07) (1.11) (1.08) (1.00) WHZ, mean (SD) *** *** *** ** (1.34) (1.16) (1.22) (1.54) (1.48) (1.56) Stunting, % ** *** *** *** 9.00* 7.40* 1.20 Underweight, % * -4.07** Wasting, % *** ** 3.43*** 7.42*** -4.20* -4.00* Significant differences: *** p<0.001, ** p<0.01, * p<0.05; Double difference impact estimates with clustered standard errors comparing 2010 and Accounts for child age, gender and geographic clustering. 49

50 % Intent-to-treat outcomes for anthropometric outcomes among children months of age There were similar trends and differences in stunting among the younger age group ( months), but the magnitudes of difference were larger (Table 5.1.2): The DID estimations for stunting among children 6-23 months of age show a significant differential impact of an increase in the prevalence of stunting between +11 pp and +21 pp for the Ag only and Ag-Nutrition study groups, compared to the control group. The impact on wasting was larger among children months with DID estimates between -11 to -14 pp in favor of the Ag-Nutrition and Ag only study groups respectively, compared to the control group. Figure 5.2. Prevalence of stunting, underweight and wasting among children months by program group and survey round Baseline Endline Ag-Nutrition Control Ag-Nutrition Control Ag-Nutrition Control Stunting Underweight Wasting 50

51 Table Anthropometric indicators among children months of age by program group and survey round Impact indicators Baseline Endline Ag- Control Ag-Nutrition Control Ag-Nutrition Control Nutrition Ag- T2-T1 T2-T1 T2-T1 (n= 478) (n=463) (n= 595) (n=433) (n= 401) (n= 498) Nutrition vs control Difference in difference vs control Agnutrition vs HAZ, mean (SD) *** *** 1.08*** 2.16*** -1.69*** -1.25** (2.08) (1.96) (1.98) (2.42) (2.08) (2.59) WAZ, mean (SD) ** 0.59*** 0.78*** -0.46* (1.46) (1.45) (1.45) (1.41) (1.25) (1.27) WHZ, mean (SD) *** * *** 0.53* 0.70** (1.61) (1.61) (1.59) (2.08) (1.74) (2.03) Stunting, % *** ** *** *** 21.10*** 11.20** Underweight, % *** -9.05*** *** Wasting, % *** *** 3.90** 17.29*** * ** 2.30 Significant differences: *** p<0.001, ** p<0.01, * p<0.05; Double difference impact estimates with clustered standard errors comparing 2010 and Accounts for geographic clustering only. 51

52 5.2 Plausibility analysis 1: High potential for exposure age groups ( months) KEY RESULTS: Due to the aging of children throughout the duration of the RAIN project, there was greater potential for certain age groups of children to be exposed to the RAIN interventions for longer than others. Caregivers of children months of age at the time of the endline survey had the opportunity to have been enrolled in the program for the entire 1000 days period between conception and age two. Therefore, we conducted impact analyses in this age group of children. Similar declines were observed in stunting, over time, for all three study groups, but no differential reductions in the prevalence of stunting were observed. This suggests a null impact of the RAIN project interventions on stunting, among those with greatest potential for exposure during the 1000 day window of opportunity. Due to the aging of children throughout the duration of the RAIN project, there was more potential for certain age groups to be exposed to the interventions than others. Caregivers of children aged over 24 months and under 48 months at the time of the endline survey had the opportunity to have been enrolled in the program for the entire 1000 days period between conception and age two, and so we conducted analyses in this child age group ( months) to see whether impact was greater for this cohort. Levels of stunting significantly reduced (and HAZ scores significantly increased) in all three study arms since baseline in this age group (Figure 5.3 and Table 5.2.1). The DID estimates show no significant difference in stunting or HAZ between study arms over time in the intent to treat group. Differences were significant in the per-protocol anlaysis however, with stunting in the Ag-nutrition group reducing less than in the other two study arms among confirmed beneficiaries (Table 5.2.2). Levels of wasting significantly increased in all three study arms since baseline in this age group (Figure 5.3 and Table 5.2.1), while WHZ scores only significantly decreased in the control arm. DID estimates show no significant differences in wasting between study arms over time in the intent to treat anlaysis in this age group, though WHZ scores were significantly increased over time in the Ag-only group compared to control. Changes approached significance in the per-protocol anlaysis, with wasting increasing less in the Ag-nutrition arm than in the other control arm among confirmed beneficiaries (Table 5.2.2). 52

53 % Figure 5.3. Prevalence of stunting, underweight and wasting among children months by program group and survey round Baseline Endline Ag-Nutrition Control Ag-Nutrition Control Ag-Nutrition Control Stunting Underweight Wasting 53

54 Table Anthropometric indicators among children months of age by program group and survey round Impact indicators Baseline Endline Ag- Control Ag-Nutrition Control Ag-Nutrition Control Nutrition Ag- T2-T1 T2-T1 T2-T1 (n= 698) (n= 759) (n= 786) (n= 809) (n=850) (n= 687) Nutrition vs control Difference in difference vs control Agnutrition vs HAZ, mean (SD) * *** 0.57*** 0.89*** (1.45) (1.43) (1.56) (1.97) (1.84) (1.82) WAZ, mean (SD) *** 0.36*** 0.31*** (1.08) (1.03) (1.08) (1.17) (1.16) (1.03) WHZ, mean (SD) (1.25) 0.41*** *** * (1.36) (1.14) (1.61) (1.48) (1.62) Stunting, % *** *** *** Underweight, % * -4.25** Wasting, % * ** 3.50*** 6.93*** Significant differences: *** p<0.001, ** p<0.01, * p<0.05, +p<0.10 Double difference impact estimates with clustered standard errors comparing 2011 and Accounts for geographic clustering only. 54

55 Table Anthropometric indicators among beneficiaries with children months, by program group and survey round Impact indicators Baseline Endline Ag- Control Ag-Nutrition Control Ag-Nutrition Control Nutrition Ag- T2-T1 T2-T1 T2-T1 (n= 698) (n= 759) (n= 786) (n= 278) (n=263) (n= 687) Nutrition vs control Difference in difference vs control Agnutrition vs HAZ, mean (SD) *** * 0.68*** 0.89*** -0.65** * (1.45) (1.43) (1.56) (1.76) (1.79) (1.82) WAZ, mean (SD) *** 0.31*** (1.08) (1.03) (1.08) (1.20) (1.18) (1.03) WHZ, mean (SD) ** *** (1.36) (1.14) (1.25) (1.61) (1.48) (1.62) Stunting, % * *** *** 12.4** * Underweight, % ** Wasting, % * 3.62** 6.93*** Significant differences: *** p<0.001, ** p<0.01, * p<0.05; +p<0.10 Double difference impact estimates with clustered standard errors comparing 2011 and Accounts for geographic clustering. 55

56 5.3 Plausibility analysis 2: Dose-response relationship between program exposure child nutrition KEY RESULTS: Since fewer households than expected participated in the RAIN project, we undertook per-protocol analyses, specifically restricting the impact analyses to households who reported a member having joined a RAIN women s group. Among households with children months of age, stunting rates decline over time in all three study groups. Additionally, we observe a significant negative impact in the Ag-Nutrition group, compared to the control group (+14pp, compared to the control group). There was no significant impact on stunting for any other comparisons. Among households with children who had the greatest potential to benefit (households with children months of age, at endline), we observe similar negative impacts on stunting. Fewer households than expected participated in the RAIN program (around 35% of eligible households; Table 4.1.2). Stunting decreased significantly over time in all three study arms among these confirmed beneficiaries (Figure 5.4). At endline, there was significantly higher stunting and lower HAZ scores in the ag-nutrition arm than the control arm; DID impact estimates for stunting show a significant differential change over time of 13.9pp in favor of the control group (Table 5.3.1). There was significantly more stunting among children whose caregivers were members of a RAIN women s group in the ag-nutrition study arm than the ag only arm (Table 5.3.2). Wasting increased significantly over time in all three study arms among confirmed beneficiaries (Figure 5.4). At endline, there was significantly less wasting and higher WHZ scores in the ag-nutrition and Ag arm than the control arm; DID impact estimates show a significant impact of 4.6pp, in favor of ag only over control arms (Table 5.3.1). 56

57 % Figure 5.4. Prevalence of stunting, underweight and wasting among beneficeries with children months by program group and survey round Baseline Endline Ag-Nutrition Control Ag-Nutrition Control Ag-Nutrition Control Stunting Underweight Wasting 57

58 Table Anthropometric indicators among beneficeries with children months of age by program group and survey round Impact indicators Baseline Endline Ag- Ag- Control Control Nutrition Nutrition (n= 963) (n= 1014) (n=11026) (n= 415) (n= 385) (n= 1080) Ag- Nutrition T2-T1 T2-T1 Control T2-T1 Ag- Nutrition vs control Difference in difference vs control Agnutrition vs HAZ, mean (SD) *** *** 0.59*** 0.87*** -0.56** (1.38) (1.39) (1.52) (1.69) (1.69) (1.65) WAZ, mean (SD) * 0.30*** 0.19*** (1.03) (1.00) (1.07) (1.09) (1.09) (0.10) WHZ, mean (SD) *** *** 0.30* 0.39** (1.34) (1.16) (1.22) (1.56) (1.42) (1.56) Stunting, % *** ** *** *** 13.90*** Underweight, % ** Wasting, % ** ** 2.79** 7.42*** * 1.10 Significant differences: *** p<0.001, ** p<0.01, * p<0.05; Double difference impact estimates with clustered standard errors comparing 2010 and Accounts for child age, gender and geographic clustering. 58

59 Table Association between program exposure with child stunting among children m Program exposure n Ag-Nutrition Stunting prevalence n Stunting prevalence A member of RAIN women s group Yes ** No Time joining the group <2 year years years years Number of RAIN group meeting attended (since January 1, 2015) >=

60 5.4 Plausibility analysis 3: Changes in the underlying determinants of child growth and nutrition In this section, we examine changes in various underlying determinants at the child, maternal and household levels over time, as potential factors influencing child growth and nutrition Child characteristics KEY RESULTS: We assessed health service coverage and child morbidity as immediate determinants of child nutritional status. Vitamin A supplementation and deworming coverage were fairly high, but full vaccination was low. Overall child morbidity improved in all study arms over time, but was worse in the ag only arm compared to the other arms. We assessed health service coverage and child morbidity as determinants of nutritional status. Vitamin A supplementation coverage was fairly high, at 75-90% coverage in both month olds and month olds. Deworming coverage was slightly lower at 60-90% coverage in both age groups. Immunizations varied widely, with up to 80% coverage of some key vaccines, but no more than 4% of children were fully vaccinated according to Zambian vaccination schedules (Table 5.4.1and Table 5.4.2). Child morbidity (fever, cough, and diarrhea) improved in almost all study arms between baseline and endline. Outcomes were worse at endline in the ag only study arm than the ag-nutrition or control arms across the board; outcomes were significantly better in the ag-nutrition arm than the ag only arm for fever and cough at months; for fever and diarrhea at months; and for all three outcomes at months, but there were no difference from control (Table 5.4.3). DID impact estimates for health outcomes show a significant differential change over time for fever at months of 9.7pp for the Ag only study group compared to the control group and -12.6pp for the ag-nutrition group compared to the ag only group; for cough at months of 9.8pp for the ag only study arm compared to control; and for cough at months of 8.9pp for ag only compared to control, and pp for ag-nutrition compared to ag only (Table 5.4.3). 60

61 Table Child immunization and supplementation status among children months, by program group and survey round Baseline Endline Impact indicators Ag-Nutrition Control Ag-Nutrition Control (n= 690) (n=645) (n= 799) (n= 635) (n=568 ) (n= 727) Vitamin A 76.81** De-worming 60.29* Immunizations - BCG ** OPV ** OPV * *** OPV ** * OPV *** * OPV DPT * DPT DPT DPT-Hep B-Hib *** DPT-Hep B-Hib ** *** DPT-Hep B-Hib ** *** Measles (MMR) at 9 months Measles (MMR) at 18 months 46.52** PCV PCV PCV ROTA ROTA

62 Table Child immunization and supplementation status among children months, by program group and survey round Baseline Endline Impact indicators Ag-Nutrition Control Ag-Nutrition Control (n= 963) (n= 1,014) (n= 1,026) (n= 1,212) (n= 1244) (n= 1080) Vitamin A 90.86* De-worming 87.54* *** Immunizations - BCG OPV ** OPV OPV OPV * OPV * DPT DPT DPT DPT-Hep B-Hib * * DPT-Hep B-Hib * DPT-Hep B-Hib * Measles (MMR) at 9 months Measles (MMR) at 18 months PCV PCV * PCV ** ROTA * ROTA

63 Table Child morbidity, by program group and survey round Baseline Endline Indicators Ag-Nutrition Control Ag-Nutrition Control Ag- Nutrition T2-T1 T2-T1 Control T2-T1 Difference in difference Ag- Nutrition vs vs control control Agnutrition vs mo (n= 212) (n= 182) (n= 204) (n= 201) (n= 159) (n= 226) Fever *** *** *** * *** Cough/ cold * * *** * Diarrhea * ** mo (n= 478) (n=463) (n= 595) (n= 433) (n= 401 ) (n= 498) Fever * *** -7.45** *** Cough/ cold *** *** * Diarrhea *** *** -8.41** *** mo (n= 963) (n= 1014) (n= 1026) (n= 1212) (n= 1244) (n= 1080) Fever *** *** -6.81*** *** Cough/ cold *** *** *** * ** Diarrhea *** *** -3.27* -6.98*** * Significant differences: *** p<0.001, ** p<0.01, * p<0.05; Double difference impact estimates with clustered standard errors comparing 2010 and Accounts for geographic clustering. 63

64 5.4.2 Maternal characteristics: Demographic, health seeking behavior, and nutritional status KEY RESULTS: We assessed caregiver demographics, health seeking behavior, time use, and nutritional status as underlying determinants of child nutritional status. Overall, maternal characteristics were similar across program groups, at both baseline, and endline. At endline, women in the Ag-Nutrition study group had a higher prevalence of >4 prenatal visits and also had higher number of prenatal visits compared to control group. However, these differences were present at baseline as well. There were significant improvements in maternal dietary diversity (measured out of a total of 7 food groups) within study arms over time, in both intent-to-treat, and per-protocol analysis, but there were no differential changes over time, in favor of any study group, in impact analyses. Maternal BMI increased over time in both the Ag-Nutrition and the Ag-only study groups, but these increases were not differential. Overall, in DID impact analyses, there was an increase of approximately 10pp in the proportion of overweight women in the Ag-only arm, compared to the control arm. There was a commensurate decrease in the proportion of women classified as of normal weight, in the Ag-only arm. In per-protocol analysis, there was a marginally significant positive program impact on maternal BMI of 0.57 kg/m 2. Baseline and endline maternal characteristics were similar across program groups, with most women married, with low levels of education, and working in agriculture; average age of women with children eligible for the RAIN project was around 30 years, but this varied between 20 to 40 years (Table 5.4.4). No difference was found for time allocation to childcare, rest, agricultural work, or any other category of time use across three groups in either per-protocol or intent to treat analysis (Table and Table 5.4.6). Significantly fewer women in the ag-nutrition study arm reported receiving nutrition or breastfeeding counselling during pregnancy, and significantly more reported night-blindness when pregnant. Additionally (not related to program activities) significantly more women in the ag-nutrition arm had 4 or more antenatal visits, and visits earlier in pregnancy, and took iron supplements for 7 or more months of pregnancy (Table 5.4.7). There were no significant differences in maternal dietary diversity between study arms over time in caregivers of younger children, either among the full sample (Table 5.4.8) or among confirmed beneficiaries (Table 5.4.9). Within all study arms, consumption increased significantly of legumes, eggs, and vitamin A rich fruits and vegetables, and decreased significantly of other fruits and vegetables and animal source foods. 64

65 Average body mass index (BMI) was around 23kg/m 2 for women at baseline and endline; at endline, normal weight and overweight were significantly different among study arms, but there were no differences in underweight between study arms in the intent to treat anlaysis (Table ). Levels of underweight reduced significantly over time in the ag only and control arms, while levels of overweight increased significantly in the ag only arm. DID estimates show a reduction over time in levels of normal weight of -8.1 to -11.4pp in the ag-nutrition and ag only arms compared to control, respectively, and an increase over time in overweight by 10.3pp in the ag-only group compared to control. Findings were similar in the per-protocol anlaysis among RAIN beneficiaries, though magnitudes of difference were larger ( Table ). Table Selected baseline maternal underlying factors, by program group and survey round Baseline 2011 Endline 2015 Ag-Nutrition (n=978) (n=1025) Control (n=1041) Ag-Nutrition (n=1212) (n=1244) Control (n=1080) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Age (years) (8.35) (8.48) (9.14) (8.35) (8.55) (9.03) Year of schooling 6.48 (2.49) 6.34 (2.60) 6.47 (2.40) 6.43 (2.38) 6.47 (2.44) 6.70 (2.40) percent percent percent percent percent percent Education Never attend school Primary school Middle school High school or higher Civil status Married (union) Living alone (unmarried, widowed, divorced, separated) Occupation Others * P<0,05, ** P<0,01, *** P<0,001 65

66 Table Time allocation, by program group Endline Ag-Nutrition Control (n= 1212) (n= 1244) (n=1080) Mean (SD) Mean (SD) Mean (SD) Sleeping and resting (2.18) (1.81) (2.05) Eating and drinking 1.56 (0.76) 1.54 (0.64) 1.56 (0.67) Personal care 0.75 (0.58) 0.73 (0.49) 0.76 (0.61) School (also homework) 0.02 (0.29) 0.02 (0.3) 0.02 (0.25) Work as employed 0.03 (0.37) 0.07 (0.64) 0.06 (0.63) Own business work 0.33 (1.52) 0.79 (1.97) 0.29 (1.33) Tending a garden 0.38 (1.05) 0.27 (0.92) 0.21 (0.76) Tending livestock 0.02 (0.22) 0.02 (0.29) 0.01 (0.14) Other farming/ agriculture 2.8 (2.77) 2.38 (2.66) 2.35 (2.42) Shopping/ getting service (include health services) 0.17 (0.83) 0.19 (0.83) 0.11 (0.55) Weaving, sewing 0.07 (0.4) 0.1 (0.49) 0.09 (0.41) Cooking 1.83 (0.85) 1.86 (0.82) 1.8 (0.9) Domestic work 1.6 (1.19) 1.7 (1.1) 1.74 (1.22) Care for children/ adult/ elderly 1.18 (1.22) 1.04 (0.97) 1.37 (1.75) Travelling and commuting 0.54 (1.23) 0.64 (1.22) 0.42 (0.94) Watching TV/ listening to radio/ reading 0.2 (0.65) 0.21 (0.61) 0.31 (0.84) Exercising 0.03 (0.21) 0.02 (0.26) 0.01 (0.12) Social activities and hobbies 0.7 (1.34) 0.63 (1.18) 0.37 (0.86) Religious activities 0.26 (0.98) 0.33 (1.09) 0.28 (1.01) Others 0.05 (0.58) 0.06 (0.55) 0.15 (0.94) 66

67 Table Time allocation among beneficiaries, by program group Endline Ag-Nutrition Control (n= 1212) (n= 1244) (n=1080) Mean (SD) Mean (SD) Mean (SD) Sleeping and resting (2.38) (1.75) (2.05) Eating and drinking 1.49 (0.74) 1.5 (0.63) 1.56 (0.67) Personal care 0.69 (0.5) 0.72 (0.5) 0.76 (0.61) School (also homework) 0.01 (0.08) 0.02 (0.25) 0.02 (0.25) Work as employed 0.01 (0.09) 0.04 (0.51) 0.06 (0.63) Own business work 0.29 (1.37) 0.38 (1.35) 0.29 (1.33) Tending a garden 0.72 (1.29) 0.5 (1.15) 0.21 (0.76) Tending livestock 0.03 (0.18) 0.03 (0.21) 0.01 (0.14) Other farming/ agriculture 2.76 (2.69) 2.78 (2.64) 2.35 (2.42) Shopping/ getting service (include health services) 0.16 (0.8) 0.15 (0.72) 0.11 (0.55) Weaving, sewing 0.08 (0.45) 0.09 (0.45) 0.09 (0.41) Cooking 1.82 (0.88) 1.99 (0.85) 1.8 (0.9) Domestic work 1.66 (1.25) 1.86 (1.14) 1.74 (1.22) Care for children/ adult/ elderly 1.24 (1.29) 1.12 (1.06) 1.37 (1.75) Travelling and commuting 0.52 (1.43) 0.62 (1.17) 0.42 (0.94) Watching TV/ listening to radio/ reading 0.15 (0.6) 0.2 (0.58) 0.31 (0.84) Exercising 0.04 (0.24) 0.02 (0.36) 0.01 (0.12) Social activities and hobbies 0.67 (1.26) 0.62 (1.18) 0.37 (0.86) Religious activities 0.22 (0.83) 0.27 (0.94) 0.28 (1.01) Others 0.03 (0.45) 0.04 (0.5) 0.15 (0.94) 67

68 Table Use of prenatal care, by program group and survey round Baseline 2011 Endline 2015 Ag-Nutrition (n=978) (n=1025) Control (n=1041) Ag-Nutrition (n=1212) (n=1244) Control (n=1080) percent percent percent percent percent percent Had prenatal consultation during pregnancy ** Had 4 prenatal visits or more 64.60*** *** Stage of pregnancy at first prenatal visit First trimester 25.88* *** Second trimester Third trimester Took iron pills when pregnant 97.35* Average months of iron supplement intake 7 months or more 5.37* *** to 6 months Less than 4 months Experienced night blindness when pregnant *** Received nutrition counselling during pregnancy *** Received breastfeeding counselling 58.58* *** Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Number of prenatal visits to a health professional 4.00*** (1.23) 3.84 (1.24) 3.63 (1.19) 4.21*** (1.63) 4.01 (1.17) 3.92 (1.15) Months of pregnancy at first prenatal consultation 4.33* (1.18) 4.46 (1.16) 4.48 (1.24) 3.83*** (1.00) 3.89 (1.04) 4.05 (1.15) 68

69 Table Maternal dietary diversity, by program group and survey round (< 24 mo) Baseline Endline Difference in difference Indicators Ag- Ag- Ag- Control Agnutrition Control Control Nutrition Nutrition Nutrition Ag- Nutrition vs (n= 690) (n= 645) (n= 799) (n= 635) (n= 568) (n= 727) T2-T1 T2-T1 T2-T1 vs vs control control Grains, roots and tubers * Legumes and nuts *** 16.11*** 10.26*** Dairy (milk, yogurt, cheese) *** 11.65*** 9.85*** Animal foods (meat, fish, poultry and liver/organ meats) a b a a b a *** *** *** Eggs *** 12.00*** 6.27** Vitamin A rich fruits and vegetables *** 18.82*** 18.61*** Other fruits and vegetables *** *** *** * Number of food groups (0/7) 3.07 (0.98) 3.15 (0.92) 3.09 (1.04) 3.26 (1.08) 3.34 (1.10) 3.24 (1.05) 0.19*** 0.20*** 0.15**

70 Indicators Table Maternal dietary diversity among beneficiaries, by program group and survey round Ag- Nutrition T2-T1 T2-T1 Control T2-T1 Difference in difference Baseline Endline Ag- Ag- Control Nutrition Nutrition Control (n= 690) (n= 645) (n= 799) (n= 199) (n= 179) (n= 727) Ag- Nutrition vs vs control control Grains, roots and tubers Legumes and nuts ** 11.91*** 10.26*** Dairy (milk, yogurt, cheese) * 9.85*** Animal foods (meat, fish, poultry and liver/organ meats) a b a *** 24.78*** *** Eggs *** 8.81** 6.27** Vitamin A rich fruits and vegetables *** 19.89*** 18.61*** Other fruits and vegetables * *** *** Number of food groups (0/7) 3.07 (0.98) 3.15 (0.92) 3.09 (1.04) 3.27 (1.01) 3.21 (1.00) 3.24 (1.05) Significant differences: *** p<0.001, ** p<0.01, * p<0.05 Note: This analysis is restricted to women with children aged less than 24 months, as this was asked alongside IYCF questions. Ag-nutrition vs 0.20* **

71 Table Women s nutritional status, by program group and survey round Indicators Underweight (< 18.5) Ag-Nutrition (n=978) Baseline 2011 Endline 2015 (n=1025) Control (n=1041) Ag-Nutrition (n=1212) (n=1244) Control (n=1080) ** *** Difference in difference vs control Ag- Control Nutrition T2-T1 T2-T1 T2-T1 Ag- Nutrition vs control Agnutrition vs *** ** 6.36*** -8.10* ** *** *** ** Mean Body mass index (3.10) (3.29) (3.72) (3.10) (3.15) (2.86) 0.28* 0.65*** (kg/m 2 ) Significant differences: *** p<0.001, ** p<0.01, * p<0.05; Double difference impact estimates with clustered standard errors comparing 2010 and Accounts for geographic clustering. Table Women s nutritional status among beneficiaries, by program group and survey round Indicators Underweight (< 18.5) Normal ( ) Overweight ( 25) Ag-Nutrition (n=978) Baseline 2011 Endline 2015 (n=1025) Control (n=1041) Ag-Nutrition (n=1212) (n=1244) Control (n=1080) ** *** Normal ( ) Overweight ( 25) Ag- Control Nutrition T2-T1 T2-T1 T2-T1 Ag- Nutrition vs control Difference in difference vs control Ag-nutrition vs *** ** 6.36*** -8.3* -13.2** *** *** ** Mean Body mass index (kg/m 2 ) (3.10) (3.29) (3.72) 23.21* (3.35) (3.17) (2.86) *** Significant differences: *** p<0.001, ** p<0.01, * p<0.05; +p<0.10 Double difference impact estimates with clustered standard errors comparing 2011 and Accounts for geographic clustering only. 71

72 5.4.3 Household characteristics: Food security, dietary diversity, socioeconomic status, and access to services KEY RESULTS: We assessed household food security, dietary diversity, socio-economic status, and access to services as underlying determinants of child nutrition. Overall, respondent s perception of their household food security decreased significantly over time for all groups, in both the full sample, and among confirmed RAIN beneficiaries. In impact analyses, there was a significant decrease in the prevalence of little to no hunger in the Ag-Nutrition group, compared to the control group, and a significant increase in the level of moderate hunger in this group. This was the case in both intent-to-treat analysis, and among confirmed RAIN beneficiaries. There was a significant positive program impact on the level of severe hunger ; DID impact estimates indicate reduction of between 4-7 pp in the Agonly group, compared to the control, in both intent-to-treat, and per-protocol analyses. At the same time, the Ag-Nutrition group had significant positive impacts on household dietary diversity, with an increase of about 1 food group, based on a 12 food group scale, in both intent-to-treat, and perprotocol analyses. Overall, there were no significant differences in socio-economic status or access to services between groups over time. Household perception of food security as measured by the Household Hunger Score is significantly worse in the ag-nutrition group in terms of responses to individual questions, hunger scores, and proportion in the category little or no hunger (Table ). The expanded Household Food Insecurity Access Scale was only undertaken at endline, but this also showed worse perception of hunger among the ag-nutrition group on less severe questions about hunger. In the beneficiary-only sub-sample, the trends and magnitudes were no different to the full group. Conversely, the Household Dietary Diversity Score shows household food security in terms of access to foods was significantly better by one food group over time in the ag-nutrition group compared to both other groups (Table ); HDDS decreased in all study arms, but decreased less in the ag-nutrition group. Household consumption of cereal, fish, oil, sweets and condiments decreased over time in all study arms; consumption of fruit increased in all study arms; and consumption of roots and legumes increased more in the intervention arms. In the final analysis, consumption of roots, legumes, vegetables, fruits, meat, eggs and milk were all better over time in the ag-nutrition group. Similar directions and magnitudes of effect were seen in the beneficiary-only sub-group analysis (Table ). There was no difference in socio-economic status (Table ) or in household assets (Table ) between groups at baseline or endline, though there was a significantly higher productive asset count in the intervention groups at endline. There were no differences between groups at endline in terms of access to services (Table ) except for lower toilet access in the ag-only group. 72

73 Table Household perceptions of food insecurity, by program group and survey round Impact indicators Baseline 2011 Endline 2015 Ag- Ag- Control Control Ag-Nutrition Control Nutrition Nutrition Ag- T2-T1 T2-T1 T2-T1 (n= 978) (n= 1025) (n= 1041) (n= 1212) (n= 1244) (n= 1080) Nutrition vs control Difference in difference vs control Agnutrition vs Worried about not having 69.22** enough food Unable to eat preferred foods 68.89** Ate just a few kinds of food 65.92*** Ate food that one did not 62.13*** want to eat Ate smaller meal 54.79*** Ate fewer meals 49.34*** No food of any kind in the *** *** 16.90*** 11.05*** 14.2** house Went to sleep at night hungry 22.82** *** * Spent a whole day without *** *** 8.71*** 9.40*** * eating anything Mean (SD) values on *** *** 0.28*** 0.25*** household hunger score (0.79) (0.93) (0.86) (1.42) (1.21) (1.30) Household hunger categories Little to no hunger *** *** *** -7.60*** -10.4* Moderate hunger *** 9.22*** ** 7.2* 5.7 Severe hunger *** *** * 2.0 Significant differences*** p<0.001, ** p<0.01, * p<0.05, +p<0.10; Double difference impact estimates with clustered standard errors comparing 2010 and Accounts for geographic clustering. 73

74 Table Household perceptions of food insecurity among beneficiaries, by program group and survey round Impact indicators Ag- Nutrition (n=978) Baseline 2011 Endline 2015 (n=1025) Control (n=1041) Ag- Nutrition (n=415) (n=385) Control (n=1080) Ag- Nutrition T2-T1 T2-T1 Control T2-T1 Ag- Nutrition vs control Difference in difference vs control Agnutrition vs Worried about not having 74.22*** enough food Unable to eat preferred foods 73.73*** Ate just a few kinds of food 70.36*** Ate food that one did not 65.06** want to eat Ate smaller meal 56.63** Ate fewer meals 51.57*** No food of any kind in the *** *** 14.52*** 11.05*** 12.9* house Went to sleep at night hungry 22.82** *** Spent a whole day without eating *** *** 5.79** 9.40*** anything Mean (SD) values on *** *** 0.17** 0.25*** household hunger score (0.79) (0.93) (0.86) (1.41) (1.09) (1.30) Household hunger categories Little to no hunger *** *** -7.5*** -7.6*** Moderate hunger *** 8.45*** * Severe hunger ** *** *** 3.0* Significant differences: *** p<0.001, ** p<0.01, * p<0.05, +p<0.10; Double difference impact estimates with clustered standard errors comparing 2010 and Accounts for geographic clustering. 74

75 Table Household dietary diversity, by program group and survey round Food dietary diversity experience 1 Baseline 2011 Endline 2015 Ag- Nutrition T2-T1 T2-T1 Control T2-T1 Ag- Ag- Control Control Nutrition Nutrition (n= 978) (n= 1025) (n= 1041) (n= 1212) (n= 1244) (n= 1080) Ag- Nutrition vs control Difference in difference vs control Agnutrition vs Cereals *** -1.36** -0.82* Roots and tubers 21.90*** * *** -3.94* *** 23.2*** ** Vegetables ** * * 6.6* * Fruits 9.62*** *** 39.45*** 29.43*** 9.7* 10.1* -0.3 Meat ** *** *** *** 11.0* ** Eggs 23.64* * *** *** 16.7** ** Fish/ sea foods 65.81*** ** *** *** *** Legumes, nuts and seeds ** *** 4.55* *** 18.9*** * Milk/ dairy products 30.50* ** -7.55*** ** Oils and fats 92.43*** ** *** *** *** -8.9** ** Sweets 66.63* * *** *** *** 13.8* -13.0* 26.8*** Spices, condiments and 97.24*** *** *** -4.91*** -4.34** * beverages Mean values on household dietary diversity (SD) (range 0-12) 6.74*** (1.97) 7.60 (2.17) 7.20 (2.33) 6.49*** (1.98) 6.38 (1.69) 6.04 (1.71) -0.24** -1.23*** -1.16*** 0.91** ** Significant differences: *** p<0.001, ** p<0.01, * p<0.05; Double difference impact estimates with clustered standard errors comparing 2010 and Accounts for geographic clustering. 1 Dietary diversity was based on reported consumption of foods in the previous 24 hours 75

76 Table Household dietary diversity among beneficiaries, by program group and survey round Food dietary diversity experience 1 Ag-Nutrition (n=978) Baseline 2011 Endline 2015 (n=1025) Control (n=1041) Ag-Nutrition (n=415) (n=385) Control (n=1080) Ag- Nutrition T2-T1 T2-T1 Control T2-T1 Ag- Nutrition vs control Difference in difference vs control Agnutrition vs Cereals *** -1.59** -0.82* Roots and tubers 21.90*** *** *** 25.1*** 12.9* 12.2* Vegetables * * Fruits 9.62*** *** 37.00*** 29.43*** Meat *** *** *** 10.7* * Eggs 23.64* *** *** 12.9* * Fish/ sea foods 65.81*** *** *** *** Legumes/nuts/pulses * ** -5.97* *** 18.8** * Milk/ dairy products 30.50* * -6.68* * Oils and fats 92.43*** *** -9.71*** *** Sweets 66.63* * *** *** *** 17.3* ** Spices, condiments and beverages 97.24*** *** *** -4.47*** -4.34** Mean values on household dietary diversity (SD) (range 0-12) 6.74*** (1.97) 7.60 (2.17) 7.20 (2.33) 6.51*** (1.89) 6.44 (1.71) 6.04 (1.71) -0.23** -1.16*** -1.16*** 0.93** ** Significant differences: *** p<0.001, ** p<0.01, * p<0.05, +p<0.10; Double difference impact estimates with clustered standard errors comparing 2010 and Accounts for geographic clustering. 1 Dietary diversity was based on reported consumption of foods in the previous 24 hours 76

77 Table Household asset ownership, by program group and survey round Ag-Nutrition (n=978) Baseline 2011 Endline 2015 Control Ag-Nutrition (n=1025) (n=1041) (n=1212) (n=1244) Control (n=1080) percent percent percent percent percent percent Ownership of house 95.08*** Durable goods count *** * Productive asset count Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Durable goods count ** (2.70) 5.02 (2.96) 5.34 (2.84) 4.77 (2.52) 4.84 (2.41) 4.71 (2.34) Total number of durable goods (6.25) 8.06 (6.47) 8.49 (6.80) 7.02** (4.85) 7.35 (4.94) 7.72 (5.37) Productive assets count 1.84 (1.03) 1.74 (1.02) 1.78 (1.06) 1.76*** (0.94) 1.70 (0.96) 1.32 (0.76) Total number of productive assets 8.58 (32.97) 8.49 (34.83) 7.40 (4.70) 6.34* (4.23) 8.79 (48.62) 5.40 (3.18) * P<0.05, **P<0.01, ***P< Durable goods count is the index that consists of a simple count of whether or not the households owned the assets listed (using yes = 1 and no = 0). The productive assets count is constructed the same way.examples of durable goods were car, motorcycle, bicycle, radio, television, refrigator, bed, matress, table, sofa, ect 2 Total number of durable goods is, as its name implies, the total number of each asset owned by the households. The total number of productive assets is constructed the same way. Examples of productive assets were plough/harrow, grain grinder/mill, Insecticide / herbicide crop spraye, water pump, hand threshing machin, motorized hammermill / thresher, tractor. 77

78 Table Household access to services, by program group and survey round Types of fuel used for cooking Ag-Nutrition (n=978) Baseline 2010 Endline 2015 Agricul-ture Control Ag-Nutrition Agricul-ture (n=1025) (n=1041) (n=1212) (n=1244) Control (n=1080) percent percent percent percent percent percent Fire wood 94.06** Charcoal Had electricity 0.51* Source of drinking water Unprotected well or river/ lake Borehole or tap Protected well Shared toilet Toilet facility Have no toilet 19.04*** *** Pit or VIP latrine Sanitary with flush * P<0.05, **P<0.01, ***P<0.001 Table Household socioeconomic status, by program group and survey round SES quintile Baseline 2010 Endline 2015 Ag-Nutrition (n= 978) (n= 1,025) Control (n= 1,041) Ag-Nutrition (n= 1,212) (n= 1,244) Control (n= 1,080) percent percent percent percent percent percent First Second Third Fourth Fifth Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) SES index 0.06 (0.97) (1.02) 0.04 (0.97) 0.01 (0.90) (0.93) (0.93) * P<0.05, **P<0.01, ***P<

79 6 Results: Impact of RAIN interventions on IYCF practices 6.1 Main impact analysis of RAIN interventions on IYCF outcomes KEY RESULTS: The second objective was to assess the impact of the RAIN package of interventions on core WHO infant and young child feeding (IYCF) indicators among children 0-23 months of age. In general, all breastfeeding-related IYCF indicators were high at endline, across all three study groups. With the exception of appropriate introduction of complementary foods at 6-8 months of age (which was approximately 95% across all three study groups), complementary feeding practices were sub-optimal ranging from approximately 25-30% for the minimum acceptable diet indicator, to 60% for the minimum meal frequency indicator. There were no differences between groups, at endline, in any of the core WHO IYCF indicators. Several of these IYCF indicators improved over time, within groups. Early initiation of breastfeeding improved across all three study arms, ranging from a pp increase over time, but this increase was not differential in favor of any study group. There were significant increases in several complementary feeding indicators (with increases ranging from 6 to 12 pp for different indicators, in the RAIN intervention groups), but these increases were not differential in favor of any group, either compared to each other, or the control group. Of note, consumption of iron rich foods decreased over time (ranging from 13-15pp), in all groups, but there was no differential change in favor of any study group. This pattern of no attributable program impact, compared to the control group, was consistent in both intent-to-treat and per protocol analyses. In terms of specific food groups, there were increases across arms and over time in consumption of legumes/nuts in both intervention arms compared to control. There were no discernable patterns in impact on breastfeeding indicators (Table and Table 6.1.2). At endline, age-appropriate breastfeeding, bottle feeding, and milk feeding for non-breastfed children were significantly different across study arms, and were all higher in the intervention arms. Early initiation of breastfeeding increased significantly and by large margins (25-30pp) in all three study arms (p<0.001). DID estimates showed predominant breastfeeding to be lower (-5.8pp) and bottle feeding to be higher (8.7pp) between the ag-only and control arms (p<0.05). No other breastfeeding results were significant. There were no significant changes between the intervention and control arms over time in any complementary feeding indicators (Table and Table 6.1.2). There were significant increases within the ag-nutrition arm over time in minimum dietary diversity (12.5pp), minimum meal frequency (10.6pp), and minimum acceptable diet (9.9pp) (p<0.001), but these were not significantly different to 79

80 the control group at endline. Consumption of iron-rich foods decreased by around 14pp in each study arm over time (p<0.001). In terms of food groups consumed (Table 6.1.4) we saw significant increases in intervention arms of dairy and eggs; we saw significant increases in all study arms of legumes/nuts, vitamin A rich fruits and vegetables; and we saw significant decreases in all study arms of animal foods, other fruits and vegetables. However, the only significant DID result across arms over time was increases of around 10pp in legumes/nuts in both intervention arms compared to control (p<0.05). 80

81 Table WHO recommended IYCF indicators, by program group and survey round Impact indicators Age group (months) n Baseline 2011 Endline 2015 Ag-Nutrition Control n Ag-Nutrition Control Core indicators Early initiation of breastfeeding ** Exclusive breastfeeding Continued breastfeeding at 1 year Introduction of solid, semi-solid or soft food Minimum diet diversity Minimum meal frequency * Minimum acceptable diet Consumption of iron-rich food Optional indicators Continued breastfeeding at 2 years Age-appropriate breastfeeding *** Predominant breastfeeding among * children <6 mo Bottle feeding * *** Milk feeding frequency for non-bf ( 2 milk * feedings/day) Significant differences: *** p<0.001, ** p<0.01, * p<0.05; 3 Minimum is defined as 2 times for breastfed infants 6-8 months; 3 times for breastfed children months; 4 times for non-breastfed children months. Meals include both meals and snacks and frequency is based on mother s report. 4 Acceptable diet is defined as who had at least the minimum dietary diversity and the minimum meal frequency during the previous day. 5 Iron-rich or iron-fortified foods include flesh foods, commercially fortified foods especially designed for infants and young children that contain iron, or foods fortified in the home with a micronutrient powder containing iron. 6 The asterisks associated with the A&T-I areas (2010 and 2014) are for the comparison between A&T-I and NI areas in the same round of survey 81

82 Table Difference in difference of WHO recommended IYCF indicators, by program group and survey round Impact indicators Age range Ag-Nutrition T2-T1 T2-T1 Control T2-T1 Ag-Nutrition vs control Difference in difference vs control Ag-nutrition vs Core indicators Early initiation of breastfeeding *** 23.63*** 25.14*** Exclusive breastfeeding Continued breastfeeding at 1 year Introduction of solid, semi-solid or soft food * Minimum diet diversity ( 4 food groups) *** 6.64* Minimum meal frequency ** * * Minimum acceptable diet *** Consumption of iron-rich food *** *** *** Optional indicators Continued breastfeeding at 2 years * * Age-appropriate breastfeeding ** 4.60* Predominant breastfeeding * * 5.40 Bottle feeding ** 7.44*** * Milk feeding frequency for non-bf ( 2 milk * feedings/day) * P<0.05, ** P<0.01, *** P< DID between Baseline and Endline adjusted for clustering effect at commune and district level only. 3 Minimum is defined as 2 times for breastfed infants 6-8 months; 3 times for breastfed children months; 4 times for non-breastfed children months. Meals include both meals and snacks and frequency is based on mother s report. 4 Acceptable diet is defined as who had at least the minimum dietary diversity and the minimum meal frequency during the previous day. 5 Iron-rich or iron-fortified foods include flesh foods, commercially fortified foods especially designed for infants and young children that contain iron, or foods fortified in the home with a micronutrient powder containing iron. 82

83 Table Pre-lacteal feeding among children 0-23 months old, by program group and survey round Baseline Impact indicators Ag-Nutrition Control Ag-Nutrition Control (n= 690) (n= 645) (n= 799) (n= 635) (n= 568) (n= 727) Fed colostrum Fed pre-lacteals during the first 3 days after birth * Things given to infants 3 days after birth - Honey Fruit juice Plain water Sugar Drink made from herbs or plants or gripe 8.33* water 24.24* Milk (other than breastmilk) Vaccine or other medicines *** Cooking or anointing oil 42.42*** Significant differences: *** p<0.001, ** p<0.01, * p<0.05; Endline 83

84 Indicators Table Food groups consumed in the past 24 hours among children 6-23 months old, by program group and survey round Baseline Endline Ag- Control Ag-Nutrition Control Nutrition (n= 478) (n= 463) (n= 595) (n= 433) (n= 401) (n= 498) Ag- Nutrition T2-T1 T2-T1 Control T2-T1 Ag- Nutrition vs control Difference in difference vs control Ag-nutrition vs Grains, roots and tubers * Legumes and nuts * *** 22.94*** 12.96*** 11.9* 10.00* 1.80 Dairy (milk, yogurt, cheese) *** 9.70** Animal foods (meat, fish, *** *** *** poultry and liver/organ meats) Eggs ** 8.79*** Vitamin A rich fruits and *** 13.77*** 17.13*** vegetables Other fruits and vegetables 19.53* *** *** *** * 9.00* Number of food groups 2.93 (1.20) 3.06 (1.17) 2.99 (1.22) Significant differences: *** p<0.001, ** p<0.01, * p<0.05, +p<0.10; 3.36* (1.21) 3.29 (1.20) 3.15 (1.05) 0.43*** 0.23** 0.16*

85 Table Timeliness of introduction of complementary foods among children 6-23 months old, by program group and survey round Baseline Endline Indicators Ag-Nutrition Control Ag-Nutrition Control (n= 478) (n= 463) (n= 595) (n= 433) (n= 401) (n= 498) Water *** Nshima or Other starchy foods ** Legumes 74.06** *** Green leafy vegetables 65.27** *** Orange fleshed vegetables *** Orange fleshed fruits 61.09* ** Other fruits 67.15* *** Meat *** Poultry *** Fish (Big) *** Fish (Small) ** Eggs *** Nuts *** Milk *** Table Reported meal frequency among children 6-23 months old, by program group and survey round Baseline Endline Indicators n Ag- Nutrition Control n Ag- Nutrition Control Number of meals and snacks for breastfed children ab months old child months old child months old child * a Adequate frequency of meal and snack for breastfed children : months: 2-3 meals, 1-2 snacks per day months: 3-4 meals, 1-2 snacks per day months: 3-4 meals, 1-2 snacks per day b Adequate frequency of meal and snack for non-breastfed children : give 1-2 additional meals per day 85

86 6.2 Plausibility analysis: Dose-response relationship between program exposures and IYCF practices KEY RESULTS: Findings for IYCF outcomes were similar among confirmed beneficiaries to the intent to treat analysis, though the magnitudes were sometimes greater. Continued breastfeeding at 2 years was significantly higher in ag-nutrition group, improved in this group over time, and the DID estimate was higher compared to control. Bottle feeding was higher over time in both intervention arms compared to control. There were larger significant increases within the ag-nutrition arm over time in minimum dietary diversity, and differences were significantly larger over time in the ag-nutrition arm compared to ag-only, but not compared to control. There were increases in minimum meal frequency and in minimum acceptable diet in intervention groups, but these were not significantly different to the control group at endline. Consumption of iron-rich foods decreased in all three arms. Findings for breastfeeding outcomes were similar among confirmed beneficiaries to the intent to treat analysis (Table and Table 6.2.2). Continued breastfeeding at 2 years was significantly higher in agnutrition group; improved 35pp in this group over time; and the DID estimate was 46pp higher compared to control (p<0.01). Age appropriate breastfeeding and bottle feeding were significantly higher in the ag-only group; bottle feeding was higher over time in both the ag-nutrition arm (7.7pp; p<0.05) and the ag-only arm (14.9pp; p<0.01) compared to control. Directions of change were similar for complementary feeding indicators among confirmed beneficiaries, but magnitudes were different (Table and Table 6.2.2). There were larger significant increases within the ag-nutrition arm over time in minimum dietary diversity (14.4pp); the DID estimates show that differences were significantly larger over time in the ag-nutrition arm compared to ag-only, but not compared to control. There were also increases in minimum meal frequency (12.5pp) (p<0.01), and in minimum acceptable diet (9.4pp) (p<0.05), but these were not significantly different to the control group at endline. Consumption of iron-rich foods decreased by around 15pp in the ag-nutrition and control arms, and by 22p in the ag-only arm over time (p<0.001). 86

87 Table WHO recommended IYCF indicators among beneficiaries (those are the member of a RAIN s women s group) by program group and survey round Impact indicators Age group (months) n Baseline 2011 Endline 2015 Ag-Nutrition Control n Ag-Nutrition Control Core indicators Early initiation of breastfeeding ** Exclusive breastfeeding Continued breastfeeding at 1 year Introduction of solid, semi-solid or soft food Minimum diet diversity Minimum meal frequency * Minimum acceptable diet Consumption of iron-rich food Optional indicators Continued breastfeeding at 2 years *** Age-appropriate breastfeeding ** Predominant breastfeeding among * children <6 mo Bottle feeding * *** Milk feeding frequency for non-bf ( milk feedings/day) Significant differences: *** p<0.001, ** p<0.01, * p<0.05; 3 Minimum is defined as 2 times for breastfed infants 6-8 months; 3 times for breastfed children months; 4 times for non-breastfed children months. Meals include both meals and snacks and frequency is based on mother s report. 4 Acceptable diet is defined as who had at least the minimum dietary diversity and the minimum meal frequency during the previous day. 5 Iron-rich or iron-fortified foods include flesh foods, commercially fortified foods especially designed for infants and young children that contain iron, or foods fortified in the home with a micronutrient powder containing iron. 6 The asterisks associated with the A&T-I areas (2010 and 2014) are for the comparison between A&T-I and NI areas in the same round of survey 87

88 Table Difference in difference of WHO recommended IYCF indicators among beneficiaries (those are the member of a RAIN s women s group) Impact indicators Age range Ag-Nutrition T2-T1 T2-T1 Control T2-T1 Ag-Nutrition vs control Difference in difference vs control Ag-nutrition vs Core indicators Early initiation of breastfeeding *** 27.83*** 25.12*** Exclusive breastfeeding Continued breastfeeding at 1 year Introduction of solid, semi-solid or soft food * Minimum diet diversity ( 4 food groups) ** * Minimum meal frequency ** * Minimum acceptable diet * Consumption of iron-rich food * *** *** Optional indicators Continued breastfeeding at 2 years ** ** Age-appropriate breastfeeding ** 6.72* Predominant breastfeeding Bottle feeding *** 13.61*** * 14.90** Milk feeding frequency for non-bf ( 2 milk feedings/day) * P<0.05, ** P<0.01, *** P< DID between Baseline and Endline adjusted for clustering effect at commune and district level only. 3 Minimum is defined as 2 times for breastfed infants 6-8 months; 3 times for breastfed children months; 4 times for non-breastfed children months. Meals include both meals and snacks and frequency is based on mother s report. 4 Acceptable diet is defined as who had at least the minimum dietary diversity and the minimum meal frequency during the previous day. 5 Iron-rich or iron-fortified foods include flesh foods, commercially fortified foods especially designed for infants and young children that contain iron, or foods fortified in the home with a micronutrient powder containing iron. 88

89 7 Results: Impact of RAIN interventions on knowledge among caregivers 7.1 Impact RAIN package of intervention on nutrition and hygiene knowledge KEY RESULTS: The third objective was to assess the impact of the RAIN package of intervention on health and nutrition knowledge among caregivers. As with IYCF practices, patterns in changes of IYCF knowledge were inconsistent, in both intent to treat and per-protocol analyses. Overall, IYCF knowledge had increased over time in the ag-nutrition group compared to control on some information and decreased on other information. In general, in DID impact analyses, improvements in breastfeeding knowledge over time, was lower in the Ag-only group, when compared to the control group. There were no other differential changes in breastfeeding knowledge over time. As with IYCF practices, overall there was a significant difference in knowledge regarding timely introduction of different foods at 6-8 months, in the Ag-Nutrition arm, compared to the control arm; this was most notable for animal source foods (flesh foods and eggs), where there was an approximate 20pp greater knowledge regarding the appropriateness of feeding these foods to children 6-8 months of age. Hygiene knowledge was significantly different among study arms at endline for many aspects relating to hand washing; protecting children from worms; and making drinking water safer, but the direction of difference was not consistently favorable to a single study arm, or to the RAIN intervention arms.. There were some changes in breastfeeding knowledge, but patterns are difficult to define (Table 7.1.1). Overall breastfeeding knowledge scores were significantly worse over time in the ag-only group compared to control (-0.41; p<0.05), and differences over time were not significant between other study arms. DID estimates show that knowledge had increased over time in the ag-nutrition group compared to control on expressing breastmilk (p<0.01) and breastfeeding to 24 months (p<0.05); but had decreased over time on the giving of colostrum; giving other liquids or foods during the first six months; and continuing to breastfeed when the mother is ill (p<0.05). There is a scattering of other significant results in the intent to treat analysis, but no discernable patterns in findings. Among project beneficiaries, results were similar, in the same directions, but magnitudes a few percentage points larger (Table 7.1.2). Overall scores on knowledge of when to introduce different complementary foods increased slightly in all study arms over time, and were significantly higher at endline in the ag-nutrition group (p<0.001) (Table 7.1.3). There were no significant differences in knowledge on feeding during illness (Table 7.1.4) or on exposure to IYCF information (Table 7.1.5). 89

90 Table Knowledge about BF among mothers of children months of age, by program area Impact indicators Baseline Endline Ag- Control Ag-Nutrition Control Nutrition (n= 690) (n= 645) (n= 799) (n= 635) (n= 690) (n= 645) Ag- Nutrition T2-T1 T2-T1 Control T2-T1 Difference in difference Ag- Nutrition vs vs control control Ag-nutrition vs Baby should be breastfed *** 12.13*** 16.27*** immediately after birth Mother should give colostrum 90.43** *** 5.99*** 11.28*** -6.40* -5.30* to her baby Baby should be breastfed 61.39** *** * *** *** 16.7** whenever he/she wants Mother should breastfeed 36.92*** *** more often/ more frequently Mother should not give other 60.47** ** * 6.99* 19.36*** -13.4* -12.4* -1.0 liquids/ foods even when she thinks the baby is not getting enough breastmilk Continuation of BF if the 94.91*** *** *** * mother is ill Mother should express 6.69*** * *** 18.07*** 5.98** 12.70** 12.1** 0.6 breastmilk to feed her baby in certain circumstances Baby should be breastfed at 38.56** *** *** 14.8* 16.3** months or above Breastfeeding score 4.71*** *** *** ** 0.20 (range 0-8) (1.17) (1.09) (1.20) (1.12) (1.05) (1.02) *** p<0.001, ** p<0.01, * p<0.05, +p<0.10; 1 DID between Baseline and Endline 90

91 Table Knowledge about BF among mothers who are members of RAIN group with children months of age, by program and survey round Impact indicators Baby should be breastfed immediately after birth Mother should give colostrum to her baby Baby should be breastfed whenever he/she wants Mother should breastfeed more often/ more frequently Mother should not give other liquids/ foods even when she thinks the baby is not getting enough breastmilk Continuation of BF if the mother is ill Mother should express breastmilk to feed her baby in certain circumstances Baby should be breastfed at 24 months or above Breastfeeding score Baseline Endline Ag- Control Ag-Nutrition Control Nutrition (n= ) (n= ) (n= ) (n= ) (n= ) (n= ) Ag- Nutrition T2-T1 T2-T1 Control T2-T1 Difference in difference Ag- Nutrition vs vs control control Ag-nutrition vs *** 11.63*** 16.27*** ** ** 7.10** 11.28*** ** *** * *** * *** 16.40* 36.92*** *** ** *** *** * *** *** ** ** *** ** *** 17.53*** 5.98** 15.50** 11.50* ** *** *** 18.60** 16.20* *** (1.17) *P<0.05, **P<0.01, ***P< DID between Baseline and Endline 4.80 (1.09) 4.51 (1.20) 5.02* (1.13) 4.82 (1.02) 5.04 (1.02) 0.31*** *** **

92 Table Reported knowledge on timeliness of introduction of complementary foods among mothers with children months, by program group and survey round Baseline Endline Indicators Ag-Nutrition Control Ag-Nutrition Control (n= 690) (n= 645) (n= 799) (n= 635) (n= 690) (n= 645) Water Nshima or other starchy foods 90.43** * Legumes 85.07** ** Green leafy vegetables 76.96** *** Orange fleshed vegetables 76.38** ** Orange fleshed fruits 74.49* ** Other fruits 77.97** ** Meat *** Poultry *** Fish (Big) ** Fish (Small) ** Eggs *** Nuts *** Milk 81.30* *** Knowledge score (range 0-14) ** (3.34) 9.41 (3.48) 9.00 (3.67) 12.52*** (2.59) (3.31) *P<0.05, **P<0.01, ***P< Each food for which respondents correctly identified the time for introduction scores 1- so 14 foods get 14 score (3.45) 92

93 Table Reported knowledge on feeding during illness among mothers with children months, by program group and survey round Baseline Endline Indicators Ag-Nutrition Control Ag-Nutrition Control (n= 690) (n= 645) (n= 799) (n= 635) (n= 690) (n= 645) What should be done when a child has diarrhea Give ORS *** Feed less than usual ** Feed as much food as usual * Feed more than usual Give less liquids than usual Give as much liquids as usual 5.80** Give more liquids than usual *** Continue breastfeeding Breastfeed more often ** Give syrups 1.30* * Give traditional medicine Give treated water ** Give Zinc * See health professional 68.70** *** *P<0.05, **P<0.01, ***P<

94 Table Reported exposure to information, by program group Baseline Endline Ever heard these information Ag-Nutrition Control Ag-Nutrition Control (n= 978) (n= 1025) (n= 1041) (n= 1212) (n= 1244) (n= 1080) Putting baby to breast immediately after birth Giving only colostrum in the first day or two until breastmilk comes in Not putting anything into the child mouth before breastmilk or colostrum (no pre-lacteal, including anointing oil) 74.74* *** Feed only breastmilk up to six months Not giving the child any water or other liquids up to six months How to hold the baby at the breast so he/ she can breastfed easily 71.88* How to put the baby s mouth at the breast so that he/she can breastfeed easily 65.54** * Emptying one breast before giving to the other side 41.82* Feeding mashed family food after 6 months 86.91** Feeding eggs, meat, fish, and other animal source foods to children older than 6 months Cooking children s food with oil (or adding oil to children s food) *** Adding vitamin and mineral powder to the child s food 49.28** *** Washing hand with water and soap before food preparation or feeding the child Washing hand with water and soap after using the toilet, defecating or urinating, or after cleaning a child who has defecated How to help your child eat better 78.22*** ** How to feed your child when he/she is sick 69.40*** ** Feeding the child an extra meal, or extra food after illness *** *P<0.05, **P<0.01, ***P<

95 Table Reported knowledge of hygiene practices, by program group Baseline Endline Indicators Ag-Nutrition Control Ag-Nutrition Control (n= 978) (n= 1025) (n= 1041) (n= 1212) (n= 1244) (n= 1080) When should a person wash their hands Before eating 75.33* Before cooking or food preparation After using the toilet /latrine 70.21*** * After cleaning a child who has defecated 32.26*** Before and after taking care of a sick person 7.92* Before feeding a child 62.13*** ** Ways to protect a child from worms Wash hands of child 29.04** *** Wash hands before preparing food and feeding 21.47*** ** child Cut nails *** Children should wear pants 5.62*** *** Wash fruits and vegetables 10.63** * Children should wear sandals Give them treated water 12.27** ** Tablets/medicines 62.27*** *** Ways to make drinking water safer Boil water Treat with chlorine, Clorin, bleach 90.49*** ** Strain through cloth *** Use ceramic, sand, or other filter 0.61* *** Solar disinfection Let it stand and settle 13.39*** *** *P<0.05, **P<0.01, ***P<

96 8 Results: Impacts of RAIN interventions on the women s empowerment KEY RESULTS: The fourth objective was to assess the impact of the RAIN package of intervention on women s empowerment. We assessed impacts on eight social and economic domains of women s empowerment, plus empowerment in agriculture. There were clear impacts of the RAIN interventions on different domains of women s empowerment, but there was no consistent impact across all domains, in favor of a single group. Over time, there were significant increases in several domains of women s empowerment, including scores for buying power, social capital, and decision making. However, there was no consistent pattern of change over time, across groups. In DID impact analyses, we observe significant program impacts of the Ag-only group, compared to the control group on asset access, financial empowerment, social capital, and a composite score of women s empowerment. The Ag Nutrition group had significant program impacts, when compared to the control group on financial empowerment, and social capital. The Ag-only group had greater impacts on different domains of empowerment, compared to the Ag-Nutrition group; these impacts were fairly consistent across 5 of the 8 different domains examined. Overall, the direction of impacts was similar in intentto-treat, and per-protocol analyses, with slightly higher impacts in per-protocol analyses. There was a clear shift over time in women s involvement in decision making in agriculture, away from having men only involved towards having women involved (either solely, or jointly with her spouse). This shift occurred across all study groups, but for different aspects of decision making (e.g. for crop production, use of money from sale of crops, as well as in a composite score of women s empowerment in agriculture), the change was greater in the RAIN intervention groups compared to the control group, suggesting a clear impact of the RAIN interventions on improving women s empowerment in agriculture. These impacts were greater still in per-protocol analysis. 96

97 8.1 Women s social empowerment There were changes in some aspects of social empowerment for women, but there was no consistent pattern of change. There are some significant differences between groups at baseline and endline on some individual variables describing communication between husband and wife (Table 8.1.1), however, no differential impact among intervention arms was found for overall relationship score, either in the whole sample (Table 8.2.4) or among RAIN benefceries (Table 8.2.5). Improved communication does not seem to add up to a feeling of greater equality among women (Table 8.1.2) or greater decision making power for women in the household ( 97

98 Table ), though intervention groups do agree more than control with statements supporting equality 4 and reporting decision-making power over certain nutrition-related decisions 5. Social capital scores across domains of social support and social networking are significantly higher in the intervention groups (Table 8.1.4); upwards of 70% of women have someone outside of their immediate household who could help them out if a significant problem occurred (no significant differences across study arms), and women meet to discuss key personal and social issues significantly more in intervention areas, though still fewer than 45% of women regularly meet with other women in the community to discuss issues or share information. DID shows higher impacts of the intervention on social capital scores for the whole sample (DID ) (Table 8.2.4) and even higher among RAIN benefceries (DID ) (Table 8.2.5). 4 If the woman works outside home, her husband or partner should help her with the daily housework and A husband should not let his wife work outside home, even if she would like to do it. 5 Whether or not you breastfeed the child and when to wean the child and What and how to feed the infant in his/her first year of life 98

99 Table Relationship with spouses, by program group and survey round Percent often communicating with spouse on the following subjects: Baseline Endline Ag-Nutrition Control Ag-Nutrition Control (n= 978) (n= 1025) (n= 1041) (n= 1212) (n= 1244) (n= 1080) Work activities/agricultural activities 65.24* * What happens at home 66.05** Expenditures 59.30** * What happens in community or area Your child s health 63.80** Your own health 60.84** Child feeding 62.78** Family planning Spouse relationship score *** *** 0.59 (0.43) (0.41) (0.44) (0.40) (0.41) *P<0.05, **P<0.01, ***P< This score is an average over the eight indicator variables presented here. The higher score means higher empowerment Table Perception of equality, by program group and survey round Percent who agree with the following statements: In a household, the man should make the important decisions 1. If the woman works outside home, her husband or partner should help her with the daily housework A husband should not let his wife work outside home, even if she would like to do it 1.. A woman has the right to express her opinion if she does not agree with what the husband or partner says A woman must accept that her husband or partner beats her in order to keep the family together 1. It is better to send a son to school than a daughter 1. Perception of equality score * Baseline Endline 0.54 (0.41) Ag-Nutrition Control Ag-Nutrition Control (n= 978) (n= 1025) (n= 1041) (n= 1212) (n= 1244) (n= 1080) 70.18* *** * *** ** ** (0.17) 0.71 (0.19) 0.65*** (0.18) (0.17) *P<0.05, **P<0.01, ***P< These items were reverse coded before creating the overall score 2 This score is an average over the six indicator variables presented here. The higher score means higher empowerment 0.70 (0.19) 0.64 (0.17) 99

100 Table Decision making power, by program group and survey round Baseline Endline Percent who can decide on her own or with the spouse the following: Ag-Nutrition (n= 978) (n= 1025) Control (n= 1041) Ag-Nutrition (n= 1212) (n= 1244) Control (n= 1080) Buying important things for the family 43.95*** What food is prepared every day 90.88*** If you have to work to earn money Visiting other family members, friends or relatives 78.59* Seeing a doctor or visiting a dispensary when you are pregnant Use of family planning methods Sending your child/children to school What to do when a child is ill 87.09* How to make children listen or obey Having another child or not Whether or not you breastfeed the child and when to wean the child ** What and how to feed the infant in his/her first year of life ** Decision making scores 1 (SD) 0.79*** (0.25) 0.72 (0.27) 0.74 (0.31) 0.83* (0.26) 0.86 (0.21) *P<0.05, **P<0.01, ***P< This score is an average over the twelve indicator variables presented here. The higher score means higher empowerment 0.85 (0.25) 100

101 Table Women s social capital, by program group and survey round Baseline Endline Ag-Nutrition Control Ag-Nutrition Control (n= 978) (n= 1025) (n= 1041) (n= 1212) (n= 1244) (n= 1080) Percent have somebody who could help when needed or when having problems To accommodate you for several nights if you need it To help you out with money or lend you some money To help when you don t have enough food at home To talk to when you have a problem Percent usually meet with other women to discuss any of the following Education problems 19.90*** * Health problems 21.64*** *** Problems of women 24.31*** *** Problems of the community 23.69*** *** To receive information on health and nutrition 21.74*** *** Social capital scores *** 0.36 (0.26) *** 0.59 (0.30) (0.32) (0.25) (0.24) *P<0.05, **P<0.01, ***P< This score is an average over the nine indicator variables presented here. The higher score means higher empowerment 0.49 (0.24) 101

102 8.2 Women s economic empowerment As with social empowerment, impacts on women s economic empowerment were not consistent. Women s overall financial empowerment has changed little since baseline, and is slightly worse in intervention areas (Table 8.2.1). Access to assets for women is slightly better in intervention areas however (Table 8.2.2) and in specific domains more closely related to the project s aims (ownership of animals) and long-term female-specific savings (ownership of jewellery) the intervention areas are doing significantly better. However, ability to make the decision to dispose of these assets is limited, with less than 45% of women reporting ability to sell jewelery, less than 25% able to sell animals, and less than 10% able to sell land. Access to assets has not therefore translated into empowerment to make purchases (Table 8.2.3): At endline, women s empowerment to make purchasing decisions and buy household items appears to have improved in many domains, but has declined relative to control. There are few significant differences among study arms on economic empowerment, and where differences are significant they are in favor of the control group. Overall socres for women s economic empowerment domains changed very little over time and between study arms (Table 8.2.4), mostly at less than 0.1 units of scores in the difference-in-difference analysis; where there was change, it was generally better in the agriculture-only arm than the agnutrition arm and the control. The story was similar when restricting the analysis to beneficiaries only (Table 8.2.5). Table Financial empowerment, by program group and survey round Baseline Endline Ag-Nutrition Control Ag-Nutrition Control Percent of women who: (n= 978) (n= 1025) (n= 1041) (n= 1212) (n= 1244) (n= 1080) Have own money that can be used when want it Know a project that can lend 13.04*** money for women to start or extend business Benefit from loan to start or 2.45*** extend business Finance empowerment score *** 0.24 (0.23) * (0.23) (0.27) (0.21) (0.19) (0.21) *P<0.05, **P<0.01, ***P< This score is an average over the three indicator variables presented here. The higher score means higher empowerment 102

103 Table Access to assets and ability to sell assets, by program group and survey round Baseline Endline Ag-Nutrition Control Ag-Nutrition Control (n= 978) (n= 1025) (n= 1041) (n= 1212) (n= 1244) (n= 1080) Percent possess alone or together with somebody else the following things: Land * This house or the house you usually live ** Another house Animals like cow, goat, pig 46.00* ** Small animals or poultry, like rabbits, ducks, chickens 73.46* *** Jewelry *** Motorbike/bicycle ** Asset access score *** *** (0.26) (0.28) (0.26) (0.27) (0.24) (0.22) Percent able to sell the following things without the authorization of someone else Land * This house or the house you usually live ** Another house Animals like cow, goat, pig * Small animals or poultry, like rabbits, ducks, chickens Jewelry Motorbike/bicycle 10.22* Asset selling score ** ** 0.14 (0.35) (0.31) (0.33) (0.29) (0.29) *P<0.05, **P<0.01, ***P< This score is an average over the seven indicator variables presented here. The higher score means higher empowerment 0.18 (0.33) 103

104 Table Purchasing decisions, by program group and survey round Baseline Endline Ag-Nutrition Control Ag-Nutrition Control (n= 978) (n= 1025) (n= 1041) (n= 1212) (n= 1244) (n= 1080) Women usually decide on buying: Maize meal, Rice, legumes, etc 43.72** Meat, fish, poultry, etc ** Fruits and vegetables 75.85** Packaged products (breads, snacks, etc) 65.16* Cooking Oil 68.66* Special foods for child 72.10** * Other food 67.88** Milk/yogurt 70.64* Medicines 65.67* A woman can decide on her own to buy: Small amounts of food like rice, vegetables, and beans Bigger amounts of food like a bag of rice? *** Clothes for yourself? *** Medicine for yourself? Toilet articles for yourself like soap, toothpaste * Clothes for the children ** Medicine for the children Special foods for your children * Purchase decision scores *** *** (0.28) (0.29) (0.29) (0.25) (0.27) (0.24) *P<0.05, **P<0.01, ***P< This score is an average over the seventeen indicator variables presented here. The higher score means higher empowerment 104

105 Table Summary key domains of women empowerment (all women), by program group and survey round Women s empowerment domains: Social capital score Decision making power Spouse relationship score Perception of equality score Asset access score Assess selling score Buying power score Financial empowerment score *P<0.05, **P<0.01, ***P<0.001 Baseline Control Ag- Control Nutrition Ag- (n= (n=1080) T2-T1 T2-T1 T2-T1 1244) Ag-Nutrition Control Ag- Nutrition (n= 978) (n= 1025) (n= 1041) (n= 1212) 0.43*** (0.30) 0.79*** (0.25) 0.60*** (0.43) 0.73* (0.17) 0.47*** (0.26) 0.20** (0.35) 0.67*** (0.28) 0.24*** (0.23) 0.36 (0.26) 0.72 (0.27) 0.62 (0.41) 0.72 (0.17) 0.42 (0.28) 0.15 (0.31) 0.61 (0.29) 0.24 (0.23) 0.47 (0.32) 0.74 (0.31) 0.54 (0.44) 0.71 (0.19) 0.48 (0.26) 0.19 (0.33) 0.61 (0.29) 0.29 (0.27) 0.58*** (0.25) 0.83* (0.26) 0.63*** (0.40) 0.65*** (0.18) 0.53*** (0.27) 0.14** (0.29) 0.76*** (0.25) 0.23* (0.21) Endline 0.59 (0.24) 0.86 (0.21) 0.59 (0.41) 0.70 (0.19) 0.53 (0.24) 0.14 (0.29) 0.77 (0.27) 0.24 (0.19) 0.49 (0.24) 0.85 (0.25) 0.54 (0.41) 0.64 (0.17) 0.50 (0.22) 0.18 (0.33) 0.81 (0.24) 0.25 (0.21) Nutrition vs control Difference in difference vs control Agnutrition vs 0.15*** 0.23*** *** 0.21*** -0.09** 0.04*** 0.13*** 0.10*** -0.07* ** *** -0.02** -0.07*** * -0.06** 0.05*** 0.11*** ** *** *** 0.16*** 0.20*** -0.11** * *** *

106 Table Summary key domains of women empowerment (among RAIN beneficeries), by program group and survey round Women s empowerment domains: Social capital score Decision making power Spouse relationship score Perception of equality score Asset access score Assess selling score Buying power score Financial empowerment score *P<0.05, **P<0.01, ***P<0.001 Baseline Endline Ag- Ag- Control Ag-Nutrition Control Control Nutrition Nutrition Ag- (n= 978) (n= 1025) (n= 1041) (n=) (n= ) (n=1080) T2-T1 T2-T1 T2-T1 0.43*** (0.30) 0.79*** (0.25) 0.60*** (0.43) 0.73* (0.17) 0.47*** (0.26) 0.20** (0.35) 0.67*** (0.28) 0.24*** (0.23) 0.36 (0.26) 0.72 (0.27) 0.62 (0.41) 0.72 (0.17) 0.42 (0.28) 0.15 (0.31) 0.61 (0.29) 0.24 (0.23) 0.47 (0.32) 0.74 (0.31) 0.54 (0.44) 0.71 (0.19) 0.48 (0.26) 0.19 (0.33) 0.61 (0.29) 0.29 (0.27) 0.60*** (0.24) 0.83 (0.25) 0.64*** (0.41) 0.64*** (0.18) 0.51*** (0.25) 0.15 (0.31) 0.75*** (0.25) 0.25 (0.22) 0.63 (0.24) 0.86 (0.20) 0.62 (0.40) 0.69 (0.19) 0.55 (0.24) 0.14 (0.30) 0.78 (0.26) 0.25 (0.20) 0.49 (0.24) 0.85 (0.25) 0.54 (0.41) 0.64 (0.17) 0.50 (0.22) 0.18 (0.33) 0.81 (0.24) 0.25 (0.21) Nutrition vs control Difference in difference vs control Agnutrition vs 0.17*** 0.28*** *** 0.26*** -0.11*** 0.04** 0.14*** 0.10*** -0.06* ** *** -0.03** -0.07*** ** 0.04* 0.13*** ** -0.10** -0.05* *** 0.16*** 0.20*** -0.12** * *** 0.06** 0.05*

107 8.3 Women s empowerment in agriculture at endline Women s empowerment in agriculture was measured across seven domains, looking at whether a woman had input (either alone or jointly with her partner) on agriculture-related decisions. In a composite score across all seven domains, women were significantly more empowered in the two intervention arms than the control arm in the intent to treat analysis (Table 8.3.1), and both the magnitides of improvement and the statistical significance were greater in the per-protocol analysis, among beneficiaries (Table 8.3.2). Spcifically, women in the intervention arms were significantly more empowered in the domains of deciding what to grow on land, and spending money from the sale of field crops; women in these arms were significantly less empowered than control however in spending money from the sale of animals (Table 8.3.1), and again the magnitudes for each of these findings were greater among beneficiaries (Table 8.3.2). 107

108 Table Women's empowerment in agriculture, by program group and survey round Baseline Endline Difference in difference Proportion reporting participation (self or joint) in decisions on: Ag- Control Ag-Nutrition Control Ag-Nutrition Control Nutrition Ag- T2-T1 T2-T1 T2-T1 Nutrition (n= 978) (n= 1025) (n= 1041) (n= 1212) (n= 1244) (n= 1080) vs control vs control What to grow on land 30.16*** *** *** 44.91*** 32.09*** 11.3* 12.6* -1.5 What to do with field crops 43.05*** ** *** 39.07*** 29.91*** Ag-nutriti vs What to do with fruits and ** *** 41.15*** 41.08*** vegetables Spending money from sale of 25.26*** *** *** 36.63*** 27.50*** 10.3* 9.0* 1.3 field crops Spending money from sale of *** 31.17*** 37.30*** fruits and vegetables Spending money from sale of 46.03** *** *** -23.2*** -21.3*** -2.1 animals Spending money from sale of *** *** *** animal products Total score of Women 1.45*** *** *** 1.36*** 0.97*** 0.43* empowerment in (0-7) Significant differences: *** p<0.001, ** p<0.01, * p<0.05, +p<0.10; Double difference impact estimates with clustered standard errors comparing 2011 and Accounts for geographic clustering. 108

109 Table Women's empowerment in agriculture among RAIN beneficiaries, by program group and survey round Baseline Endline Difference in difference Proportion reporting participation (self or joint) in decisions on: Ag- Control Ag-Nutrition Control Ag-Nutrition Control Nutrition Ag- T2-T1 T2-T1 T2-T1 Nutrition (n= 978) (n= 1025) (n= 1041) (n= 415) (n=385) (n= 1080) vs control vs control What to grow on land 30.16*** *** *** 57.01*** 32.09*** 20.0*** 24.9*** -5.0 What to do with field crops 43.05*** *** *** 40.40*** 29.91*** Ag-nutriti vs What to do with fruits and *** *** 51.26*** 41.08*** vegetables Spending money from sale of 25.26*** *** *** 44.00*** 27.50*** 13.7** 16.4** -2.9 field crops Spending money from sale of *** 35.18*** 37.30*** fruits and vegetables Spending money from sale of 46.03** *** -24.8*** -23.2** -1.8 animals Spending money from sale of *** *** *** animal products Total score of Women 1.45*** *** *** 2.21*** 0.97*** 1.15*** 1.23*** empowerment in (0-7) Significant differences: *** p<0.001, ** p<0.01, * p<0.05, +p<0.10; Double difference impact estimates with clustered standard errors comparing 2011 and Accounts for geographic clustering only. 109

110 9 Results: Impact of RAIN interventions on production of nutrient-rich foods 9.1 Food production KEY RESULTS: The fifth objective was to assess the impact of the RAIN agriculture package of interventions on the availability of and access to a year-round supply of diverse and micronutrient-rich plant and animal source foods at household level. Overall the RAIN interventions had a consistent significant attributable impact on several different dimensions of agricultural production and consequent availability during the year of nutritious foods. Both the Ag-Nutrition and the Ag-only arms, had greater increases over time, compared to the control group, on the total number of foods produced (constructed using different indices), the total number of agricultural activities engaged in by the households, and the number of months producing Vitamin A rich foods, and dairy. There were no significant program impact differences on these outcomes, between the two RAIN intervention groups. Consistently, program effect sizes were approximately two-fold larger in per-protocol analyses. The dominant field crops in the study area are maize, groundnuts, and seed cotton with less than 5% of households planting other specific field crops (Table 9.1.1). At endline, all households surveyed are engaged in cultivating a field crop and the average number of crops is modestly higher in the treatment areas but lower in control areas. Production of garden crops, too, is concentrated on a relatively small number of vegetables and fruits. For example, at baseline only two crops, rape and tomato, are produced by more than 10% of households (Table 9.1.2). Both of these, as well as in particular other fruits, are somewhat more common by endline. Nearly all households also own animals or poultry (Table 9.1.3). Here again, there is a concentration in similar activities with most (90%) keeping chickens, about half with cattle, and a little less than half with goats or sheep. The variety of different agricultural activities carried out by RAIN households makes assessment of the program using specific individual activities difficult; instead, we begin an assessment of program impacts by categorizing agricultural outputs in line with the program design. The first way of doing this is shown in Table 9.1.4, organizing production into seven different food groups. At baseline, most households are producing grains, about a third producing pulses, and one-half producing eggs, with smaller percentages producing in the other categories. At endline, production in treatment areas is maintained or increased across all of these food groups; in control groups, in contrast, it declines in all but grains. 110

111 With this context, we next examine the impact of RAIN on agricultural production using the research design and difference-in-differences analyses. Overall, there is little difference in whether households participated in agriculture at all, though RAIN households do have more plots in 2015 and are cultivating greater overall plot area (not shown). Table presents the intent to treat estimates of the full sample and Table the estimates for households who reported participating in RAIN. In the first five rows we present measures of diversity in agricultural production; in the bottom four rows, access to different food types during the year. Baseline values make clear the relatively low level of diversity in agricultural production in field and, especially, garden crops. The double difference estimators indicate large and statistically significant increases in that diversity due to RAIN, increasing the number of total crops by nearly 1.5, with much of this increase concentrated in garden crops. Even with this increase, however, diversity remains low. At least part, and on average approximately half, of the double-difference improvement stems from unexplained declines in the control group over time. In analyses among beneficiary households (Table 9.1.6), the impact estimates are nearly double, and the share explained by declines in the control group much smaller. Together, the analyses suggest that the RAIN interventions increase diversity in agricultural production. The comparatively high change in per protocol compared to intent to treat suggests that the interventions, and not merely changes in the control group, drive increases in diversity of agricultural production. Consistent with the increase in crops, there was a concurrent increase in the number of food groups (from Table 9.1.4) and the number of agricultural activities carried out. With this evidence of increased diversity of production and potential availability in hand, we turn to an examination of the availability of nutritious foods throughout the year, as targeted by the program. Table indicates that the number of months home-produced fruits and vegetables rich in vitamin A are available increases by more than one and in beneficiary households this increase is doubled. Smaller, and significant, increases are also seen in the availability of dairy though there is no evidence of increased availability of meat or eggs despite a small increase in the probability of keeping chickens due to the program (not shown). The difference in estimated program effects by program arm ( versus and Nutrition) are generally small and both positive and negative across the different measures. In only one case in Table (and two in Table 9.1.6) are they significantly different, leading us to conclude that overall the program arms had largely similar impacts on the agricultural outcomes measured. 111

112 Table Number of field crops cultivated, by program group and survey round Baseline Endline Ag-Nutrition (n= 978) (n= 1025) Control (n= 1041) Ag-Nutrition (n= 1212) (n=1244) Control (n= 1080) Field crops: Maize Sorghum Finger millet Pearl millet/ bulrush Rice Groundnuts Soyabeans Seed cotton Irish potato Virgina tobacco Burley tobacco Mixed beans Bambara nuts Cowpeas Velvet beans Coffee Sweet potatoes Cassava Kenaf Cashew nut Paprika Wheat Sugar cane Sunflower Other Growing any field crops 92.54*** * P<0.05, ** P<0.01, *** P<

113 Table Number of vegetables or fruits cultivated, by study arm and survey round Ag-Nutrition (n= 978) Baseline (n= 1025) Control (n= 1041) Ag-Nutrition (n= 1212) Endline (n=1244) Control (n= 1080) Vegetables Cabbage Carrot Chilli Fresh maize Cucumber Egg plant Impwa Green leaves Okra Onion Sweet pepper Cleome/ Luyuni Rape Tomato Pumpkin Other vegetables Fruits Avocado Bananas Guava Lemon Mangoes Masuku Oranges Papaya Water melon Other fruits Growing any vegetables and fruits 27.91*** *** * P<0.05, ** P<0.01, *** P<

114 Table Rearing animals and production of animal source foods, by program group and survey round Baseline Endline Ag-Nutrition (n= 978) (n= 1025) Control (n= 1041) Ag-Nutrition (n= 1212) (n=1244) Control (n= 1080) percent percent percent percent percent percent Own animals or poultry Beehives Cattle Chicken Turkey Guinea fowl / pigeon Duck Goat / Sheep Donkey/ Mule /Horse Pig Rabbit Fish (aquaculture) Others Own any animals or poultry Animal products Animal meat / offal Poultry meat / offal Milk and products Eggs Honey Hide/ leather/ wool Others Produced any products * P<0.05, ** P<0.01, *** P<

115 Table Production of seven different food groups, by program group and survey round Baseline Endline Ag-Nutrition Control Ag-Nutrition Control (n= 978) (n= 1025) (n= 1041) (n= 1212) (n=1244) (n= 1080) percent percent percent percent percent percent Grains Pulses Meats Milk and products Eggs Vitamin A rich fruits and vegetables Other fruits and vegetables * P<0.05, ** P<0.01, *** P<

116 Table Difference in difference of food production indicators, by program group and survey round Agricultural Indicators: Number of crops Number of field crops Number of garden crops Food groups (of 7) Agricultural activities (of 3) Mths producing vit-a rich plants Mths producing meat Mths producing dairy Mths producing eggs Baseline Endline Difference in difference Ag- Ag- Ag- Ag- Ag- Nutrition Control Nutrition Control Nutrition Control Nutrition Nutrition (n=978) (n=1,025) (n=1,041) (n=1,212) (n=1,244) (n=1,080) T2-T1 T2-T1 T2-T1 vs vs vs Control Control *** 0.64*** -0.89*** 1.36*** 1.53*** (1.78) (1.89) (1.95) (1.89) (1.97) (1.10) (0.19) (0.28) (0.28) *** -0.42*** 0.50*** 0.84*** -0.34** (1.16) (1.19) (1.16) (0.95) (1.09) (0.76) (0.10) (0.15) (0.16) *** 0.22*** -0.47*** 0.86*** 0.70*** 0.16 (1.01) (1.11) (1.22) (1.43) (1.34) (0.65) (0.12) (0.15) (0.16) *** 0.50*** -0.48*** 0.87*** 0.98*** (1.53) (1.73) (1.53) (1.54) (1.66) (1.25) (0.19) (0.23) (0.24) *** 0.23*** -0.38*** 0.48*** 0.60*** (1.02) (1.38) (1.07) (1.04) (1.17) (1.06) (0.12) (0.19) (0.19) *** 0.53*** -0.66*** 1.25*** 1.18*** 0.07 (2.20) (1.67) (2.49) (2.77) (2.52) (1.55) (0.24) (0.25) (0.24) *** (3.35) (3.70) (2.62) (3.28) (3.57) (2.84) (0.36) (0.37) (0.41) * -0.54*** 0.55** 0.76*** (3.25) (3.00) (3.27) (3.18) (3.28) (2.34) (0.24) (0.23) (0.24) *** (4.53) (4.58) (4.39) (3.87) (3.71) (3.40) (0.51) (0.47) (0.45) * P<0.05, ** P<0.01, *** P<

117 Table Difference in difference of food production indicators in households participating in RAIN, by program group and survey round Agricultural Indicators: Number of crops Number of field crops Number of garden crops Food groups (of 7) Agricultural activities (of 3) Mths producing vit-a rich plants Mths producing meat Mths producing dairy Mths producing eggs Baseline Endline Difference in difference Ag- Ag- Ag- Ag- Ag- Nutrition Control Nutrition Control Nutrition Control Nutrition Nutrition (n=978) (n=1,025) (n=1,041) (n=415) (n=385) (n=1,080) T2-T1 T2-T1 T2-T1 vs vs vs Control Control *** 2.16*** -0.89*** 2.65*** 3.06*** (1.78) (1.89) (1.95) (2.15) (2.13) (1.10) (0.23) (0.31) (0.34) *** 1.01*** -0.42*** 0.79*** 1.43*** -0.65*** (1.16) (1.19) (1.16) (0.88) (0.96) (0.76) (0.11) (0.16) (0.17) *** 1.15*** -0.47*** 1.87*** 1.63*** 0.24 (1.01) (1.11) (1.22) (1.82) (1.74) (0.65) (0.16) (0.18) (0.21) *** 1.43*** -0.48*** 1.62*** 1.91*** -0.3 (1.53) (1.73) (1.53) (1.51) (1.57) (1.25) (0.21) (0.26) (0.28) *** 0.78*** -0.38*** 0.91*** 1.16*** (1.02) (1.38) (1.07) (0.96) (0.93) (1.06) (0.13) (0.20) (0.20) *** 1.81*** -0.66*** 2.87*** 2.47*** 0.4 (2.20) (1.67) (2.49) (3.56) (3.22) (1.55) (0.32) (0.29) (0.36) *** 0.35*** (3.35) (3.70) (2.62) (2.82) (3.70) (2.84) (0.40) (0.41) (0.48) *** 0.52*** -0.54*** *** -0.65*** (3.25) (3.00) (3.27) (2.84) (3.41) (2.34) (0.27) (0.27) (0.29) * -0.8*** (4.53) (4.58) (4.39) (3.29) (3.62) (3.40) (0.52) (0.52) (0.52) * P<0.05, ** P<0.01, *** P<

118 10 Results: Decomposition analysis of the determinants of changes in child growth outcomes over time KEY RESULTS Finally, we conducted decomposition analysis to explain factors that may have contributed to the large reductions in stunting over time, and increase in wasting. Our analysis however explain only a small proportion of these changes. The model explains 8.4% and 6.3% of the actual change in HAZ scores and stunting, respectively, in our key age group (24-59 months) during this period. Among the sources of predicted change, receipt of nutrition counseling and reductions in child morbidity stand out as the largest factors, explaining the largest proportion of the predicted change in HAZ scores. Unexpectedly, household hunger and agriculture production variables did not predict reductions in stunting or increases in HAZ. The model only explains 1% actual change for WHZ and 13% actual change for wasting. For children aged 0-59 months, the models explain 4-6% of actual change for HAZ/stunting and around 16% for WHZ and wasting. Similar analyses with several rounds of the DHS data from Zambia showed similar results Determinants of child growth Figure 10.1and Figure 10.2Figure 10.1 and Figure 10.2 show HAZ and WHZ scores by child age, respectively. HAZ scores significantly improved among all ages of children over time, but improved more among children <24 months of age. In contrast, WHZ scores were lower in 2015 compared to 2011, and the reduction is more visible among children <20 months or >50 months of age. 118

119 Figure 10.1 HAZ by child age, RAIN program areas 2011 and Age of child (months) Figure 10.2 WHZ by child age, RAIN program areas 2011 and Age of child (months) 95% CI % CI 2011 Table presents the changes in the means or percent of key outcomes between baseline and endline surveys, by different age groups. There were significant improvements 119

120 between for HAZ increases and reduced stunting prevalence for all age groups. In contrast, there were deteriorations as WHZ reduced and wasting increased over time, with steeper WHZ reductions in children months. Table Changes in the means or percent of outcomes between 2011 and 2015 in Zambia Percent or mean Percent or mean Changes % change Percent P value mo HAZ <0.001 WHZ <0.001 Stunting <0.001 Wasting < mo HAZ <0.001 WHZ <0.001 Stunting <0.001 Wasting < mo HAZ <0.001 WHZ <0.001 Stunting <0.001 Wasting <

121 % Figure 10.3 present trends in undernutrition in children months and months by survey round. Stunting reduced from 44.7% to 28.8% between 2011 and 2015 among children months. A similar trend was observed for stunting among children months. In contrast, wasting increased in both two age groups. Figure 10.3 Trends in stunting and wasting prevalence, by child age and survey time in Zambia Stunting Wasting Stunting Wasting months months Figure 10.4 and Figure 10.5 show kernel density estimations of the distribution of child HAZ and WHZ for both 2011 and These figures suggest an improvement in HAZ where the entire distributions shift to the right. For WHZ, the distribution is wider with some move to the left and the peak of the bell curve is lower. 121

122 0 Kdensity Kdensity Figure 10.4 Distributions of child HAZ scores among children months in Zambia by survey round HAZ Figure 10.5 Distributions of child WHZ scores among children months in Zambia by survey round WHZ

123 Table looks at trends in key explanatory variables over time in the entire sample of children aged months (all three study arms). We did not see economic and social changes in the last 4 years in terms of socio-economic status scores. The household hunger score was higher at endline, and more households were categorized as perceiving themselves to experience moderate or severe hunger. Household dietary diversity reduced significantly, though the number of food crops, vegetables and fruits cultivated as well as the number of food groups produced increased. The number of children under 5 years in the household reduced. Among this full group, women s education did not change over time, but more women classified themselves primarily as farmers at endline. Maternal health changes were seen in increased BMI scores, though these were still in the normal range on average. There was improvement in maternal IYCF knowledge and women s empowerment. The mean number of prenatal visits increased over time, with more mothers attending at least 4 prenatal visits, and more mothers receiving counseling on nutrition and breastfeeding. Child morbidity decreased over the last 4 years, however child deworming and immunization also decreased. Next, we conducted pooled regression for both rounds, then regressions by survey round. We tested for stability of coefficients between rounds (Table for HAZ, Table for stunting, Table for WHZ and Table for wasting) using a barrage of Chow tests. Our findings showed few differences in coefficients between baseline and endline for HAZ, but we saw evidence of stability for most parameters for stunting, WHZ and wasting. Based on these tests, we conclude that there are few signs of any secular changes in the main parameters of interest. We therefore conducted simple linear decompositions, in which we use the coefficients from these four tables. 123

124 Table Changes in the means or percent of key variables in Zambia (samples of children months) Percent or mean Percent or mean Changes % change Percent P value Household factors SES index (0/10) Household hunger score (0/6) <0.001 Moderate or severe hunger (HHS) <0.001 Household dietary diversity <0.001 Household economic shock (0/2) Number of children <5y <0.001 production Number of food crops, vegetables and <0.001 fruits cultivated Number of agricultural activities Number of food groups produced <0.001 Maternal factors Maternal education (years) Maternal occupation as farmer <0.001 Maternal BF knowledge (0/10) <0.001 Maternal CF knowledge (0/10) <0.001 Maternal BMI <0.001 Women s empowerment (0/10) Health care service access Number of prenatal visits prenatal visits Received nutrition counselling during pregnancy Received BF counselling during pregnancy Child factors Child gender Child age ARI (any fever, cold or fast breathing) Diarrhea Deworming Child immunization score (1/13)

125 Table Height-for-age z regressions by round with test for coefficient differences over time among children months Pooled Baseline 2011 Endline 2015 Significant Coefficient Coefficient Coefficient differences (a) (b) between a and b Year (2015) 0.877*** Program intervention (Ag- Nutrition) 0.163* Program intervention (Ag) ** - Year * Ag- Nutrition ** Year * Ag- Nutrition *** SES index (0/10) No Moderate or severe hunger (HHS) 0.303*** ** <0.01 Household dietary diversity 0.032** ** <0.001 Household economic shock (0/2) No Number of children <5y ** 0.161** <0.001 Number of food crops, vegetables and fruits cultivated *** ** <0.001 Number of agricultural activities 0.110** No Number of food groups produced 0.061* ** <0.01 Maternal education (years) <0.01 Maternal occupation as farmer ** <0.01 Maternal BF knowledge (0/10) No Maternal CF knowledge (0/10) No Maternal BMI <0.05 Women s empowerment (0/10) No 4 prenatal visits No Received nutrition counselling during pregnancy 0.140* No Received BF counselling during pregnancy No Child gender *** ** ** No Child age ** <0.01 ARI (any fever, cold or fast breathing) ** ** No Diarrhea *** ** No Deworming No *** p<0.001, ** p<0.01, * p<0.05, +p<

126 Table Stunting regressions by round with test for coefficient differences over time among children months in Zambia Pooled Baseline 2010 Endline 2015 Significant Coefficient Coefficient Coefficient differences (a) (b) between a and b Year (2015) *** Program intervention (Ag- Nutrition) * Program intervention (Ag) Year * Ag- Nutrition 0.095** Year * Ag- Nutrition 0.084* SES index (0/10) ** ** No Moderate or severe hunger (HHS) * No Household dietary diversity * ** <0.05 Household economic shock (0/2) No Number of children <5y ** <0.001 Number of food crops, vegetables and fruits cultivated 0.016** ** <0.05 Number of agricultural activities ** ** No Number of food groups produced No Maternal education (years) No Maternal occupation as farmer ** <0.01 Maternal BF knowledge (0/10) No Maternal CF knowledge (0/10) No Maternal BMI * ** No Women s empowerment (0/10) * No 4 prenatal visits No Received nutrition counselling during pregnancy <0.1 Received BF counselling during pregnancy No Child gender 0.085*** 0.088** 0.082** No Child age *** ** ** No ARI (any fever, cold or fast breathing) 0.049** 0.064** No Diarrhea 0.074** No Deworming No + P<0.1, * P<0.05, ** P<0.01, *** P<

127 Table Weight-for-Hight z regressions by round with test for coefficient differences over time among children months Pooled Baseline 2011 Endline 2015 Significant Coefficient Coefficient Coefficient differences (a) (b) between a and b Year (2015) *** Program intervention (Ag- Nutrition) ** Program intervention (Ag) *** Year * Ag- Nutrition Year * Ag- Nutrition 0.353*** SES index (0/10) <0.05 Moderate or severe hunger (HHS) *** *** *** No Household dietary diversity ** ** No Household economic shock (0/2) Number of children <5y Number of food crops, vegetables and fruits cultivated Number of agricultural activities Number of food groups produced Maternal education (years) Maternal occupation as farmer Maternal BF knowledge (0/10) *** ** * Maternal CF knowledge (0/10) <0.05 Maternal BMI Women s empowerment (0/10) 4 prenatal visits Received nutrition counselling during pregnancy Received BF counselling during pregnancy Child gender Child age ARI (any fever, cold or fast breathing) Diarrhea Deworming + P<0.1, * P<0.05, ** P<0.01, *** P< *** 0.038*** 0.023* * *** *** *** No No No No No No No No No No No No No No No No No No 127

128 Table Wasting regressions by round with test for coefficient differences over time among children months Pooled Baseline 2011 Endline 2015 Significant Coefficient Coefficient Coefficient differences (a) (b) between a and b Year (2015) 0.060*** Program intervention (Ag- Nutrition) *** Program intervention (Ag) ** Year * Ag- Nutrition ** Year * Ag- Nutrition SES index (0/10) No Moderate or severe hunger (HHS) 0.073*** 0.040** 0.079*** No Household dietary diversity 0.004* ** <0.05 Household economic shock (0/2) * * No Number of children <5y No Number of food crops, vegetables and fruits cultivated ** ** <0.05 Number of agricultural activities No Number of food groups produced No Maternal education (years) * No Maternal occupation as farmer No Maternal BF knowledge (0/10) 0.007* * No Maternal CF knowledge (0/10) ** <0.01 Maternal BMI No Women s empowerment (0/10) No 4 prenatal visits ** ** No Received nutrition counselling during pregnancy No Received BF counselling during pregnancy No Child gender No Child age No ARI (any fever, cold or fast breathing) No Diarrhea No Deworming No + P<0.1, * P<0.05, ** P<0.01, *** P<

129 Table reports detailed decomposition results for HAZ and stunting for children months using only those variables that are statistically significant at the 10 percent level or higher in the first column of Table and Table The first column reports the estimated coefficient from that regression. The next three columns, respectively, report the 2011 and 2015 sample means and the change in means across time. The predicted change in HAZ scores is the product of this change in means and the estimated coefficient. The last column reports the share of predicted change accounted for by each variable. There are two important findings in Table First, the model explains 8.4% (or standard deviations-sd) of the actual change in HAZ scores observed during this period (0.670 SD). The model also explains only 6.3% of the actual change in stunting. Second, among the sources of predicted change, received nutrition counseling and reduction in child morbidity stand out as the largest factors, explaining 3-4 percent of the predicted change in HAZ scores. Unexpectedly, the household hunger and agriculture production variables predicted the changes in HAZ or stunting in the opposite direction. Table also shows that the model only explains 1% actual change for WHZ and 13% actual change for wasting. For children aged 0-59 months, the models explain 4-6% of actual change for HAZ/stunting (Table ) and around 16% for WHZ and wasting (Table ). 129

130 Table Decomposing sources of HAZ and stunting change by survey round among children months in Zambia Variables Estimated coefficient Sample mean 2011 Sample mean 2015 Change in means Predicted change Share of predicted change (a) (b) (c) (d=c-b) (e= a*d) (%) HAZ Moderate or severe HH hunger HH dietary diversity Number of crops, vegetables and fruits cultivated Number of agricultural activities Number of food group produced >4 prenatal visits Received nutrition counselling during pregnancy Child gender ARI Diarrhea Ratio of predicted HAZ change to actual (%) 8.35 Unexplained (%) Stunting SES index (0/10) Moderate or severe HH hunger Household dietary diversity Number of food crops, vegetables and fruits cultivated Number of agricultural activities Maternal CF knowledge (0/10) Maternal BMI Women empowerment (0/10) Child gender Child age ARI (any fever, cold or fast breathing) Diarrhea Ratio of predicted stunting change to actual (%) 6.34 Unexplained (%) Table Decomposing sources of WHZ and wasting change by survey round among children months in Zambia Variables Estimated coefficient Sample mean Sample mean 2015 Change in means Predicted change Share of predicted 130

131 2011 change (a) (b) (c) (d=c-b) (e= a*d) (%) WHZ Household hunger categories (moderate or severe hunger) Household food diversity Number of food crops, vegetables and fruits cultivated Number of agricultural activities Maternal education Maternal BMI >4 prental visits Child age Ratio of predicted WHZ change to actual (%) 1.90 Unexplained (%) Wasting Moderate or severe HH hunger Household dietary diversity Economic shock Number of food crops, vegetables and fruits cultivated Maternal BF knowledge (0/10) Maternal CF knowledge (0/10) >4 prental visits Child gender Ratio of predicted wasting change to actual (%) Unexplained (%)

132 Table Decomposing sources of HAZ and stunting change by survey round among children months in Zambia Variables Estimated coefficient Sample mean 2011 Sample mean 2015 Change in means Predicted change Share of predicted change (a) (b) (c) (d=c-b) (e= a*d) (%) HAZ Moderate or severe HH hunger Household dietary diversity No of children < Number of food crops, vegetables and fruits cultivated Number of agricultural activities Number of food groups produced Maternal occupation as farmer Maternal BF knowledge BMI Women empowerment (0/10) >4 prental visits Received nutrition counselling during pregnancy Child gender Diarrhea Deworming Ratio of predicted HAZ change to actual (%) 3.72 Unexplained (%) Stunting SES Household dietary diversity No of children < Number of food crops, vegetables and fruits cultivated Number of agricultural activities Maternal occupation as farmer BMI Women s empowerment (0/10) Received nutrition counselling during pregnancy Child gender Child age ARI Diarrhea Ratio of predicted stunting change to actual (%) 5.60 Unexplained (%)

133 Table Decomposing sources of WHZ and wasting change by survey round among children months in Zambia Variables Estimated coefficient Sample mean 2011 Sample mean 2015 Change in means Predicted change Share of predicted change (a) (b) (c) (d=c-b) (e= a*d) (%) WHZ Moderate or severe HH hunger Number of food crops, vegetables and fruits cultivated Number of agricultural activities Maternal education Maternal CF knowledge BMI >4 prental visits Child age Diarrhea Deworming Ratio of predicted HAZ change to actual (%) Unexplained (%) Wasting SES Moderate or severe HH hunger Economic shock No of children < Number of food crops, vegetables and fruits cultivated Maternal CF knowledge BMI Women empowerment (0/10) >4 prental visits Received nutrition counselling during pregnancy Diarrhea Ratio of predicted stunting change to actual (%) Unexplained (%)

134 11 Discussion This report presents findings from a large, complex, multi-year, inter-sectoral project that combined agriculture and nutrition interventions to impact child nutrition. The RAIN project is one of a handful of projects that includes a rigorous randomized design to examine impact, and responds directly to recent calls for stronger evaluation designs of agriculture and nutrition interventions to strengthen the evidence base on these links. Overall, the RAIN project had mixed impacts. The project had: 1) large and consistently positive impacts on agricultural production, 2) impacts on different domains of women s social and economic empowerment, as well as women s empowerment in agriculture, 3) impacts on household food security as measured by household dietary diversity, and 4) a potential protective effect on child wasting. In general, where there were significant program impacts, the magnitude of these impacts was larger in per-protocol analyses, among confirmed RAIN beneficiaries. There were however no discernable impacts on reducing the prevalence of stunting, on improving IYCF practices among young children, or on improving caregiver health and nutrition knowledge. There appears to be little to no additional benefit of the Ag-Nutrition arm, compared to the Ag-only intervention arm for the impacts achieved. This is further evidenced by greater exposure to the agricultural intervention components of the RAIN project, compared to the nutrition intervention components. The agricultural production element of the RAIN intervention achieved its aim of increasing both diversity and stability of plant foods available to households. Though the animal interventions were largely unsuccessful in increasing the consumption of animal sourced foods, there were clear improvements in production of diverse foods across the households surveyed, and improvements were around double in per-protocol analyses (confirmed beneficiary households). Given increases in production of different food groups, it follows that food security as measured by access of households to diverse food groups also improved in this study. Previous reviews of the effects of intersectoral agriculture-based projects for nutrition outcomes have found that these projects do tend to achieve increases in both production and consumption of the food groups they promote (Masset, Haddad et al. 2011). In this study, as well as the objective Household Dietary Diversity measure of food security (HDDS), we also measured subjective household perception of hunger using the Household Hunger Scale (HHS). Interestingly, while household access to diverse food groups improved, household perception of food security is worse in intervention households. The main domains of empowerment most affected by the program were social capital involving contact with other women to discuss issues and share information; ownership of animals; and input into decisionmaking on what foods to grow and spending money from the sale of field crops. These 134

135 outcomes make sense in the context of the specific RAIN interventions involving creating women s groups and providing agricultural inputs. However there is still a long way to go in improving women s perceptions of equality, decision making power, and power to hold and use economic assets for the nutrition and health of themselves and their children. Improvements in IYCF practices require both food security and caregivers who are empowered to make decisions for their children. Several elements of the RAIN agricultural interventions aimed to improve determinants of IYCF, including making diverse foods available and accessible, and bringing women together into groups. The nutrition interventions in the Ag-Nutrition group built on these to provide information to improve knowledge on breastfeeding, complementary feeding, and knowledge. No differences were seen in IYCF knowledge or practices attributable to the RAIN project, other than an increase in children s consumption of legumes (likely groundnuts, which were promoted by RAIN). The low exposure within the study area is one possible factor contributing to the lack of consistent impacts across project objectives. The larger effect sizes in per-protocol analyses support this hypothesis. We have explored program implementation further in Appendix 4: Comparing RAIN and DHS This study was not designed to assess the role of external factors in impact findings; the inclusion of a control group and the use of difference-in-difference and decomposition analyses allows us to assess the differential effect of the RAIN project without this kind of anlaysis. However, given the apparently steep changes in stunting and wasting seen in the endline analysis, we show below how the RAIN data fits with similar data from two rounds of DHS in Central Province, in 2007 and Figure 12.1 shows stunting rates broadly in line with ongoing reductions seen in the DHS, though with a major reduction in Wasting rates have increased in the RAIN data but decreased in the DHS. 135

136 Figure 12.1 Comparison of stunting and wasting between RAIN and DHS DHS 2007_central, all <59m RAIN 2011, all 24-59m DHS 2014_central, all <59m RAIN 2015, all 24-59m Stunting Wasting 136

137 Appendix 5: Explaining the impact of RAIN: Program impact pathways. As noted previously, only a third of eligible households surveyed had participated in the project through joining a women s group the main point of entry into the RAIN project. In addition, home visits by SMFs and CHVs, a critical component of the intervention delivery by the program, did not materialize as envisaged, losing a key one-to-one element that would bolster the project. Across both the women s groups and home visits, it was clear that the agriculture frontline workers (SMFs) were more active than the health-side workers (CHVs), which plays out in the improved agriculture outcomes compared to nutrition and health outcomes in our evaluation. Only percent of households had both high participation and high levels of program delivery; around half of households scored either medium or high. The process evaluation (conducted in 2013 & 2014) suggests that the SMF, a position created specifically for the RAIN project, were initially more incentivized to act for RAIN through RAIN s provision of incentives (agricultural inputs) than the CHVs, positions that already existed in the community and which did not receive similar incentives until after 2014, though both groups received additional training. In addition, CHVs serviced the entire community, whereas SMFs were working specifically with RAIN groups; also SMFs are newly created positions, and the information they provide was new, whilst the CHVs were existing positions so women should have been exposed to many of the messages before. However, the project strengthened the knowledge of the CHVs and increased the number of CHVs in the area. It is clear from the evaluation that by and large, the Ag-only group fared better than the Ag-Nutrition group, suggesting no additional value of nutrition interventions in this project, implemented primarily by CHVs. As noted in the results section, per-protocol analysis (whereby analysis was restricted to those individuals that were confirmed RAIN beneficiaries) did not alter the main findings, though almost twothirds of the sample was lost in this process. As such it is hard to make definitive claims about lack of impact being solely driven by low participation. The impact evaluation design attempts to attribute changes in key impact indicators of interest to the RAIN interventions. As such, it is important to note a limitation of the design is its inability to accurately account for elements beyond the RAIN project interventions. For several indicators of interest, we observe changes over time in both the RAIN intervention groups, as well as the control group. In Appendix 4: Comparing RAIN and DHS we can see that reductions in stunting are broadly in line with other surveys in Central province, but increases in wasting are not. While we are able to document the lack of leakage of formal program delivery of RAIN interventions to the control group, we are unable to document informal leakage of secondary intervention components (knowledge, agricultural inputs etc. provided by non-project staff) among peers or relatives across program study arms. Additionally, we are unable to adequately account for general improvements in government health services across the 137

138 district, regardless of RAIN study group. There are clear improvements over time, between baseline and endline, in access and use of government-run health services across all study groups, as documented in this report. This is evident in increased access and use of prenatal care, as well as receipt of health and nutrition counseling services at government clinics. As such, it is plausible that this increase in access and use directly impacts health and nutrition knowledge, and IYCF practices, components of prenatal counseling and under 5 clinics at government health centers. This increase may be sufficiently large to prevent detection and attribution of RAIN interventions, over and beyond general increases and secular trends. While the decomposition analyses found large changes in nutrition outcomes across study arms, analyses left a lot of this change unexplained, meaning that the food, health, and care variables available in the RAIN dataset could not explain the change. There is limited evidence from other evaluations of similar programs. An evaluation of HKI s International s Enhanced Homestead Food Production (EHFP) intervention in Burkina Faso (one of only a very few rigorous impact evaluations of similar agriculture-nutrition programs designed to improve child nutrition) demonstrated results that were similar for many, but not all, of the outcomes examined for the RAIN project (Olney, Pedehombga et al. 2015). The evaluation of the EHFP program in Burkina Faso found 1) positive impacts on wasting among children, 2) small positive impacts on anemia, 3) small positive impacts on maternal dietary diversity, 4) positive impacts on maternal underweight, and 5) positive impacts on several dimensions of women s empowerment. Of note, the EHFP program had no impact on reducing the level of stunting, and no impact on food security. Overall, the results from the RAIN evaluation contribute to remedying the dearth of evidence from rigorous impact evaluations of integrated agriculture and nutrition programs. There are clear and important maternal and household level benefits of this program, which may be achieved in other, similar programs, and lessons used to to scale-up the existing project. The clear and consistent impact of the RAIN intervention on agriculture production, and on women s empowerment, two core objectives of the program, are noteworthy and consistent with the limited evidence to date from similar interventions. 138

139 References Central Statistical Office, Ministry of Health, et al. (2009). Zambia Demographic and Health Survey Calverton, Maryland, USA, CSO and Macro International Inc. Central Statistical Office, Ministry of Health, et al. (2014). Zambia Demographic and Health Survey Calverton, Maryland, USA, CSO and Macro International Inc. Central Statistical Office, Ministry of Health, et al. (2009). Zambia Demographic and Health Survey Harris, J., W. Quabili, et al. (2011). RAIN baseline survey report. Washington, DC, IFPRI. Headey, D. (2014). "An analysis of trends and determinants of child undernutrition in Ethiopia, " IFPRI discussion paper: WORKING PAPER 70. Headey, D. D., J. Hoddinott, et al. (2014). "The Other Asian Enigma. Explaining the Rapid Reduction of Undernutrition in Bangladesh." IFPRI Discussion Paper K. von Grebmer, J. Bernstein, et al. (2015) Global Hunger Index - Armed Conflict and the Challenge of Hunger. Bonn, Washington D. C., Dublin, Welthungerhilfe, IFPRI, and Concern Worldwide. Martorell, R. (1995). "The effects of improved nutrition in early childhood: The INCAP follow-up study." J Nutr 125: Masset, E., L. Haddad, et al. (2011). A systematic review of agricultural interventions that aim to improve nutritional status of children. London, EPPI-Centre, Social Science Research Unit, Institute of Education, University of London. Olney, D. K., A. Pedehombga, et al. (2015). "A 2-Year Integrated and Nutrition and Health Behavior Change Communication Program Targeted to Women in Burkina Faso Reduces Anemia, Wasting, and Diarrhea in Children Months of Age at Baseline: A Cluster- Randomized Controlled Trial." The Journal of Nutrition. Smale and T. Jayne (2009). Breeding an Amaizing Crop: Improved maize in Kenya, Malawi, Zambia, and Zimbabwe. Millions Fed: Proven successes in agricultural development. R. Pandya-Lorch. Washington, DC. UNDP (2010). Human Development Report th Anniversary Edition: The Real Wealth of Nations: Pathways to Human Development. HDR, UNDP. UNICEF (1990). Strategy for improved nutrition of children and women in developing countries New York, UNICEF. UNICEF (2015). State of the world's children 2015: Reimagine the future. New York, UNICEF. WFP and FAO (2010). The State of Food Insecurity in the World Addressing food insecurity in protracted crises. Rome, FAO. WHO (2010). "The WHO Child Growth Standards. WHO Multicentre Growth Reference Study Group (2006). WHO Child Growth Standards: Length/heightfor-age, weight-for-age, weight-for-length, weight-for-height and body mass index-for-age: Methods and development. Geneva, World Health Organization. ZVAC, T. Z. N. V. A. C. (2004). REPORT ON THE NVAC CONSULTATION PROCESS. Lusaka. 139

140 12 Appendices Appendix 1: Detailed sample size calculations For both Baseline and Endline, all sample size calculations were carried out using STATA 11 software. A two stage process was followed: 1. Sample sizes were estimated to detect changes in stunting and height-for-age Z scores assuming simple random sampling from a single cluster. This was carried out using the sampsi command 2. The sample size estimates obtained in (1) were then adjusted to account for clustering using the sampclus command Two different sample size estimates were obtained. The first was to detect changes in stunting, and the second was to detect changes in height-for-age Z scores. The highest was used. Baseline Stunting The following assumptions were used: 1. A baseline prevalence of stunting of 53 percent (Based on the prevalence in Central Province from the 2007 Zambia DHS), and a reduction of 8 percentage points 2. A β of 0.8, and an α of A one sided test 4. The availability of 13 clusters per study arm 5. An intra-cluster coefficient of 0.01 Height-for-Age Z scores The following assumptions were used: 1. A change in height-for-age Z score of A standard deviation of A β of 0.8, and an α of A one sided test 5. The availability of 13 clusters per study arm 6. An intra-cluster coefficient of for HAZ; 829 for stunting: children aged 2-5 per arm Rounded up to 1000 per arm for the survey, as we didn t know exact stunting rates in Mumbwa +10% = 3300 total 140

141 STATA Output- Stunting. sampsi.53.45, p(0.8) onesided Estimated sample size for two-sample comparison of proportions Test Ho: p1 = p2, where p1 is the proportion in population 1 and p2 is the proportion in population 2 Assumptions: alpha = (one-sided) power = p1 = p2 = n2/n1 = 1.00 Estimated required sample sizes: n1 = 507 n2 = 507. sampclus, numclus(26) rho(0.01) Sample Size Adjusted for Cluster Design n1 (uncorrected) = 507 n2 (uncorrected) = 507 Intraclass correlation =.01 Average obs. per cluster = 64 Minimum number of clusters = 26 Estimated sample size per group: n1 (corrected) = 827 n2 (corrected) = 827 STATA Output- HAZ. sampsi 0 0.2, sd(1.3) p(0.8) onesided Estimated sample size for two-sample comparison of means Test Ho: m1 = m2, where m1 is the mean in population 1 and m2 is the mean in population 2 Assumptions: alpha = (one-sided) power = m1 = 0 m2 =.2 sd1 = 1.3 sd2 = 1.3 n2/n1 = 1.00 Estimated required sample sizes: n1 = 523 n2 = 523. sampclus, numclus(26) rho(0.01) Sample Size Adjusted for Cluster Design n1 (uncorrected) = 523 n2 (uncorrected) = 523 Intraclass correlation =.01 Average obs. per cluster = 67 Minimum number of clusters = 26 Estimated sample size per group: n1 (corrected) = 869 n2 (corrected) =

142 Endline Sample size estimate AMENDED based on actual stunting rate found in Baseline survey (44%): 959 children aged 2-5 per arm x 3 arms = % un-useable data = over-sampling of beneficiaries 5% in each intervention arm = 3453 total STATA output- amended. sampsi , p(0.8) onesided Estimated sample size for two-sample comparison of proportions Test Ho: p1 = p2, where p1 is the proportion in population 1 and p2 is the proportion in population 2 Assumptions: alpha = (one-sided) power = p1 = p2 = n2/n1 = 1.00 Estimated required sample sizes: n1 = 554 n2 = 554. sampclus, numclus(26) rho(0.01) Sample Size Adjusted for Cluster Design n1 (uncorrected) = 554 n2 (uncorrected) = 554 Intraclass correlation =.01 Average obs. per cluster = 74 Minimum number of clusters = 26 Estimated sample size per group: n1 (corrected) = 959 n2 (corrected) = 959 per arm 142

143 Sampling Methodology Sampling of children in baseline surveys was specific to households that had at least one child aged months of age, the age range for detecting impacts on stunting i.e. the primary RAIN project impact indicator. However, to capture impacts of the RAIN project on key IYCF indicators, children between 0-23 months of age were also sampled. The endline survey will follow a similar protocol. Logistical considerations prevented us from sampling children 0-23 months and months of age from unique households; sampling unique household would substantially increase our sample size which was not feasible. Therefore, preference was given to households that had both a child months of age as well as a child 0-23 months of age. Where multiple children within these age ranges were present in a household, the youngest child within the household was selected. A listing exercise was undertaken in the 6 wards to determine eligible households i.e. households with the presence of at least one child aged months of age. A child aged months old was identified as the INDEX child. A child aged 0-23 months old was identified as the NON-INDEX child. All households were categorized into 3 possible categories: 4. Households with an INDEX child only 5. Households with both an INDEX and a NON-INDEX child 6. Households with a NON-INDEX child only The following procedures and guidance were taken in sampling households for the RAIN project baseline survey: 1. A listing exercise was conducted to identify all households with a child between the ages 0-23 months of age, and months of age. This was done using maps provided by the Government central statistics office (CSO) and experienced mappers who were involved in the Zambia 2010 household census. 2. The total number of households listed was This included the total number of households for which no information was obtainable (identified as non-contact households) or who refused to participate in this listing exercise (identified as refusal households) which totaled 682 households. 3. Households were then categorized into one of the three categories listed above. Details of this listing exercise can be found in Annex 3- Listing book. 4. Of the identified 9541 households listed, 6435 households were identified as having a child between 0-59 months of age. A systematic sampling process (outlined in steps 4-6, below) was then followed to yield the total number of households within each unit of randomization (the CSA) as required by the sample size calculations. 5. Because the CSA is the unit of randomization, it is therefore the unit of most interest from an evaluation perspective as it is the unit by which treatments are applied. However for sampling 143

144 purposes the CSA covers a very large geographic area and a large number of households and is therefore not practical as a unit for sampling. For sampling purposes the unit that is most practical is the SEA, which is a clearly identified geographic area as per the Government CSO. The majority of CSAs have 3 SEAs within them, though within the 6 wards from which the baseline survey sample was drawn, there was a range of 2-5 SEAs per CSA, as per the 2010 CSO sampling frame. 6. The sample size calculations require that a total of 69 households be sampled in the baseline survey per unit of randomization-the CSA. The majority of CSAs had 3 SEAs, and within these SEAs 23 households were sampled. In CSAs that had 4 or 5 SEAs, 3 SEAs were randomly sampled and 23 households were sampled from each SEA. In CSAs that had only 2 SEAs, 35 households were sampled from the first SEA, and 34 households from the second. Therefore, based on this sampling procedure, a total of 69 households were sampled per CSA as required by the sample size calculations. 7. Within each SEA, households were sampled based on the household listing exercise conducted at the start of field work, using a systematic random sampling procedure that used a random number table. This sampling procedure is outlined in Annex 4- Stratification and selection of sample households. 8. A total of 3044 households were sampled for this baseline survey. Only 34 households that were selected as part of the systematic random sampling procedure refused to be interviewed. 144

145 Appendix 2: Variables included in the decomposition analysis The dependent variables in this analysis include child nutrition indicators. We primarily focus on HAZ and WHZ as measured against World Health Organization growth standards, which are described in de Onis et al. (2007) and WHO (2006). We also use outcomes as stunting (HAZ<-2) and wasting (WHZ<-2) as they are commonly used to indicate child nutritional status and stunting is the main interest of the intervention. As study design, we looked at the determinants for child growth among children months of age, as we expected seeing the most change in this group. Additionally, we also looked at the determinants for child growth among children 0-59 months of age. The independent variables included two sets. The first set was time-varying intermediate determinants which can help both for understanding which factors explain nutritional differences across children, and for decomposing changes across time. These include household factors (household economic status, HH hunger score, HH food dietary diversity and economic shock), maternal factors (education, occupation, knowledge on IYCF), health service access (prenatal and postnatal care), sanitation, and child morbidity (acute respiratory infection ARI and diarrhea). The second set was control variables that are essentially fixed over time such as maternal BMI, child and maternal age, child gender, location fixed effects, and RAIN intervention. Definition of variables used in the analysis was presented in Table below. Table 0.1 Definitions of variables using in the analyses Variable name Definition Outcomes HAZ Height-for-age z score (HAZ) measured against WHO (2006) norms Stunting HAZ < 2 WHZ Weight-for-height z score (WHZ) measured against WHO (2006) norms Wasting WHZ < 2 Determinants Household factors SES index Index based on principal component analysis and standardized score 1-10 Household hunger score (0/6) Continuous variable Household hunger categories Dummy = 1 if household experienced moderate or severe food hunger in the last 30 days Household food diversity 12 food groups, used as continuous variables Household economic shock (0 3) Disaster events that household experienced in the last 12 months 145

146 production Number of food crops, vegetables and fruits cultivated Number of agricultural activities Number of food groups Maternal factors Maternal education (years) Maternal occupation as farmer Maternal BF knowledge Maternal CF knowledge Maternal BMI Women empowerment Health care service access 4 prenatal visits Received nutrition counselling during pregnancy Received BF counselling during pregnancy Child factors Child gender Child age ARI Diarrhea Deworming Child immunization score (0/13) Others Ward including field crops and fruits and vegetables Including production of field crops, fruits and fruits/vegetables, rearing animals and production of animal source foods. The production of 7 different food groups that correspond to those groups used in the child dietary diversity index Mother s years of education Dummy = 1 if mother was a farmer Score of knowledge on breastfeeding Score of knowledge on timeliness of complimentary food Kg/m2 Use composite score Dummy = 1 if mother received 4 or more prenatal care visits Dummy = 1 if mother received nutrition counseling Dummy = 1 if mother received BF counseling Dummy = 1 if child is female Child s age as continuous variable Dummy = 1 if the child had any symptoms of fever of cough Dummy = 1 if the child had symptom of diarrhea Dummy = 1 if the child got deworming Each immunization get one score. Use variables that are common between baseline and endline Code for commune, used to control for cluster effect 146

147 Appendix 3: Detail on the RAIN project Frontline workers The women groups were the target for recurring agriculture and nutrition trainings delivered by Smallholder Model Farmers (SMFs) and Community Health Volunteers (CHVs). All SMFs were female and they provided training to vegetable/fruit production and rearing small livestock. The initial project set up was that each of the 112 SMFs would be responsible for the training and support of one group in the first year, with an additional group in the second year of the project as part of a phased-in approach. However, due to the low population density and the scattered housing set up in Mumbwa District, it was not possible for all SMFs to include a second group and therefore, an additional group of 69 SMFs was trained in the second year. The total number of SMFs is 181, reaching out to 4499 beneficiaries. A total of 96 CHVs were engaged in the project and responsible for the delivery of nutrition and health education and communication. Each CHV is linked to one or two women groups. In each of the four wards where the project is being implemented, there were two Community Development Facilitators (CDF -MCDA partner staff) who provided training and mentorship to the SMFs and CHVs, with assistance from government workers (Camp Extension Officers, Rural Health Centre Staff), other MCDA and Concern staff. Each CDF was responsible for 30 women groups. Training Cascading trainings were used to build capacity for specific agricultural and nutrition skills and gender awareness. The SMFs received an intensive training course on agricultural practices including homestead gardens, organic manure, integrated pest management and rearing small livestock. Bi-monthly refresher trainings were organised at community level by the CDFs with assistance from government workers. The CHVs received an intensive training course in Infant and Young Child Feeding (IYCF). In addition, Health Facility Staff trained as Trainer of Trainers (ToT) provided monthly nutrition refresher trainings to the CHVs on various subjects including maternal health, how to conduct cooking demonstrations, WASH and micronutrient deficiencies. CHVs and SMFs received a bicycle to facilitate their movements. The women groups aimed to meet for 2 hours each week, either at one of the member s home garden for a hands-on demonstration or at any central gathering point in the community. The SMFs were providing the agricultural trainings to the members of her group. Agricultural inputs were provided to all group members during the dry and wet season. The crops were chosen based on their nutritional value and included legumes (cowpeas, groundnuts, Fe/Zn bio fortified beans), vegetables (e.g. amaranthus, rape, tomatoes, carrots, spinach, pumpkin (leaves), paprika, green beans), fruits (banana suckers, granadillas, watermelons) and orange fleshed sweet potato vines. In order to facilitate the watering of the dry season gardens, each group was provided with treadle pumps. Each group also received a food solar dryer in order to facilitate preserving fruits and vegetables to increase access to micronutrient rich foods throughout the year. Seeds, vines and fruit tree saplings are provided to all group members with the goal of diversifying their homestead food production. SMFs received more seeds than other group members with the rationale that they will establish small-scale nurseries for seed production and distribution in the future. Lack of adequate water supply especially during the dry season was affecting all year round nutritional gardens. To address this, the project identified and rehabilitated boreholes and conducted training to the village WASH committees on borehole management. 147

148 To facilitate livestock production, a pass-on scheme was designed whereby all SMFs were provided with a male and a female goat, whilst some group members received a female goat, and passed on their first off spring to the other group members. Half of the group members received a chicken. Nutrition and Health The nutrition and health activities focussed on behaviour change communication for improved child and maternal nutrition, especially infant and young child feeding, and linkages to the existing health system, including the prevention of mother to child transmission services. A Behavioural Change Communication (BCC) strategy was developed based on a barrier analysis conducted for several behaviours related to infant and young child feeding. The BCC materials used are based on the standard IYCF training package (counselling cards, participant s manual and brochures) developed by the government, but translated in the local language. In order to promote the integration of nutrition and agriculture at field level, the SMFs have received some basic nutrition knowledge in addition to their agricultural training. A small booklet with the main nutrition message around the first 1000 days was provided to them as a reference book. Gender At the initial stage of the project, a gender need assessment was conducted to better understand the gender constraints related to food and nutrition security. Community sensitizations were conducted for all relevant community members including men and village elderly. Special gender trainings were conducted for community leaders. Women s group members and their husbands were trained in the importance of gender in improving nutrition (e.g. husbands providing land for homestead gardens, assisting with chores like fetching water and supporting health facility visits). Continuous gender awareness is created by integrating gender messages in the nutrition and agricultural messages provided to the beneficiaries, drama group performances and a series of IEC materials with gender messages distributed within the project area. CDFs of MCDA and government extension workers have received a ToT in gender, and are now capable of facilitating the trainings. Activities aimed at reducing the workload of women include the promotion of a fuel efficient cooking stoves and the provision of glyricidea seed (fast growing firewood tree). Coordination and alignment A multi sectoral committee, the Mumbwa District Nutrition Coordinating Committee (DNCC), was established as a coordination mechanism to align activities of the key stakeholders to effectively address malnutrition. This DNCC includes district representatives from the Ministry of and Livestock (MAL), Community Development, Mother and Child Health (MCDMCH), Ministry of Education (MoE) and from civil society organisations. The District Commissioner for Mumbwa has endorsed the committee as a subcommittee of the District Development Coordinating Committee (DDCC) and reports through that mechanism, to provincial and national bodies. Through reflective and learning processes designed to bring sectors together, new and interesting ways are being explored to promote coordination between officials in the agriculture, health and community development sectors, starting at district level in Mumbwa, and then cascading down to extension workers at community level. The coordinating structures at ward level have received regular orientation in order to build their capacity in nutrition. The DNCC has developed a multi-sectoral district nutrition plan which will be implemented as part of the Most Critical 1000 days programme/ Scaling Up Nutrition (SUN). 148

149 HIV and AIDS The main objective of the HIV and AIDS mainstreaming is to ensure that the project does not pose an increased risk of HIV transmission and that people living with HIV and AIDS can benefit from the project interventions. The CDFs have received intensive HIV and AIDS training with a special focus on the prevention of mother to child transmission (PMTCT). Both SMFs and CHVs received HIV and AIDS training, and the CHVs have received additional training on PMTCT as part of the IYCF training, and a number of additional refresher trainings. The SMFs and CHVs sensitize the communities. A total of 16 local drama groups have been trained on how to convey messages in the area of nutrition, health, gender and agriculture diversification. Field days are organised during both dry and wet season in order to demonstrate best practices, share experience and lessons learnt among beneficiaries as well as other community members. 149

150 Appendix 4: Comparing RAIN and DHS This study was not designed to assess the role of external factors in impact findings; the inclusion of a control group and the use of difference-in-difference and decomposition analyses allows us to assess the differential effect of the RAIN project without this kind of anlaysis. However, given the apparently steep changes in stunting and wasting seen in the endline analysis, we show below how the RAIN data fits with similar data from two rounds of DHS in Central Province, in 2007 and Figure 12.1 shows stunting rates broadly in line with ongoing reductions seen in the DHS, though with a major reduction in Wasting rates have increased in the RAIN data but decreased in the DHS. Figure 12.1 Comparison of stunting and wasting between RAIN and DHS DHS 2007_central, all <59m RAIN 2011, all 24-59m DHS 2014_central, all <59m RAIN 2015, all 24-59m Stunting Wasting 150

151 Appendix 5: Explaining the impact of RAIN: Program impact pathways Figure 1 below shows the Program Impact Pathways (PIP) for the RAIN project. Generated through a collaborative process involving Concern and IFPRI during the initial year of the RAIN project, the PIP draws on program implementation documents, RAIN objectives, and international best practices frameworks to map out the entire project from inputs and implementation processes to the expected outputs, outcomes and impacts of the project. This document uses data from the baseline and endline surveys, and two rounds of process evaluation (PE), to assess each step in the PIP. A traffic light system 6 is used to indicate to what extent a particular step in the PIP was achieved, and notes are provided on each step in order to explain these achievements. The first round of PE was undertaken in 2013 and considered program delivery. For this round, key informant interviews were undertaken with managers; a small non-representative survey of qualitative and quantitative questions was undertaken with frontline workers and supervisors; and focus group interviews were undertaken with a small number of beneficiaries (Table 1). The aim was to understand the process of implementation, both to enable understanding of the final evaluation results, and to enable changes to be made to the program in real time. The second round of PE was undertaken in 2014 and considered program utilization. For this round, focus groups and observations were undertaken with known beneficiaries, and in addition a small survey was undertaken to provide interim reporting to donors on key outcomes (though this was undertaken in a different season and so cannot be compared with the main evaluation survey). Sampling for the survey was of adopters and non-adopters in intervention areas (note that samples were purposive, so we can t say anything about enrollment rates from this data). The main evaluation survey was undertaken at baseline and endline, sampling around 1000 households eligible for the RAIN project in each study arm, and using a detailed household survey to assess differences between study arms and over time in key outcome and impact indicators. Table 2 Process evaluation sampling Round 1: Assessing program delivery Managers Supervisors and trainers Frontline workers Beneficiaries Interviews Survey - 8 CDFs 7 TOTs At least 2 from each cadre in ag and health= CHVs per arm (40 for health arm) =80 25 SMFs per intervention arm =50 See note 4 on gender workers = 4 Focus groups Same 5 groups as observation =10 Plus 2 non-beneficiary=2 Observation group meetings in each intervention arm =10 - Round 2: Assessing program use Control (potentially eligible) Ag only (participating) Beneficiaries Ag only (not participating) Ag+Health (participating) - Ag+Health (not participating) Survey Focus groups Observation Green indicates that the step was broadly fulfilled; yellow indicates that there were some problems with the step; and red indicates that the step was broadly not fulfilled as planned. 151

152 IMPACT OUTCOMES OUTPUTS PROCESS INPUTS Figure 12.2: RAIN program impact pathway (PIP) achievement 2 inputs procured 3 master trainers identified 4 Training manuals and tools created 1 RAIN partnerships Health master Gender master Boreholes assessed trainers identified trainers identified 3a master trainers trained 8 Women s groups established 5a Health master trainers trained 6a Gender master trainers trained 2a Women s groups receive agricultural inputs 3b SMFs recruited and 8a Women attend 5b trained (+ refreshers) groups 3c SMFs retain skills and knowledge 2c SMFs receive agricultural inputs 9a SMFs establish model farms 9 SMFs deliver group ag messaging and home visits CDFs facilitate frontline work 9b CHVs recruited and trained (+ refreshers) CHVs retain skills and knowledge 5c 9c CHVs deliver group BCC and home visits 6b Gender trainers deliver messaging to groups Introduction of labor saving technology (stoves) 6c 7a Borehole rehabilitation undertaken 10 Women acquire Women acquire Women acquire ag health knowledge resources knowledge and skills 2b and skills 11 6d Change in gender attitudes Improved access to water 7b Women establish gardens 12 Women establish small animal production 13 Reduced burden on women 6e 12a Increased fruit and vegetable production 13a Increased animal production Improved postharvest 14 management 18 Increased consumption of nutrient-rich foods Change in resources (controlled by women) Change in women s influence/ decisionmaking power Change in women s time use 152 Improved health knowledge and 19 practices 20 Improved IYFC knowledge and practices 21 Change in stunting rates

153 1. RAIN management and partnerships- A major success of the RAIN project was the process of formation of a District Nutrition Coordination Committee (DNCC) in Mumbwa district. The process - catalyzed by the Concern, facilitated by an international consultant, and now increasingly led by members of the DNCC itself - has involved strengthening both technical and strategic capacity within the DNCC, with emphasis on its leadership. This is not a prescribed process, but it has been carefully designed to both bring in key nutrition and management information at the right times, and to respond to the needs and processes of the DNCC itself. This is an innovative design: normally these processes are started at national level, often with little attempt to understand or engage with structures at the local level, where implementation actually occurs. In this case, the focus on intersectoral action started in international discourse; jumped directly to local level implementation through the facilitation of Concern; and then contextually-relevant learning from this pilot was fed back up to national level where it altered policies and program designs to incorporate a focus on coordination between sectors, before being rolled out more broadly in 7 of the Most Critical Days Programme pilot districts. In essence, this is good development NGO practice in action: taking a key international idea; supporting its piloting it in a local context; rigorously documenting the experience 7, with explicit attention to informing national policy and working through existing systems; and supporting government in trying to take successful elements of the project to scale. The innovations of NGOS like Concern who are willing to support these emergent processes are showing that change is possible. More centrally to the RAIN project, bilateral partnerships were formed through MOUs with the ministries of agriculture, health and community development; with Women for Change (WfC), a prominent national gender NGO that would plan and implement the gender component of the project; and with Mumbwa Child Development Agency (MCDA), a local CBO who would undertake implementation: The partnership with WfC dissolved early in implementation, due in part to a lack of management and administrative capacity in WfC to deal with Concern s financial accountability systems. Relations with the ministries remained strong and grew stronger through joint work on the DNCC over the course of the project. However it is unclear to what extent the relationships through the DNCC were helpful to RAIN engagement with CHVs and health centers on the ground; in the 2013 PE it was found that Government staff in the field do not appear to be undertaking RAIN duties other than training; it is unclear exactly what is expected of them in terms of supervision or monitoring, if anything. In particular, the 2013 PE found issues with incentives for CHVs (an existing government volunteer position) in the field. The key partnership, with MCDA, remained throughout the duration of the project and throughout numerous changes in personnel on both sides, and implementation remained through this CBO. The 2013 PE found that this CBO partnership was amicable, but there were concerns around the systems imposed by Concern and lack of an inception period to cement relationships. Capacity was not built to deal with the large administrative and monitoring load required of MCDA by the project; capacity for financial and other administrative reporting was low, causing significant delays to funding throughout the duration of the project, and therefore delays to implementation. Several respondents in 2013 felt that direct implementation by Concern should have been pursued from the start in such an important proof of concept project. 7 In addition to notes, briefs and papers generated throughout the lifetime of the RAIN and DNCC projects, this work has been written up partly under the Stories of Change project, and will be available for dissemination in May Further work on this coordination aspect of RAIN forms Jody s PhD, and will be ready in

154 Finally, a challenge to overall management of RAIN was lack of an effective monitoring system for the majority of the project. At the time of the 2013 PE, roll-out of a monitoring systems was delayed, with only basic monitoring data available at the end of year one; this did not allow for appropriate action to be taken to correct any problems, and a functioning monitoring system was not established until mid- 2014, a year before the end of implementation. 2. Ag inputs procured and transferred- The 2013 PE found that most inputs planned for 2012 were procured and distributed. However, for many items significantly more was required than planned (particularly seeds, but also some larger items such as drums, treadle pumps and solar dryers), and for some items it appeared that planned items could not be procured in sufficient numbers (goats, chickens, granadilla). In particular, animal distribution was limited, reaching 35-30% of first round beneficiaries (likely mostly SMFs), which equaled 14-16% of all beneficiaries. The 2014 PE then found that regularity of input provision was variable, and in particular that the animal pass-on scheme was slow to function, causing tension among beneficiaries and limiting potential nutrition impact. At endline, significantly more women in intervention areas owned animals than in control (around 65% of women in intervention areas), though far fewer reported themselves able to dispose of these assets without authorization (around 12%). 3. training cascade- CDFs hired by MCDA, along with ministry of agriculture employees, were used to train SMFs. In the 2013 PE, the process of training was found to be effective, with adequate use of materials and use of teaching aids and engaging training sessions and high satisfaction from participants. There were some challenges with refresher training participation however, mostly due to seasonal weather issues. SMF training was found to be a little chaotic, with women bringing their babies to the classes, which could undermine detailed learning. There was some concern over the volume of topics covered over the residential training, and therefore the quality of learning. In the knowledge assessment in 2013, agriculture supervisors were found to focus on quantity of food as a cause of malnutrition, and at that point SMFs did not appear to have good nutrition knowledge. 4. RAIN materials- The 2013 PE found The nutrition training uses the government IYCF manual, which is comprehensive in scope and locally-relevant, and the IYCF counseling cards are in a locally-appropriate language. The agriculture manual is also comprehensive and clear. It is likely that some CHVs have already been trained using the IYCF manual, so there is a question over value added by the RAIN nutrition training for them. SMF training is likely to be new to them, given their previous training and experience. In the nutrition manual the focus is on IYCF only, and not on the broader determinants of nutrition, and the agriculture manual does not contain nutrition considerations; requiring frontline workers to consider agriculture, nutrition and health holistically or connection with the aims of RAIN therefore are not happening at training level. There is some concern over literacy levels of CHVs and SMFs, and ability to read and assimilate the content, particularly of such a long manual, but the embedded illustrations and quality of training sessions themselves could overcome this. It is unclear that more frontline workers can read Tonga than English, but having manuals in both languages is likely to be an advantage. Links with RAIN are not clear in the manuals, either through content or branding. The 2014 PE found that many respondents had not seen some RAIN materials, and that was borne out in the endline survey, where less than 30% of those surveyed had seen RAIN posters, and around 60% had heard RAIN radio messages. 5. Health and nutrition training cascade- Ministry of Health staff were hired to train CHVs on nutrition and IYCF issues. In the 2013 PE, CHV training was observed and reported to be generally participative and clear, with clear learning objectives and sufficient training materials, and engaged classes. IYCF refresher trainings were happening regularly, but suffered from the distances participants have to travel 154

155 (particularly in rainy season) and therefore low attendance. Trainings appeared to be engaging to those who attend, and use similarly participative and clear teaching methods. Government IYCF trainers were however sometimes not engaged with the aims of RAIN, and reverted to standard IYCF training rather than relating content to RAIN objectives. Health-side training to this point had only been on IYCF, with no attention to broader nutrition, sanitation or health issues, or conceptual or practical links to agriculture. The 2013 PE recommended that Concern needs to think through how the two sectors overlap (for instance, around complementary feeding and maternal diet and time use), and put in place some explicit activities at different levels Concern needs to be explicit about whether the added value is in the technical content of the trainings, or the training in, for instance, how to interact with women s groups, adult education skills, and problem solving skills. In the knowledge assessment, CDF knowledge on nutrition and health was generally poor. The knowledge of health supervisors was at a broadly similar level to that of CHVs, which was adequate. 6. Gender training and inputs cascade- In the 2013 PE, the gender trainings provided by Women for Change were almost universally acclaimed among respondents and there was enthusiasm for the messages. It is likely that the model used by WfC was appropriate for the area, however, due to inadequate monitoring by WfC or Concern, it was unclear how much of the community was covered by these messages. At endline, 22% of households reported participating in RAIN community events, including gender events. Some change in attitudes on gender issues can be seen in the endline empowerment data, particularly in social capital, and a clear shift in women s involvement in agricultural decision-making, but there is still a long way to go. Another side to the gender strategy was the provision of labor saving technology in the form of fuelefficient stoves, to reduce the burden of firewood collection. After difficulty finding community groups to manufacture these, some stoves were made and distributed, but these did not cover all beneficiaries, and there was no difference at endline in type of fuel used for cooking (though amount used was not assessed). 7. Borehole rehabilitation and water access- Several boreholes were assessed and rehabilitated in the project area at the start of the project. At baseline, around 63% of respondents in all study arms reported using a borehole for drinking water; at endline, this had risen to 70% in the control area and 83% in the intervention areas, however differences were not statistically significant. However, the 2013 PE noted that the water issue in Mumbwa, as relates to home gardens, is bigger than borehole rehabilitation; water availability generally has led some respondents in the evaluation to question whether this was the best model for this agro-ecological area. Borehole rehabilitation is taking longer than anticipated due to procurement issues (part of a broader procurement problem at CWZ), without clarity that the result will be more water for gardens. 8. Establishment of women s groups- Women s groups were established successfully in the project areas. The 2013 PE noted that Both monitoring data and the responses of focus group participants suggest that women are joining the women s groups; monitoring data is not sufficient to say whether the number of women joining is as expected, or what proportion of women in different areas are not joining. There was no exit strategy for transitioning women out of the groups as their children aged, but inputs were only provided once to each woman, and IYCF messaging would become repetitive over time, so a natural fall-off of women was anticipated over time. Initially, there were also no plans for ongoing enrollment of women beyond the first two years of enrollment rounds, but in planning for the endline it became clear that without ongoing recruitment it would be likely that we would not see enough beneficiary women with children aged 0-24 months in the endline survey, so additional rounds of recruitment were added through 2014 and

156 In 2013, Concern undertook a Participation Survey in order to understand participation rates and reasons in the absence of monitoring data. Purposively selecting 7 villages in two wards, this study suggested that overall around 88% of eligible households were participating in RAIN, ranging from 100% to 70% in the villages surveyed, with a majority of those not enrolled citing not being available at the time enrollment was taking place. However, at endline only 35% of households with an eligible woman reported having a member of a RAIN women s group. Investigations by Concern are ongoing into flawed Census numbers that may have led to an underestimation of eligible women in the project area; it is also clear that the 2013 Participation Survey used a flawed methodology. Of women who did join the RAIN groups, attendance was generally high. The 2013 PE found that General attendance is seasonal, with rainy/agricultural season attendance dwindling to nothing. The monitoring data are not clear enough to say what proportion of beneficiaries is actually attending regularly. The focus group data indicates that at least some women are attending the groups. The 2014 PE found that Enrolled women are attending groups and perceive them positively. The 2013 Participation Survey found that of those households enrolled but not attending regularly, a majority cited clashes with other household business, not benefiting from inputs, and being unaware of meeting dates as key reasons. At endline, 50% of those who reported being a member of a RAIN group received medium or high levels of program exposure, comprising attendance and delivery. 9. Frontline program delivery- The program was delivered in multiple streams (predominantly agriculture, nutrition, and gender streams, but also labor-saving technology and water streams). Apart from gender, which was a community-focused intervention, these streams came together at the level of the RAIN groups, with intervention streams delivered according to study arm (ag-nutrition or ag-only), with supplemental home visits planned from both agriculture and health frontline workers. On the agriculture side, 90% of RAIN group meetings were attended by an SMF, and 50% of participating households received home visits in the first half of 2015; overall, 39% of surveyed households reported receiving any agricultural training, of which 61% was provided by RAIN. This tallies with the overall participation rate of around 35% of households. On the health side, CHVs were present at around 40% of RAIN group meetings- but notably were also present at around 40% of RAIN group meetings in agonly areas also. Only 14% of participating households received a home visit from a CHV, and 8% of households did in the ag-only areas. The 2013 PE found that Little information is available on home visits, either from monitoring data, frontline interviews, or focus groups- we are led to believe these are not happening as intended, which would appear to still be the case in The 2013 PE found that Frontline workers spend an average of 3-4 hours per day, 4-6 days per month on community work, though there is quite a lot of variation within this. A majority of CHVs in both RAIN and non-rain areas say that their workload has increased since one year ago (mostly due to increased number of people to visit or an increased area to cover), but most SMFs do not; no RAIN workers say they cannot manage their workload, but around a quarter of RAIN CHVs and 40% of non-rain CHVs say it is too much. Time is a constraint to CHVs (but not so much to SMFs), with community workloads affecting other responsibilities and vice-versa; other responsibilities include agriculture, childcare, and domestic duties. Thus motivation, confidence and enjoyment was generally high among frontline workers, who are motivated by a combination of community spirit and personal development, but was lower among health-side workers, who received fewer incentives and worked through government systems. An explanation suggested by Concern for the low CHV engagement may have been that the CHVs were not selected among the group members (as the SMFs were), but they were selected by the health facility. This resulted in CHVs being either men, or much older than the group members. This may have influenced the participation of the CHVs in the project. 156

157 The other key frontline role in RAIN was the CDF, a cadre of community development workers recruited by MCDA to train, coordinate and supervise the SMFs and CHVs. There were originally 8 CDFs for the entire project, which was increased to 12 when ongoing recruitment was established and beneficiary numbers rose. Thus there were very few CDFs to coordinate the work of many frontline workers and RAIN groups, and these were cited by some RAIN project workers as a bottleneck for the program. The 2013 PE found that CDFs have been an issue for the project so far, with several replaced recently Frontline workers appear ambivalent to supervision structures, which neither excessively help nor hinder them in their routine work. A key aspect of frontline delivery was the combined facilitation of groups by SMFs and CHVs in the agnutrition areas. At endline, while SMF participation in RAIN groups was high, CHV participation was low; group members therefore had more opportunity to interact with trainers from the agriculture side than from the health side. The 2013 PE found that strategic coordination between sectors was not happening, from RAIN management down to field level: CDFs appear to have stronger links to SMFs than to CHVs, who report directly to government staff (in theory); this further underlines the disconnected nature of the RAIN project s agriculture and nutrition elements. At this time, there was little interaction between sectors, and a lack of planning for the harmonization of frontline worker training, or beneficiary messaging at group meetings. The major issue identified for RAIN project delivery related to a lack of harmonization of inputs, messages and information provided to groups between the agriculture and health sectors of RAIN; this lack of coordination was seen at all levels from management to manual design to training to frontline worker interaction, and was therefore not surprising at the level of service delivery. By design, the burden of integration is placed at the level of the program beneficiaries, who are left to make their own links between what the program advocates in terms of agriculture, and in terms of child feeding; without integration in service delivery, the current model requires mothers in women s groups themselves to make the links between agriculture and nutrition. The following advice was given to Concern in the 2013 PE report: The structures that are in place for reporting and the pressure to implement fast are not conducive to strategic planning that is vital for the achievement of RAIN aims. Concern needs to think explicitly about reconciling the two implicit aims for RAIN- to provide services to beneficiaries, and to produce evidence on agriculture interventions for nutrition; these aims are not mutually exclusive, but certain elements of the project (such as capacitybuilding of local partners, and HR processes) need to be addressed with this explicit lens. While the theory underlying the RAIN interventions is clear, this is not being translated well in implementation, in its current form; it is important to get service delivery right, otherwise impact at the level of households is unlikely to be realized. Tweaks made now will likely greatly improve the quality of the intervention, increase exposure and utilization of services, and increase adherence to recommended behaviors. 10. knowledge- Monitoring data show that agriculture was covered in trainings at an average of 80% of group meetings by mid-2015, making agricultural knowledge and skill dissemination one of the strongest elements of the project. Concern s Irish Aid Programme Funding (IAPF) evaluation found that many women reported using new skills such as seed saving, and many beneficiaries noted that they will continue to practice the new approaches to agriculture, nutrition and gender equality that they have learned through the program; many have built relationships with their SMFs who continue to live within the community. In the second round of process evaluation in 2014, women in intervention areas were much more confident that they could maintain a garden than their counterparts in the control area (over 90% in intervention groups, compared to 60% in control). The IAPF evaluation also noted, however, that it was clear that some of the beneficiaries were at different levels in terms of knowledge and practices, and that those beneficiaries with the highest level of dependency on the 157

158 project are often those who need the most support maintaining these practices. As monitoring data show the proportion of SMFs maintaining teaching gardens fluctuated from less than 30% to over 80% in different months (2014 and 2015), and not all eligible women were reached by the project, the overall population impact on agricultural knowledge and skills among women was not as strong as it could have been if all were reached and all had access to demonstration plots. 11. Health and nutrition knowledge- At endline, patterns in changes of IYCF knowledge were inconsistent, in both intent to treat and per-protocol analyses. Overall, IYCF knowledge had increased over time in the ag-nutrition group compared to control on some information and decreased on other information. Hygiene knowledge was significantly different among study arms at endline for many aspects relating to hand washing; protecting children from worms; and making drinking water safer, but the direction of difference was not consistently favorable to a single study arm, or to the RAIN intervention arms. In the 2013 PE, there was a question around the added value of RAIN trainings over routine government trainings for health volunteers, where the curriculum and training materials used are those provided and used by the government. It is therefore possible that the RAIN nutrition and health messaging was not adding additional information, and that women were receiving this information through government channels already. This would suggest that linking CHVs to a group model did not add value to the program. 12. Establishment of gardens and vegetable production- At endline (in the dry season) around 13% of households used some land for the establishment of a garden, as opposed to 6% in control areas. The 2013 PE (in the wet season) found that most women participating in FGDs [sampled of beneficiaries and SMFs] had established gardens. However, it is important to note that women distinguish between rainy season and dry season gardens; the latter are harder to establish because of water supply issues. There was a significant increase over time and compared to control of households producing any vegetables and fruits (34% in ag-only and 41% in ag-nutrition groups at endline compared to 22% in control), and more intervention households were producing vitamin-a rich plant foods, and for more months of the year. 13. Establishment of small animal production- Procurement and distribution of small animals was problematic in the RAIN project. It was difficult to procure sufficient numbers of goats locally; chickens frequently died of Newcastle disease; and the pass-on schemes were slow and caused tension among some beneficiaries. The 2013 PE found that While construction of chicken houses is reported to be common, goat houses are less frequent because of the expected delays in receiving goats through the pass-on system. At endline, there were no significant differences between study arms in ownership of animals or production of animal products, though intervention arms did have more months producing dairy. 14. Improved post-harvest management- The main strategy for post-harvest management was drying of vegetable foods to extend the season for which these were available to eat, and drying of seeds for re-use (though many seeds provided by the project were hybrid variety, and therefore would not yield so well in following years). The 2013 PE [sampling banaficiaries and SMFs] found that Use of solar dryers and seed banking techniques was widely reported in FGDs, and seems to be going well, although not all types of seeds are kept: this is determined by availability/ type of seed (hybrid seed cannot be reused). However only 8% of women in intervention areas reported having access to a solar dryer at endline, suggesting that coverage/access to these did not keep pace with beneficiary recruitment. 15. Change in resources controlled by women- At endline, impacts on women s economic empowerment were not consistent. Women s overall financial empowerment changed little since baseline, and was slightly worse in intervention areas. Access to assets for women was slightly better in intervention areas however, and in specific domains more closely related to the project s aims 158

159 (ownership of animals) and long-term female-specific savings (ownership of jewelry) the intervention areas were doing significantly better. In agriculture specifically, women in intervention areas had significantly less input on decisions relating to use of money from the sale of animals but slightly more on money from the sale of crops. 16. Change in women s decision making- In the 2014 PE [purposive sample of beneficiaries], women in RAIN intervention areas were confident in both producing and feeding the foods children need. At endline, decision-making and equality scores were no better than control. More broadly that decision making however, women s social capital scores in terms of interactions with her spouse, social interaction, support, and information sharing were significantly improved in RAIN intervention areas compared to control, likely due to interaction facilitated by RAIN groups. 17. Change in women s time use- At endline, there were no significant impacts on women s time useeither positive or negative. 18. Improved food security and diets- Overall, respondent s perception of their household food security decreased significantly over time for all groups, in both the full sample, and among confirmed RAIN beneficiaries, though there was a significant positive program impact on the level of severe hunger. At the same time, the Ag-Nutrition group had significant positive impacts on household dietary diversity, with an increase of about 1 food group, based on a 12 food group scale. There were significant improvements in maternal dietary diversity (measured out of a total of 7 food groups) within study arms over time, in both intent-to-treat, and per-protocol analysis, but there were no differential changes over time, in favor of any study group, in impact analyses. 19. Improved health practices- RAIN did not try to change health practices beyond hygiene. At endline, vitamin A supplementation and deworming coverage were fairly high in intervention areas, but these differences were present at baseline. Overall child morbidity improved in all study arms over time, but was worse in the ag-only arm compared to the other arms. There were no clear patterns in reduction of disease over time. 20. Improved IYCF practices- There were no differences between groups, at endline, in any of the core WHO IYCF indicators.. There were significant increases in several complementary feeding indicators (with increases ranging from 6 to 12 pp for different indicators, in the RAIN intervention groups), but these increases were not differential in favor of any group, either compared to each other, or the control group. In general, all breastfeeding-related IYCF indicators were high at endline, across all three study groups. Complementary feeding practices were sub-optimal ranging from approximately 25-30% for the minimum acceptable diet indicator, to 60% for the minimum meal frequency indicator. Concern s 2013 Barrier Analysis suggests that perceived self-efficacy, perceived positive consequences, perceived social norms, perceived action efficacy, and perceived risk are the key determinants for exclusive breastfeeding among the target population, and that perceived self-efficacy and perceived social norms are the key determinants for dietary diversity during complementary feeding. Access to foods was mentioned as a barrier to maternal but not child dietary diversity. 21- Change in stunting rates- Between baseline and endline, the prevalence of stunting decreased significantly in all three study groups, but decreased more in the control group. Conversely, the prevalence of wasting increased in all three study arms, but increased less in RAIN intervention areas. While we do not have data to explain the overall changes in prevalence, the presence of a control group allows us to look at differential changes related to the project. By this metric, the RAIN project had no impact on stunting, and a potentially protective effect on wasting. 159

160 160