Cost effectiveness of responsive stimulation and nutrition interventions on early child development outcomes in Pakistan

Transcription

1 Ann. N.Y. Acad. Sci. ISSN ANNALS OF THE NEW YORK ACADEMY OF SCIENCES Issue: Integrating Nutrition and Early Childhood Development Interventions Cost effectiveness of responsive stimulation and nutrition interventions on early child development outcomes in Pakistan Saima Gowani, 1 Aisha K. Yousafzai, 2 Robert Armstrong, 3 and Zulfiqar A. Bhutta 2 1 Department of Education Policy and Social Analysis, Teachers College, Columbia University, New York, New York. 2 Department of Pediatrics and Child Health, Division of Women and Child Health, Aga Khan University, Karachi, Pakistan. 3 Medical College, Aga Khan University, Nairobi, Kenya Address for correspondence: Saima Gowani, Teachers College, Columbia University, 525 W. 120th Street, Grace Dodge Hall, Room 371, New York, NY Saima.Gowani@gmail.com Early childhood programs are heralded as a way to improve children s health and educational outcomes. However, few studies in developing countries calculate the effectiveness of quality early childhood interventions. Even fewer estimate the associated costs of such interventions. The study here looks at the costs and effectiveness of a clusterrandomized effectiveness trial on children from birth to 24 months in rural Sindh, Pakistan. Responsive stimulation and/or enhanced nutrition interventions were integrated in the Lady Health Worker program in Pakistan. Outcomes suggest that children who receive responsive stimulation had significantly better development outcomes at 24 months than those who only received enhanced nutrition intervention. A cost-effectiveness analysis of the results verifies that early childhood interventions that include responsive stimulation are more cost effective than a nutrition intervention alone in promoting children s early development. Costs of a responsive stimulation intervention integrated in an existing community-based service providing basic health and nutrition care is approximately US$4 per month per child. We discuss these findings and make recommendations about scaling up and costs for future early child development programs. Keywords: cost effectiveness; early childhood development; nutrition;responsivestimulation;pakistan Introduction There is increasing recognition that the first few years of a child s life are a particularly sensitive period in children s development, laying a foundation for their cognitive functioning, their behavioral, social and self-regulatory capacities, and physical health. Yet, many children face stressors during these years that can impair their healthy development. Early child development (ECD) programs are designed to mitigate the factors that place children at risk of poor developmental outcomes. Numerous studies have provided evidence supporting the conclusion that ECD programs make a significant positive contribution to childhood development, in both the short (e.g., improved physical health, cognitive skills, and school readiness skills) and long term (e.g., academic achievement and attainment in schools, as well as reduced remedial education and dropout). 1,2 Thesepositiveeffectsareparticularlystrongfor children from disadvantaged backgrounds and exposed to multiple risks, such as malnutrition, inadequate psychosocial stimulation, and poverty. 3,4 Although ECD programs can have a demonstrated positive affect on child development, policymakers often require more information than is currently available to make ECD investment decisions. Cost analyses, such as cost benefit or costeffectiveness analysis, of ECD programs are rarely conducted. 5 Only a limited number of international studies have calculated the economic benefits of ECD programs. Cost analyses are more readily available on the benefits of preschool (focusing on children ages 3 years to school-aged); however, limited information is available on the doi: /nyas

2 Cost effectiveness of ECD interventions in Pakistan Gowani et al. cost effectiveness of programs working with younger children. 5 There are several limitations in conducting cost analyses of ECD programs. Generally, it is difficult to identify which programs provide the greatest return on investment, because different programs monetize different outcomes. 6 For example, whereas economic evaluations in high-income countries may focus on the net-financial benefits to society (as measured by reduced participation in the criminal justice system or increased maternal employment leading to higher taxes being paid into the system), 2,7 economic analysis of an ECD program in a low- and middle-income country (LAMIC) calculates benefits based on increased productivity and reduced fertility. 8 Because the list of benefits varies between programs, it is hard for policymakers to decide among programs and their individual merits. Moreover, it can be very difficult, if not impossible, to estimate a monetary value for all the outcomes a given program aims to affect. Home visiting programs, for example, commonly promote outcomes like improvement in children s socialemotional skills, improved mother child interactions, or increased use of language with young children. It is difficult, however, to monetize the effects of better language usage or improved social skills at 2 years on overall development as adults. Hence, many cost benefit evaluations overlook meaningful outcomes in their analysis. 8 Cost effectiveness analyses have their own limitations. Cost-effectiveness analyses only allow one effectiveness measure to be evaluated at a single time. Hence, programs that have multiple outcomes must choose one measure to calculate the cost effectiveness, which reduces the complex and integrated nature of ECD programs into a single outcome. 9 Keeping all caveats in mind, economic evaluations still present valuable information for policy makers to help them decide between public investment alternatives. Given the limited evidence of the cost effectiveness of ECD programs in LAMIC, this paper adds to the existing literature by calculating the costs and effects of an integrated responsive stimulation (RS) and nutrition intervention in a public sector community health service, the Lady Health Worker (LHW) program in rural Sindh, Pakistan. Methods Background of the research trial The LHW program selects community women to assist in family health initiatives. Their main tasks are to maintain records of all married couples, pregnant women, births and deaths in their catchment; offer family planning advice, distribute contraceptives, provide care for pregnant women (including iron and folate supplementation), health and hygiene education, basic child nutrition education, monitor child growth; provide education about Malaria, HIV/AIDS, and TB, and make referrals to primary healthcare facilities. LHWs are responsible for visiting five to seven households per day in a total catchment of 1000 persons. 10 Program data suggest that each home visit should last minutes. 11 LHWs are supported by a lady health supervisor (LHS); the expected LHS: LHW ratio is 1:20 25, but there are many uncovered supervisor posts. The LHSs have access to a vehicle and are expected to have at least two supervisory contacts with the LHW, including one in her catchment. LHSs are also expected to complete a supervisory checklist as a component of an effective or supportive supervisory strategy. Our research team was interested in strengthening ECD outcomes by integrating a RS intervention in LHWs routine services. LHWs selected developmentally appropriate play and communication activities, and mothers (primary caregivers) had the opportunity to try the activity with their young children and receive coaching and feedback from the LHW on how to build the quality of the interaction and enhance responsiveness in the child. Following discussion with the LHW program management, it agreed to integrate this new intervention in existing routine home visits, but also to optimize delivery through the monthly community group strategy (an aspect of the existing LHW program that was not optimally delivered because of a lack of objectives and curriculum). Moreover, an external evaluation by the Oxford Policy Management group 11 reported that impact on children s nutrition well-being was limited in the LHWprogram.Giventhatnutritionisariskfactor for children s development outcome, it was necessary to evaluate whether an enhancement of the 150

3 Gowani et al. Cost effectiveness of ECD interventions in Pakistan basic nutrition education messages delivered by the LHWs may improve growth and whether the combination of RS and nutrition interventions would have additive benefits to development. The cluster-randomized controlled effectiveness trial, with a 2 2 factorial design, was implemented by a research team from the Aga Khan University (AKU) to evaluate the addition of RS, enhanced nutrition (EN), and combined RS and EN (integrated) interventions in the LHW program on children s development and growth outcomes. Hereafter, the trial will be referred to as the Pakistan Early Child Development Scale Up (PEDS) trial. Because the PEDS trial aimed to test the feasibility of integrating interventions in an existing government health program, implementation was done within the existing LHW structure; intervention dosage, frequency of home visits, supervisory support, and refresher trainings were implemented as stipulated in the LHW program. Twenty LHWs were selected for each intervention group: RS, EN, integrated (I), and control (C). The RS group received age appropriate guidance based on the UNICEF and WHO Care for Child Development curriculum. The EN group received a multiple micronutrient powder (MNP) (Sprinkles R ) and additional nutrition education. The integrated group received a combination of both interventions mentioned. All groups continued to receive standard LHW services (including basic nutrition education). Interventions were delivered through a combination of monthly group and home visits. Group visits were approximately 1 h and 20 min each; home visits ranged from seven to 30 minutes. A team of ECD facilitators worked closely with the LHSs and LHWs, and provided training, supervision, and mentorship to the LHWs to support the integration of these new interventions within their existing services. The LHWs were expected to deliver interventions to every child less than 2 years of age in their respective catchments. The research team followed a birth cohort of 1489 children to assess primary outcomes on child developmental and growth. The enrolled birth cohort represents 42% of all young children receiving LHW services in the catchment area. Analysis of baseline data indicated that groups were comparable across the four cells, and any significant differences were controlled for in the analysis of child outcomes. The results at 24 months of age show significant impacts of all three interventions on cognition, language, and motor scores with larger effect sizes amongst the two RS groups. 12 No significant treatment effect was found on growth outcomes at 24 months. The intervention ran for 33 months (July 2009 March 2012) to capture data for children at 24 months of age who were born later into the cohort. Additional background information is presented in Table 1. Method for cost-effectiveness analysis A mixed-method approach was used to collect data to inform the costing analysis. Quantitative data came from project budget and expenditure reports and qualitative information was garnered from program staff to understand the depth of the implementation and the associated costs. Informant interviews validated the quantitative findings about costs incurred during the interventions. The cost-effectiveness analysis was organized into four phases: (1) review of budget/expenditure sheet and consultation with project staff about costs; (2) calculating costs; (3) calculating effectiveness; and (4) defining the cost-effectiveness ratio with discussion of recommendations. Phase I: Review of budget/expenditure statements. We utilized the ingredients method 9 as an organized way of thinking about the incremental costs of physical resources necessary to implement the three interventions within the existing LHW program. We grouped an initial list of ingredients within categories utilized in other LHW expenditure reports. 11 Thecategoriesused for the cost analysis are (1) technical support; (2) supportive supervision and mentoring; (3) LHWs; (4) facilities; (5) materials and resources; (6) capacity building for management; (7) capacity building for LHWs; and (8) other program inputs. The research team reviewed the budget and expenditure documents from the PEDS trial and isolated the intervention costs from the research costs, only including intervention costs in the analysis. Additional line items were added to the ingredients list that would not have appeared as expenditure by the research trial, but were costs of implementing the intervention -for example, LHW salaries. Once all expenditures were reviewed, the list of ingredients was finalized. Phase II: Calculating costs. Costs for the three intervention models were then calculated. AKU 151

4 Cost effectiveness of ECD interventions in Pakistan Gowani et al. Table 1. Background information about the four groups Control Enhanced nutrition (EN) Responsive stimulation (RS) Integrated Catchments for LHW intervention delivery Number of catchments (clusters) Number of LHWs serving in the catchments at the start of the intervention Number of LHSs supporting the LHWs Average number of households in each catchment (SD) b Average number of children (0 2 years) per catchment (SD) c Enrolled infants for evaluation of outcomes Number of infants enrolled at the beginning of the intervention d Delivery content and strategy Content Treatment of minor ailments, family planning advice, antenatal care, immunization support, health and hygiene, and education, including basic nutrition education messages for infant and young child feeding a (24) 113 (27) 124 (19) 112 (26) 40 (17) 47 (13) 43 (15) 42 (16) Treatment of minor ailments, family planning advice, antenatal care, immunization support, health and hygiene, and education with enhanced nutrition education messages for infant and young child feeding and MMP Treatment of minor ailments, family planning advice, antenatal care, immunization support, health and hygiene, and education, including basic nutrition education messages for infant and young child feeding with responsive stimulation Treatment of minor ailments, family planning advice, antenatal care, immunization support, health and hygiene, and education with enhanced nutrition education messages for infant and young child feeding and MMP and responsive stimulation Continued 152

5 Gowani et al. Cost effectiveness of ECD interventions in Pakistan Table 1. Continued Control Enhanced nutrition (EN) Responsive stimulation (RS) Integrated Basic program delivery strategy e Averagetimeto conduct one home visit (from intervention monitoring data at the cluster level) f Evidence of content delivered at home (from household report of last visit from interview with enrolled mother) Group meetings conducted as part of the intervention g (from intervention monitoring data at the cluster level) Participation of group meetings (from intervention monitoring data at the cluster level) Monthly home visits to every household in catchment (5 7 households per day for min duration) and a monthly group meeting Monthly home visits to every household in catchment (5 7 households per day for min duration) and a monthly group meeting Monthly home visits to every household in catchment (5 7 households per day for min duration) and a monthly group meeting 7 min 11 min 30 min 30 min 12% reported receiving a nutrition message Occasional monthly community group sessions conducted (health awareness session) 75% reported receiving a nutrition message Occasional monthly community group sessions conducted (health awareness session) 75% reported receiving a nutrition message, and 62% reported receiving responsive stimulation advice Yes (average duration 1hand20min) N/A N/A 31% of female caregivers Monthly home visits to every household in catchment (5 7 households per day for minutes duration) and a monthly group meeting 77% reported receiving a nutrition message, and 55% reported receiving responsive stimulation advice Yes (average duration 1hand20min) 31% of female caregivers a Four LHWs from this cluster left their posts officially before the end of the intervention. b Approximately 1000 total population as per guidelines of the LHW program (Hafeez et al. 10 ). c Interventions delivered to all children in the catchment. d All infants born in the cluster from April 2009 March 2010 were identified and invited to enroll for outcome evaluation by an independent surveillance team. e LHW program details have been discussed by Hafeez et al. 10 f Seventy-eight percent of all households in the clusters reported receiving their last expected monthly routine home visit from the LHW. g The group meetings strategy was developed after stakeholder meetings with the LHW program management group. The enhanced nutrition strategy would complement existing basic nutrition services that were delivered through routine home visits. The responsive stimulation strategy was an opportunity to promote community group meetings that were stipulated in the LHW program guide but were not conducted because neither objectives nor curriculum for the meetings had been designed by the LHW program. 153

6 Cost effectiveness of ECD interventions in Pakistan Gowani et al. provided data on actual and assumed costs for the interventions. Phase III: Effectiveness analysis. Next, intention to treat analysis following the factorial design was conducted as part of the PEDS trial. First, exposure to RS with no RS was compared, then exposure to EN with no EN was compared, and finally the interaction between the two treatments was examined to determine whether there was any additive benefit when combining the two. For this analysis, child development outcomes (assessed by the Bayley Scale for Infant and Toddler Development, Third Edition BSID III) were employed because they yielded significant differences between the two interventions. 12 Phase IV: Cost-effectiveness analysis and considerations for scaling up. Lastly, costeffectiveness ratios were calculated to provide the cost per unit of effectiveness. 9 The intervention with the lowest ratio of cost effectiveness is considered the most cost effective. Discussion of the results and considerations on scaling up the interventions into the national program follow. Results Cost analysis and description Table 2 summarizes the actual costs of the three interventions. Intervention costs are broken down into ingredient categories listed above. Some general costs of the program were shared equally amongst the three intervention cells. In other cases, costs were specific to the different intervention packages. Technical support. All technical support costs were shared equally amongst the three interventions. The management team based in Karachi supporting the intervention consisted of the coprincipal investigator and an research associate. This role is similar to the role of the national LHW Trainers who support the provincial training teams. Because the management team divided their time equally between research and intervention, half of the management team s salary was included as intervention costs. Additional costs included supportive supervisory visits from the management team to the team members who were based at the intervention site. Fourday visits occurred three times a month for the duration of the intervention. One ECD coordinator was hired to oversee administrative affairs and training for the ECD facilitators. Supervision and mentoring. Cost for mentoring varied on the basis of the number of LHSs. Because catchment areas were randomly chosen, each group had a different number of supervisors appointed to work with the LHWs (the result of vacant posts in the LHW program). Therefore, three LHSs supported the EN cell, four supported the RS cell, and two supported the integrated intervention. Six ECD facilitators were recruited for this study. Similar to the LHSs, ECD facilitators mentored the LHW, observed them during home and group visits, and offered feedback and coaching to LHWs to support their interactions with families. Each ECD facilitator used a monitoring sheet, similar to the LHW supervisory checklist, to collect data on the progress of the LHWs. Each LHW received a minimum of two supervisory contacts per month, and a supervisory checklist was completed one time per month. LHSs earned a salary of Rs. 7000/month (US$77) from the government for their work. Moreover, LHSs received a stipend of Rs. 100/month (US$1.10) to support the LHWs for this intervention. Lady health workers. Twenty LHWs were employed in each cell, earning a salary of Rs (US$38.59) at the start of the intervention. In August 2011, LHW salaries were increased to Rs. 7,000/month (US$77.18). LHWs earned an additional Rs. 100/month (US$1.10) stipend for assisting in the PEDS interventions (the rationale for a small monetary incentive was that the new interventions were integrated into existing routines and no additional visits were expected). Salaries for the LHWs vary across intervention groups because in year 2 of the intervention, four LHWs from the integrated cell left their jobs. Salaries and stipends were not paid to these LHWs in the second year of the intervention. Facilities. The ECD coordinator and ECD facilitators shared an office space with other research and data collection staff. This cost was shared equally amongst the cells. Materials and resources. Various materials were given to each LHW. Job aides were printed for each LHW that contained materials that the LHW used during home and group visits. In addition, 154

7 Gowani et al. Cost effectiveness of ECD interventions in Pakistan Table 2. Costs of the three treatment alternative interventions (in US$ a ) Enhanced nutrition (EN) Responsive stimulation (RS) Integrated Technical support $ 23,789 $ 23,789 $ 23,789 Management team supporting intervention 15,404 15,404 15,404 ECD coordinator Supportive supervision Supervision and mentoring $ 15,043 $ 17,626 $ 12,460 ECD facilitators Lady health supervisors salaries b , Incentives lady health supervisors Monthly record keeping sheets LHWs $ 31,594 $ 31,594 $28,609 LHWs salaries c,d 30,866 30,866 27,934 Incentives LHWs Facilities $ 1011 $ 1011 $1011 Rent of office space Materials/resources $ 2613 $ 4773 $5599 Manuals (job aides) Stationary/supplies Sprinkles Resource kits Play and communication guides Picture books Refreshments Capacity building management $ 1391 $ 1391 $ 1391 ECD coordinator training ECD facilitators training Training support materials Capacity building LHWs $ 421 $ 509 $ 685 Baseline training Refresher training Other inputs transportation $ 6791 $ 6791 $ 6791 Vehicles Total $ 82,651 $ 87,482 $ 80,334 Cost per month $ 2505 $ 2651 $ 2434 Costs per year $ 30,055 $ 31,812 $ 29,212 Cost per LHW (annualized) $ 1503 $ 1591 $ 1461 a 1US$= 90.7 PKR (as of 5/ b Because the LHWs were chosen at random, the intervention supported and trained the lady health supervisors who were responsible for each intervention cell. Three LHSs supported the enhanced nutrition cell, four LHSs supported the ECD cell, and two LHSs supported the integrated cell. c Twenty LHWs were chosen at random from the areas of intervention. Each LHW was responsible for a different area of the overall district that was chosen as the intervention area. d In the second year of the intervention, four LHWs from the integrated cell were not considered functional. Hence, salaries and incentives were only paid for 16 LHWs in the integrated cell in year

8 Cost effectiveness of ECD interventions in Pakistan Gowani et al. various stationary and office supplies were procured for the trial. We assume that half of the total program costs were for the intervention and have included the amount as such. The EN and integrated interventions also included MNP packets to families of children 6 24 months of age and an illustrated nutrition education guide. In the RS and integrated interventions, LHWs received resource kits that included play materials and toys to show parents developmentally appropriate games and activities that could be played using household items and homemade toys. A play and communication guide was developed to aid LHWs with activity and game ideas for children. For each parent that participated in the group sessions, a small picture book was given to them. The picture book was only given once per family, irrespective of whether the family chose to participate in future group or home visiting sessions. As a token of participation, a cup of tea/juice and biscuits were offered to all mothers and their children who participated in the group sessions. Capacity building management. For the purposes of the intervention, an initial management training (one session training for 5 days) was offered. In addition, to train the staff more effectively, a video camera and other multimedia supplies were purchased. This helped the ECD coordinators see how LHWs were conducting the sessions, and use the information for feedback in refresher trainings and supervision contacts. All costs were equally shared across the three cells. Capacity building LHWs. Each cell trained 20 LHWs; however, each cell received different training days to equip the LHWs with the knowledge and capacity necessary to be functional in their tasks. EN training lasted two days; RS training spanned 3 days, and the integrated training totaled five days. In addition, costs of the facility, materials, refreshments, and training allowance (an allowance expected by the LHW program) for the baseline training were added to training costs. Refresher trainings occurred every 6 months for 1 day at the Health Office. Costs of refreshments for the participating LHW and LHSs made up the costs of the refresher trainings. Other client inputs. To help ECD facilitators conduct field visits in an efficient manner, two vehicles were leased by the project team. Total. Overall, most costs were shared equally across the intervention arms. Costs varied on the basis of the number of LHSs and LHWs supporting the intervention, the number of baseline training days, and the materials and resources utilized. The RS group s intervention annual costs were estimated to be US$87,482 and the EN group intervention costs were US$82,651 annually, whereas the integrated group costs were US$80,000 annually. Even though the integrated group had the most training days, materials, and supplies, it only utilized two LHSs in its intervention areas and had fewer LHW salaries to pay at the end of the trial. Because of the lower cost of human resources of this cell, the integrated intervention was the least expensive to implement. Effectiveness of child development Table 3 displays the results on cognitive, language, and motor development at 12 and 24 months evaluated by the BSID III. The analysis followed the factorial design of the trial to test for additive treatment effects. To calculate the effect of RS on child development, we present the results based on those who received RS versus those that did not. At 12 and 24 months, children who received RS interventions performed significantly better on BSID III outcomes than those that did not receive RS. No significant additive effect was observed when integrating EN and RS interventions. Cost effectiveness Cost effectiveness is calculated by taking the costs per LHW (annualized) for each of the intervention cells divided by the effectiveness figures estimated in Table 3. Table 4 demonstrates that the most cost-effective intervention is the integrated intervention that promotes children s psychosocial and nutritional development in a cohesive manner. This finding aligns with our hypothesis that a combined RS/nutrition intervention would be more cost effective than RS or nutrition alone. However, these data should be interpreted with care because reviewing the cost structure suggests that the lower costs of human resources specifically LHSs salaries led to the cost savings in the integrated group. Because these were actual costs, the initial analysis has reported the results as such. The sensitivity analysis provides cost-effectiveness data in a scenario when cost structures for the human resources are equal across all groups. 156

9 Gowani et al. Cost effectiveness of ECD interventions in Pakistan Table 3. Effectiveness outcomes on BSID III with and without responsive stimulation Without responsive stimulation a With responsive stimulation 12 Month BSID III cognitive composite score *** 24 Month BSID III cognitive composite score *** 12 Month BSID III language composite score *** 24 Month BSID III language composite score *** 12 Month BSID III motor development composite score ** 24 Month BSID III motor development composite score *** Notes: 12 months: responsive stimulation n = 696, no responsive stimulation n = months: responsive stimulation n = 701, no responsive stimulation n = 680. Source:Yousafzai,et al. 12 ***P < **P < a Without responsive stimulation includes outcomes for children from the enhanced nutrition and control groups. With responsive stimulation includes outcomes for children from the responsive stimulation and integrated groups. Sensitivity analysis Because varying supervisory costs were not a design feature of the study but instead the reality, we equalized supervisory and other variable costs across the three cells (three supervisors and 20 LHWs) to understand which intervention alternative is the most cost effective under generalizable conditions. Results are presented in Table 5. This analysis shows that RS intervention was the most cost effective across various outcomes of interest. Interesting to note, whereas the difference in supervision costs had a large effect in driving actual costs, it does not explain the difference in performance when supervisory costs are presumed equal, suggesting a critical role for ECD facilitators (or master trainers/technical collaboration) when rolling out new intervention in existing programs. 13 In general, it is estimated that the cost of implementing the three interventions within existing health and basic nutrition services ranges from US$ per child per month. Stated differently, the cost of integrating a RS intervention within the existing community health program to parents of children under 2 years of age is estimated to cost $4 per child per month. This cost is an upper bound estimate because duplicate human resources were procured for the trial; when scaling up, those costs would be absorbed by the existing LHW program budget. Discussion The PEDS trial interventions introduced RS into an existing government health program and yielded positive development outcomes for children. Although it is acknowledged that the basic nutrition services need to be improved further to achieve better child growth 11,12 (and potentially additive benefits to child development), the current results suggest that the integration of RS did not negatively affect delivery of existing services or harm early childhood outcomes pertaining to health and growth. Parents saw positive changes in their children by using the RS materials, which led to a positive cycle of appreciation for the LHW, acceptance of the information she presented, and greater motivation from the LHW in delivering the information. 14 When evaluating the cost effectiveness of the interventions, in both real terms as well as under generalizable conditions, RS interventions yielded the most cost-effective outcomes at 12 and 24 months of age. In scaling up the RS interventions, attention must be given to how best to integrate and share the cost burden of the PEDS trial interventions with the existing LHW model. For the purposes of the trial, a parallel supervision system was created to determine a functioning quality model of supervision support that could potentially lead to recommended changes in the supervision approach in the LHW program. 157

10 Cost effectiveness of ECD interventions in Pakistan Gowani et al. Table 4. Cost-effectiveness ratios for the PEDS trial interventions Without responsive stimulation With enhanced nutrition (EN) With responsive stimulation Without enhanced nutrition (RS) With enhanced nutrition (integrated) Costs per year $30,055 $31,812 $29, Month BSID III cognitive composite score C/E 12 months BSID III cognitive Month BSID III cognitive composite score C/E 24 months BSID III cognitive Month BSID III composite language composite score C/E 12 months BSID III Language Month BSID III composite language composite score C/E 24 months BSID III language Month BSID III motor development composite score C/E 12 months BSID III motor Month BSID III motor development composite score C/E 24 months BSID III motor Note: Numbers in bold signify the most cost-effective intervention. This model included the cost of facilities, vehicles, ECD coordinator, ECD facilitators, monitoring and evaluation checklists all of which are already budgeted for in the national program, but perhaps not utilized or always functional. If these costs were covered by the existing program, the costs to scale-up the intervention would be much lower. Next, to maintain the quality of the intervention, additional training and support for supervisors should be considered. In the PEDS trial model, the team hired ECD facilitators to play a supervisory role to support the LHWs in delivering the interventions. To build the LHSs capacity to support the LHWs in a similar way, master trainers could be hired at the local (provincial) levels that support the supervisors. Costs of master trainers would be similar to the management team costs already accounted for in the trial. Moreover, supervisory visits by master trainers would ensure that LHSs have the technical knowledge to support the LHWs in delivering a high quality integrated program to the communities. The fact that the combined program does not increase costs explained by the lower coverage of government supervision posts may not be generalizable. However, like similar programs in other settings, the LHW program may have some slack and may respond to increased training and motivation necessary for change, yielding positive results. In addition, an estimate of how much additional time is necessary to implement the PEDS trial interventions should be reviewed; that is, can the LHWs deliver the PEDS trial content and their existing LHW program requirements during the same visit without creating an excessive work schedule for themselves? PEDS trial results 12 and an external review of the LHW program 11 suggest that LHWs are not spending the requisite time making home visits as per their job description. Data from the process evaluation report of the PEDS trial suggest that the RS interventions are not only 158

11 Gowani et al. Cost effectiveness of ECD interventions in Pakistan Table 5. Sensitivity analysis (n = 20 LHWs per cell, LHSs = 3 per cell, cost in US$) Without responsive stimulation With enhanced nutrition (EN) With responsive stimulation Without enhanced nutrition (RS) With enhanced nutrition (integrated) Costs per year $30,075 $30,892 $31,257 Cost per child (monthly) a $4.04 $4.15 $4.20 Cost per LHW (annualized) $1504 $1545 $1563 Cost of LHW 12 months $1504 $1545 $ Month BSID III cognitive composite score C/E 12 months BSID III cognitive Cost of LHW 24 months $1504 $1545 $ Month BSID III cognitive composite score C/E 24 months BSID III cognitive Cost of LHW 12 months $1504 $1545 $ Month BSID III composite language composite score C/E 12 months BSID III language Cost of LHW 24 months $1504 $1545 $ Month BSID III composite language composite score C/E 24 months BSID III language Cost of LHW 12 months $1504 $1545 $ Month BSID III motor development composite score C/E 12 months BSID II motor Cost of LHW 24 months $1504 $1545 $ Month BSID III motor development composite score C/E 24 months BSID III motor Note: Numbers in bold signify the most cost-effective intervention. a Based on an average of 372 enrolled children per cell. feasible in the time allotted (30 min), but that they also actually assist in explaining nutrition and health information to parents in a more effective manner. 14 However, if changes to the workload are necessary, then structural costs would also be added to the cost of the RS scaled-up model. Furthermore, recent external evaluations by the Oxford Policy Management group indicate impact on children s nutrition well-being in Pakistan needs to be improved; 11 therefore, given nutrition is a risk factor for children s development outcomes, further review is needed on how to improve the basic nutrition services to improve growth outcomes and possibly have additive benefits to the development outcomes. If another nutrition intervention is designed and evaluated, its cost effectiveness should also be calculated to ensure that the most costeffective intervention is being implemented in the national program. A limitation of the cost-effectiveness analysis is the impossibility to sum cost-effectiveness ratios across different outcomes. In this study, the RS interventions had significant effects on different development outcomes. This can be confusing to policymakers who might be interested in an intervention s cumulative effect. To clarify the case further to policymakers, researchers should conduct a cost benefit analysis to calculate the cumulative benefits of the program. Future cost benefit analysis can take into accountcostsavingsofparents(fromfewerhospital visits or other medical issues linked to children s health); cost savings to the education system (from starting school on time and ready to learn, resulting 159

12 Cost effectiveness of ECD interventions in Pakistan Gowani et al. in lower repetition and dropout rates in kacchi [pre-primary] and primary classes); cost-savings beyond the education system (e.g., better health and reduction of crime); and socioeconomic returns such as higher labor force participation and income. Cost-effective models home-based versus group-based visits and intensity Other parenting programs 5 cite the higher expense incurred in doing home visits, because program staff are not able to meet with as many families, and travel costs to individual homes is sometimes costprohibitive. Hence, it would be important to evaluate the efficacy of home visits versus group visits and calculate the cost effectiveness of the two models separately. Group visits are cheaper than individual home visits from a time and reach perspective; however, not all curriculum of parenting programs can be delivered effectively in a group setting. 15 Equally important will be defining the frequency of the intervention because it directly affects program costs. Moreover, cultural and logistical challenges might exist in communities where mothers/caregivers cannot travel away from the family home. Further qualitative analysis of the variation in impact across and within LHW cohorts may begin to identify characteristics that may lead to a more refined set of interventions LHWs can deliver to families and how they should most appropriately deliver them. Targeted versus universal interventions A question for further research is whether the combined intervention is differentially better for some groups versus other groups. With limited funds, it would be helpful to know whether ECD interventions should be targeted to particular groups or those living in particular geographical areas. Given the prevalence of floods in rural Sindh and the international appeal for ECD in emergencies work, there might be more political and financial support available to implement ECD in specific geographical areas of the country. In addition, the problem of food insecure households may need to be addressed by interventions other than MNP and might require specific targeting. Alternatively, given the nature of homes visited by LHWs, the impact of a universal approach across the population may have a greater societal benefit than specific targeted interventions servicing a smaller population. If targeted interventions are considered, a full cost analysis should be conducted, including the bureaucratic costs necessary to manage the targeted intervention versus the cost of universal implementation. Conclusion Policy decisions regarding expenditure of limited public funds require the development of quality information on the cost benefit and cost effectiveness of interventions. This is particularly important for programs that are to be delivered on a large scale like the national LHW program. In a country with high rates of socioeconomic disadvantage, such as exists for families served through the LHW program, the results of this study are encouraging. The analysis suggests that, with further refinement, integrating early stimulation with nutrition support can be scaled-up effectively; on the basis of existing data in other settings, the cost benefit to the country could be very significant. Given the importance of ECD and its relevance in assisting countries meet both its EFA and MDG commitments, it behooves us to scale-up these interventions to achieve national and global child and maternal health and education goals. Acknowledgments The authors wish to acknowledge Clive Belfield, Henry Levin, and Jan van Ravens for their critical feedback and comments on multiple versions of this manuscript. In addition, Jan van Ravens provided oversight during the initial design of this study. Valuable comments were received from Kathy Bartlett, Rebecca Gomez, and Brooks Bowden. The PEDS trial was funded by UNICEF. Additional support was received from the Aga Khan Foundation and private donors. Conflicts of interest The authors declare no conflicts of interest. References 1. Reynolds, A., J.A. Temple, D.L. Robertson & E.A. Mann Age 21 cost-benefit analysis of the Title 1 Chicago Child-Parent Center program. Educ. Eval. Policy Anal. 24: Schweinhart, L.J Outcomes of the HighScope Perry preschool study and Michigan school readiness program. In Early Child Development: from Measurement to Action. M.E. Young & L.M. Richardson, Eds.: Washington, DC: The International Bank for Reconstruction/The World Bank. 160

13 Gowani et al. Cost effectiveness of ECD interventions in Pakistan 3. Walker, S.P., T.D. Wachs, S.M. Grantham-McGregor, et al Inequality in early childhood: risk and protective factors for early child development. Lancet 378: doi: /s (11) Walker, S.P., T.D. Wachs, J.M. Gardner, et al. International Child Development Steering Group Child development: risk factors for adverse outcomes in development countries. Lancet 369: Karoly, L.A., M.R. Kilburn & J.S. Cannon Early Childhood Interventions: Proven Results, Future Promise. Santa Monica, CA: RAND Corporation. 6. Engle,P.,M.Black,J.Behrman,et al.the International Child Development Steering Group Strategies to avoid the loss of developmental potential in more than 200 million children in the developing world. Lancet 369: Schweinhart, L.J., H.V. Barnes & D.P. Weikart Significant Benefits: the High/Scope Perry Preschool Study through Age 27. Monographs of the High/Scope Educational Research Foundation, Number 10. Ypsilanti, MI: High/Scope Press. 8. Van der Gaag, J. & J.-P. Tan The Benefits of Early Child Development Programs: An Economic Analysis. Washington, D.C.: World Bank. 9. Levin, H. M. & P.J. McEwan Cost-effectiveness analysis. (Second Edition). New York: Russell Sage. 10. Hafeez, A., B. Mohamud, M.R. Shiekh, et al Lady health workers programme in Pakistan: challenges, achievements and the way forward. J. Pak. Med. Assoc. 61: Oxford Policy Management Lady health worker programme: third party evaluation of performance. Islamabad, Pakistan: Oxford Policy Management. Retrieved from: pp Yousafzai, A.K., M.A. Rasheed, A. Rizvi, et al Effect of integrated responsive stimulation and nutrition interventions in the Lady Health Worker programme in Pakistan on child development, growth, and health outcomes: a cluster randomized controlled effectiveness trial.karachi:agakhanuniversity.in review. 13. Powell, C., H. Baker-Henningham, S. Walker, et al Feasibility of integrating early stimulation into primary care for undernourished Jamaican children: cluster randomized controlled trial. doi: /bmj c. 14. Yousafzai, A.K. & M.A. Rasheed The Pakistan early child development scale up trial: process evaluation. Project report, Karachi, Pakistan: Aga Khan University. 15. Aracena, M., M. Krause, C. Perez, et al A costeffectiveness evaluation of a home visit program for adolescent mothers. J. Health Psychol. 14: