End-of-Project Technical Report. 13 October 2014

Transcription

1 Measuring the impacts of health facility reinforcement and EID and EPI service integration on HIV testing and immunization services in Southern Province, Zambia End-of-Project Technical Report 13 October 2014 Prepared by: Zambia Ministry of Health Zambia Ministry of Community Development, Mother and Child Health IDinsight Clinton Health Access Initiative Zambia Center for Applied Health Research and Development

2 CONTENTS Acronyms... iii Executive Summary... i Background... i i Evaluation Design... i Results... i Recommendations... ii Background... 1 Study Design and Methods... 3 Study Area... 3 Study Sample and Randomization... 3 Description of Interventions... 3 Data Collection... 5 Pre-Evaluation Activities... 5 Monthly Facility Visits and Administrative Data Sets... 6 Exit Interviews, Focus Groups, and Structured Interviews... 6 Ethics... 7 Data Quality and Monitoring... 7 DBS Data... 7 Maternal Retest Data... 8 Immunization Data... 8 Data Analysis... 9 Exploratory Analysis... 9 Regression Analysis... 9 Secondary Analysis Exit Interviews and Qualitative Analysis Results Infant DBS Testing Differences between Baseline and Intervention Periods Regression Results Time Trends Maternal HIV Retests Differences between Baseline and Intervention Periods Regression Results Time Trends Immunization Differences between Baseline and Intervention Period Regression Results... 19

3 Time Trends Secondary Analyses Age at First DBS Test Time for DBS Sample Processing Exit Surveys Staff Interviews Focus Group Discussions Discussion Interventions Increase HIV Maternal Retesting No Detectable Impact on Immunizations Stock Outs Integration of Services Evaluation Strengths Evaluation Limitations Recommendations Sources Appendix A Timeline of Activities Appendix B Randomization Approach Appendix C Simple Intervention CE model Appendix D Power Calculations Appendix E Additional DBS Regression Results Appendix F Additional Retest Regression Results Appendix G Additional DPT1 Regression Results ii

4 ACRONYMS 3DE Demand Driven Evaluations for Decisions AIDS Acquired Immunodeficiency Syndrome ANC Antenatal Care ART Antiretroviral Treatment ARVs Antiretroviral Drugs BUPIP Boston University PMTCT Integration Programme CHAI Clinton Health Access Initiative CHW Community Health Worker DBS Dried Blood Spot DHO District Health Office DMO District Medical Office DPT1 First dose of Diphtheria, Pertussis, and Tetanus Vaccine EHT Environmental Health Technicians EID Early Infant Diagnosis EPI Expanded Programme on Immunization GEE Generalized Estimating Equation GNI Gross National Income HAHC - Hospital Affiliated Health Center HIA2 Health Information Aggregation Form 2 HIV Human Immunodeficiency Virus IRB Institutional Review Board ITNs Insecticide-treated bednets LGH Livingstone General Hospital MCDMCH Ministry of Community Development, Mother and Child Health MoH Ministry of Health ODK Open Data Kit OPV Oral Polio Vaccine PCR Polymerase Chain Reaction PMTCT Prevention of Mother-to-Child Transmission PMO Provincial Medical Office iii

5 RCT Randomized Controlled Trial PCV Pneumococcal Conjugate Vaccine PNC Postnatal Care SMAG Safe Motherhood Action Group U-5 Under-five UTH University Teaching Hospital WHO World Health Organization ZCAHRD Zambia Center for Applied Health Research and Development iv

6 EXECUTIVE SUMMARY Background Zambia has made significant progress expanding services for mothers living with HIV. However, gaps still remain in the identification of HIV-exposed and HIV-infected infants. Zambia s Ministry of Health (MoH) guidelines dictate that known HIV-exposed babies should be tested twice for HIV status after birth. Although an estimated 81,000 HIV-positive women gave birth in Zambia in 2012, only 48,000 infants received an HIV test by eight weeks of age in this same year. 1 Thus, improving maternal and early infant diagnosis (EID) testing rates is critical to advancing Zambia s performance along the mother-child HIV testing and treatment cascade. Zambia MoH guidelines state that mothers with unknown HIV status or a previously negative status should continue to be retested every three months until their infant reaches 18 months of age. Yet, these retesting rates are similarly low, estimated at less than 20% for facilities with available data. i For the purposes of this technical report, these postpartum maternal HIV rapid tests are referred to as retests. ii Zambia s MoH and Ministry of Community Development, Mother and Child Health (MCDMCH) requested an evaluation to assess two interventions intended to improve maternal HIV retest and EID dried blood spot (DBS) testing rates: The Simple Intervention 1) provided a guaranteed stock of HIV testing commodities and 2) emphasized current MoH guidelines around EID and HIV-testing for mothers. The Comprehensive Intervention included the components of the Simple Intervention and added 1) hands-on operational optimization to integrate EID testing into routine six-week immunization visits and 2) opt-out HIV testing for all mothers with previously negative or unknown HIV status at the six-week under-five (U-5) visit. The goal of the evaluation was to inform Zambia s HIV testing guidelines and assess the potential for integration of HIV services with other health services. Evaluation Design A three-arm cluster randomized evaluation involving 60 health facilities in Southern Province was used to measure the impact of the two interventions on the number of DBS tests, HIV retests and first doses of diphtheria, pertussis, and tetanus immunizations (DPT1) administered. Quantitative findings were complemented by qualitative inquiries to assess client and health staff perceptions toward HIV testing, U-5 activities, and the two interventions. Results This evaluation revealed that the Simple Intervention significantly improved HIV retesting for mothers but had no detectable effect on DBS testing rates for infants. The Comprehensive Intervention also positively impacted maternal HIV retesting rates but did not have a clear effect on DBS testing rates outside of two outlying facilities (one in the Control group, one in the i These data come from ZCAHRD s 2013 BUPIP programme data and is only for Southern Province. National figures are not available for this outcome. ii This group could contain some mothers that are tested for the first time postnatally, although the wide majority of mothers in this group are being retested after having been tested at ANC i

7 Comprehensive Intervention group). Intensification of HIV testing at the six week U-5 visit via the Simple and Comprehensive Interventions did not negatively impact overall immunization rates. Exit surveys and focus groups revealed that mothers viewed additional HIV testing for themselves and their infants positively with few mothers expressing reservations. Interviews with Comprehensive Intervention facility staff revealed that staff viewed the intervention positively. Specific findings include: No statistically significant effects on DBS uptake were detected for either intervention. The monthly number of DBS tests per facility was 1.01 (P-value = 0.14) tests greater in Simple Intervention facilities versus Control facilities, and 1.54 tests (P-value = 0.12) greater in the Comprehensive Intervention facilities versus Control facilities. However, these results were predominantly driven by two large facilities, and subsequent regressions that diminished the influence of outliers estimated small, non-significant effects for both interventions on DBS uptake. Statistically significant impacts on retests were detected for both interventions. The change in monthly number of retests per facility was tests (P-value < 0.01) greater in Simple Intervention facilities versus Control facilities, and tests (P-value = 0.06) greater in Comprehensive Intervention facilities versus Control facilities. No statistically significant impacts on DPT1 doses were detected for either intervention. The change in monthly number of DPT1 doses per facility was 0.86 (P-value = 0.53) doses smaller in Simple Intervention facilities versus Control facilities, and 0.51 (P-value = 0.78) doses smaller in Simple and Comprehensive Intervention facilities versus Control facilities. Recommendations Based on the findings and observations from this evaluation, the study team offers the following recommendations: This evaluation provides evidence that improvements to Zambia s HIV test kit supply chain can yield important health benefits and should be prioritized. Integrating EID with EPI services can be explored as a feasible strategy for improving HIV maternal retest rates without negatively affecting immunization uptake. However, integration should only be pursued if sustained monitoring and operational support is possible to support implementation. Simple, low-cost steps to improve coordination between EPI and HIV policy counterparts to streamline health services to facilitate integration can be explored. These include: o Updating HIV fields on U-5 cards to match existing guidelines. Currently there is no place to record postpartum retest results or the second DBS test result. o Integrating HIV services into other U-5 data collection tools such as tally sheets and increase size of facility monitoring tools to improve legibility. o Aligning postpartum retesting time periods with the immunization schedule rather than time since last test. Opportunities include: 6 weeks (DPT, OPV, PCV, RV), 9 months (Measles 1), and 18 months (Measles 2). o Joint messaging and sensitization between PMTCT and immunization programming. ii

8 BACKGROUND Significant progress has been made over the past decade in reducing the global burden of HIV/AIDS. Improved access to life-saving treatments and coordinated prevention efforts have contributed to a 29% decrease in AIDS-related mortality since 2005 and a 33% decrease in the number of new HIV infections since Gains have been greatest among children, who primarily acquire HIV through vertical (mother-to-child) transmission, with a 52% decrease in the number of new HIV infections among children between 2001 and Advances in prevention of mother-to-child transmission of HIV (PMTCT) have dramatically shifted the outlook for HIV-infected mothers and their newborns. Currently available prophylactic antiretroviral drugs (ARVs) can reduce mother-to-child transmission to less than 5%. 4 As a result, vertical HIV transmission and HIV-related mortality among pregnant women and children has virtually been eliminated in high-income countries. 4, 5 Despite these gains, children are still vulnerable to HIV, particularly in low- and middle-income countries. Roughly 230,000 of the estimated 260,000 new HIV infections among children each year occur in Sub-Saharan Africa. 2 An integral part of expanding PMTCT is improving early infant diagnosis (EID) of HIV. Early identification of HIV-positive infants is critical to improving chances of survival. It is estimated that over 50% of HIV-positive infants will not survive through their second year. 6 Up to 76% of these deaths can be avoided with early identification and initiation of treatment before 12 weeks of age. 7 However, only 39% of children in low- and middle-income countries have access to HIV testing services within the first two months of life. 5 Zambia is one of 22 countries in the world with the highest estimated numbers of pregnant women living with HIV. Recently, it has made strong progress in expanding PMTCT services, with 94% of women undergoing HIV testing at antenatal care (ANC) and 95% of HIV-positive women receiving antiretroviral therapy (ART) for PMTCT in Unfortunately, these successes have not carried over into the postnatal period. Current MoH guidelines stipulate that HIV-exposed iii infants should be tested twice using the dried blood spot test (DBS) test, first at six weeks after birth and again at six months. With an estimated 80,000-90,000 HIV-positive women giving birth in Zambia each year, an equal number of newborns should receive a DNA polymerase chain reaction (PCR) test within the first two months of life. 8 However, in 2012, only 45,000 infants were tested by eight weeks of age around 50% of the national target. 1 Similarly, MoH guidelines stipulate that mothers with unknown HIV status should be retested every three months while their infants are still breastfeeding (and, therefore, still at risk of transmission). The Zambia Centre for Applied Health Research and Development (ZCAHRD), an affiliate of Boston University in the United States, has worked with the Southern Province Provincial Medical Office (PMO) to test postpartum mothers at six weeks, six months, nine months, 12 months, and 18 months. Despite this support, fewer than 20% of mothers who should be tested actually receive a postpartum retest. iv Many low-income countries have achieved impressive childhood immunization rates. As a result, the World Health Organization (WHO) and UNICEF recommend routine childhood immunization services iii An infant born to a mother living with HIV. iv Calculated using the BUPIP data from January to August 2013; Mothers who should receive tests are defined as those attending immunizations who are not known to be HIV positive. 1

9 to be a platform for delivery of other maternal and child primary healthcare interventions. 9 Studies have demonstrated that services such as provision of vitamin A supplementation, deworming medications, hygiene activities, growth monitoring, and insecticide-treated bed nets (ITNs) can be successfully integrated with immunization services Zambia currently integrates many of these activities at U-5 clinic days rather than running separate immunization days. At U-5, facilities regularly provide immunizations, growth monitoring, and nutrition counseling. Occasionally, they provide supplemental services such as distribution of ITNs, postnatal care (PNC), and some HIV services. The WHO Expanded Programme on Immunizations (EPI) guidelines schedule routine immunizations at birth, six weeks, ten weeks, 14 weeks and nine months. 13 The six week visit aligns with the first recommended EID testing point and the Zambia MoH s postpartum retesting goals. The coverage of first dose Diphtheria, Pertussis, and Tetanus (DPT1) vaccinations in Zambia was estimated to be 83% in , suggesting that the six-week visit could be a particularly useful way of reaching a high percentage of mother-infant pairs. This has prompted Zambian health officials to explore the possibility of utilizing routine U-5 services as an opportunity to boost Zambia s EID and maternal retesting rates. Recent qualitative and observational studies suggest that integrating universal opt-out maternal and infant HIV testing with routine immunizations can be an acceptable and effective approach to improving maternal and infant diagnosis However, there is little rigorous evidence that aims to quantify the impact of integration of HIV and U-5 services on HIV testing and immunization rates. This evaluation examined the impact of two interventions on levels of HIV testing and immunization uptake. The Simple Intervention focused on preventing supply stock outs of HIV testing commodities and reinforcing current MoH guidelines around EID and HIV-testing for mothers. The Comprehensive Intervention coupled the activities in the Simple Intervention with 1) an intentional operational optimization and integration of EID testing into routine six-week U-5 visits, and 2) opt-out rapid HIV testing for all mothers with previously negative or unknown HIV status at the six-week U-5 visit. The specific study aims were to: 2 1. Estimate the impact of both the Simple Intervention and Comprehensive Intervention on monthly averages of the following HIV-related outcomes: a. The total number of DBS tests administered i. The number of first DBS tests administered ii. The number of second DBS tests administered b. The total number of postpartum retests administered for HIV negative or status unknown mothers at all U-5 visits i. The number of postpartum retests administered at the six-week immunization visit 2. Estimate the impact of the Comprehensive Intervention on the monthly average number of DPT1 doses administered compared to the Simple Intervention and Control, and the joint impact of the Simple Intervention and Comprehensive Intervention on the monthly average number of DPT1 doses compared to Control. 3. Examine attitudes and perceptions of mothers and facility staff on HIV-testing and integration of services. This evaluation was launched under Demand Driven Evaluations for Decisions (3DE), an initiative designed to use impact evaluations in a demand-driven and efficient manner to generate evidence for catalyzing at-scale implementation of cost-effective and impactful policies.

10 STUDY DESIGN AND METHODS This evaluation was a cluster randomized controlled trial (RCT) randomized at the health facility level in the Southern Province of Zambia. The Comprehensive Intervention was piloted in assigned facilities from August 12 th to September 30 th, 2013, with full implementation continuing through the intervention period of October 1, 2013 to March 31, Appendix A contains the full evaluation timeline. Study Area This evaluation took place in Livingstone, Monze, and Choma districts in the Southern Province of Zambia. Located in southern Africa, Zambia has a population of more than 13 million people and one of the highest population growth rates in Sub-Saharan Africa. 19 Zambia s gross national income (GNI) per capita is $1,350, with over 60% of its population living below the national poverty line. Zambians life expectancy at birth is 57 years. 20 Zambia s Southern Province is comprised of thirteen districts. v It is home to 1.5 million people with 75% of its population living in rural settings and 48% of its population below the age of In 2007, it was estimated that 14.5% of the women and men age who received an HIV test in Southern Province were found to be positive. 21 The selection of Southern Province was made in conjunction with MoH officials. The evaluation districts of Choma, Livingstone, and Monze were selected based on geographic dispersion, urban / rural characteristics, current HIV prevalence rates, and the absence of conflicting research projects. Study Sample and Randomization The evaluation took place in 60 government-run health facilities providing PMTCT and EID services, with 20 facilities in each evaluation arm. These facilities are supported by ZCAHRD, who provide training for HIV service provision and staff support, including the training and hiring of one PMTCT lay counselor per facility. ZCAHRD also served as a partner for this evaluation, providing guidance in the development of the research protocol, access to baseline data, and facilitating meetings with provincial, district, and facility staff. The final sample was achieved using a stratified and constrained randomization approach (see Appendix B). Statistical power calculations were based on the immunization outcome variable, as this was the outcome where the policy relevant detectable effect size was smallest. A description of power calculations is found in Appendix D. Description of Interventions The evaluation had three study arms: 1. Control facilities continued status quo activities and provided HIV and immunization services as they typically would. 2. The Simple Intervention targeted two potential causes of low EID rates: (1) testing supply stock outs and (2) poor understanding of existing MoH testing requirements and guidelines. The two components of this intervention were: Reinforcement of HIV testing supplies The research team worked with facility staff to ensure that orders for necessary HIV testing supplies were placed on time and in sufficient quantity. In the event of a generalized stock out of HIV testing supplies at the provincial or v Prior to 2013, Southern Province had 10 districts; three districts were added in

11 district level, the evaluation team provided facilities with an additional outside supply. This support was limited to HIV tests and was not extended to vaccines. Reinforcement of current MoH guidelines The research team facilitated one-time visits to health facilities by District Health Office (DHO) officials to meet with relevant health staff at the beginning of the evaluation. The purpose of these meetings was to remind providers of existing MoH HIV testing guidelines and to reinforce that improving EID and postpartum testing is a MoH priority. Additionally, an HIV Activity Sheet created by the evaluation team was introduced for use at all U-5 sessions to help facility staff track postpartum retesting, DBS testing, and distribution of medication. This sheet was designed to centralize HIV services data collection for the facility at U-5 and provide a reminder of what services to offer. 3. The Comprehensive Intervention included all components in the Simple Intervention, but additionally operationalized the integration of universal, opt-out HIV screening and testing for mothers and infants with in-facility (static) U-5 services. These components were not included in U-5 services that are delivered outside of the facility in the community (outreach services). The unique components of this intervention were: Communication of the intervention: Health facility staff were instructed to communicate all aspects of the Comprehensive Intervention during ANC appointments, in-facility deliveries, and PNC visits. Additionally, at six week immunization visits, mothers and caregivers received group counseling on opt-out HIV screening service and the importance of regular HIV screening for mother and child health. Finally, the research team engaged Safe Motherhood Action Groups (SMAGs), community health workers (CHWs), and active neighborhood health committee members to further increase awareness in facility catchment areas. These community members completed low-touch community sensitization over changes that would be made to U-5 services. Operations optimization and integration of testing services: This component optimized the patient flow during U-5 services to include new testing procedures and to more closely integrate EID and EPI services, while mitigating increases in staff workload and patient waiting times (Figure 1). The evaluation team worked with facility staff to identify efficient allocations of staff and tailor the order of services. A key component of the operational optimization was a new patient triaging approach that sorted patient U-5 cards into bins and used separate queues for three types of patients: 1) first visit infants (six weeks), 2) second visit or later infants who required immunizations and 3) infants who were scheduled to only receive growth monitoring. The six week visit was emphasized as a key patient touch point, as this is when both the first round of childhood immunizations (DPT, oral polio vaccine (OPV), pneumococcal conjugate vaccine (PCV), and rotavirus vaccine) and first maternal postpartum retest should take place. Additionally, DBS testing at six weeks allows facility staff to use the ten week or fourteen week immunization visit to deliver DBS test results. Implementation of universal, opt-out screening of HIV for both mothers and infants: Facility staff were instructed to examine the maternal HIV status on the U-5 or ANC card for all infants attending their first U-5 visit and do the following: o If marked Confirmed Exposed, the health care worker conducted the DBS test on the infant only. o If marked, Mother Status Unknown or Confirmed Not Exposed the mother or caregiver was asked if the mother had ever tested HIV-positive. If yes, a DBS was done on the infant only. If no, the mother was offered an HIV antibody test in an opt-out manner. 4

12 Figure 1: Overview of Comprehensive Integration Approach In all Simple and Comprehensive Intervention facilities, normal antibody testing procedures were followed using the Determine TM Antibody Tests and Unigold TM confirmatory test along with all standard counseling messaging per existing practice. Infants of mothers who tested HIV-positive should have then received a DNA PCR test the same day. These procedures reinforce standard HIV testing policies set by the Zambian MoH and WHO. Data Collection Pre-Evaluation Activities All facilities included in the evaluation were provided with larger U-5 tally sheets to improve data quality. vi A short training was held with representatives from all facilities to introduce the research team, to explain what data would be collected, and, for all intervention facilities, to deliver the guidelines reinforcement portion of the evaluation. The DHO was represented at each meeting to complete the guidelines reinforcement. Separate meetings were held for each intervention group in each district. All Comprehensive Intervention facilities received an initial onsite workshop to introduce the components of the intervention and do a full walk-through of the new patient flow procedures. The intervention was piloted for six weeks between August 12th and September 30 th, During this period, each Comprehensive Intervention facility was visited at least twice with additional supervisory visits as needed to troubleshoot difficulties before the start of data collection. vi Previous tally sheets had extremely small printing, making them difficult to use. All data collection fields on these forms and layout remained the same. 5

13 Monthly Facility Visits and Administrative Data Sets Research team members conducted monthly visits to all 60 sample facilities to collect data from facility registers and evaluation team-supplied data sheets ( HIV Activity Sheets and larger U-5 tally sheets). All data were either used for data analysis or for validation of the main source of data. Since facility register data could not leave the facility, data were collected by evaluation staff using Open Data Kit (ODK) surveys programmed on Android mobile phones. The data from evaluation teamsupplied sheets were collected and transported to the IDinsight office for data entry and secure storage. During these monthly data collection visits, the evaluation team also briefly interviewed facility staff to discuss any staffing changes and to take stock of HIV testing and immunization supplies. Evaluation staff also collected facility-level data from several additional administrative data sources, including historical data on immunizations and ANC data from DHOs and the MoCDMCH, DNA PCR laboratories for DBS at both University Teaching Hospital (UTH) in Lusaka and Livingstone General Hospital (LGH) in Livingstone, and baseline HIV testing records from ZCAHRD. Exit Interviews, Focus Groups, and Structured Interviews Quantitative and qualitative methods were used to assess and to compare perceptions of HIV testing as well as of the interventions among mothers and facility staff. Exit surveys were conducted with mothers and other caregivers attending under-five services at 51 out of 60 facilities using convenience sampling. vii Evaluation staff identified the first visit mothers and other caregivers and invited them to complete the exit interview. While facility staff helped direct mothers and other caregivers to the evaluation staff after delivery of U-5 services, facility staff members were not informed on the specific topic of the survey, and interviews were not conducted in front of facility staff. All mothers and other caregivers who were invited to participate in the exit survey were informed that the survey was separate from the services that they received. Informed consent procedures were followed with each participant. Mothers and other caregivers were asked about U- 5 services, as well as any potential impact of prior knowledge of HIV testing services on the likelihood of attending U-5 services. Focus groups were conducted in catchment area communities of eight study facilities in the Simple or Comprehensive Intervention arms to better understand why mothers do or do not attend U-5 services and their perceptions of the changes at U-5 clinic regarding HIV testing. These facilities were purposively sampled across intervention arms to achieve a mix of urban and rural settings and large and small facilities. In each location, CHWs identified women who had successfully delivered a baby between September 1 st, 2013 and February 1 st, 2014, since these babies would have been eligible for six week vaccination within the intervention period. Mothers who lived in an area that provided both static and outreach U-5 services were divided into groups for the type of U-5 sessions they attended. viii All focus groups were conducted in the local language of Tonga. vii The nine facilities that were not included for exit interviews typically only had U-5 once per month and had fewer mothers per U-5 session. Some of these facilities were visited during a scheduled U-5 session, but there were no first visit mothers to be interviewed. viii Focus group discussions were also planned to include groups of women who had not attended U-5 services at all. However, these women could not be identified in the communities. Evaluation staff took the following steps to try to identify them: 1) CHWs were asked to identify anyone within their communities who had not attended U-5 services, including inquiring with other community leaders and groups; 2) Evaluation staff asked mothers in each focus group if they knew of other women in the community who had not attended U-5 services. 6

14 Finally, structured interviews were conducted with health facility staff from ten Simple and ten Comprehensive Intervention facilities during their final data collection visit to gain more insight into staff perspectives of the intervention, including perceived benefits and challenges. Ethics This evaluation was approved by the ERES Converge Ethics Review Board in Lusaka, Zambia and the Boston University Institutional Review Board (IRB) in Boston, Massachusetts, United States. Additionally, the evaluation received formal approvals from Zambia s MoH, MoCDMCH, and the District Health Offices of Choma, Livingstone, and Monze. Data Collection, Quality, and Monitoring The primary data sources used in this evaluation were administrative and generated by non-research staff members who worked at health facilities, districts, and hospitals in low resource settings as part of normal day-to-day operations (Table 1). To mitigate potential inaccuracies in these data sources, the evaluation team cross-verified data whenever possible. Table 1. Summary of Data Sources Used for Analysis and Quality Control Outcome Primary Data Quality Control Sources Source DBS Testing UTH and LGH PCR Lab Databases DBS Tracking Register DBS Lab Requisition Book Interviews at clinic of mothers of infants receiving 2 nd / 3 rd series of immunizations Interviews of mothers in communities Postpartum Retesting BUPIP Monthly Report HIV Activity Sheets (Simple and Comprehensive Intervention facilities) VCT / General Counseling Register Immunizations Under-5 Tally Sheets HIA2 Monthly Report to Districts Interviews at clinic of mothers of infants receiving 2 nd / 3 rd series of immunizations Interviews of mothers in communities The three primary outcomes included in the analysis were 1) The average monthly number of DBS tests administered per facility per month; 2) the average monthly number of maternal postpartum retests administered per facility per month; and 3) the average monthly number of DPT1 doses administered per facility per month. These outcomes were measured as follows: Average Monthly Number of DBS Tests This outcome was measured using the number of DBS tests administered for each facility for each month from January 2012 through March The primary data source for this outcome was the DBS lab database that received the samples collected by each facility. A DBS test was defined as administered if the sample arrived at the lab, regardless of the result of the test. Therefore, samples with indeterminate results were included, but samples that were taken but did not make it to the lab were not. These data were compared to DBS test figures from the facility. At the facility, the numbers of first DBS tests and second DBS tests were collected from the DBS tracking register and the total 7

15 aggregate number of DBS tests was collected from the lab requisition register. All three data sources were highly correlated with an average difference of less than 5%. Additionally, DBS records from HIV-exposed infants were checked between U-5 cards inspected during data verification interviews and the DBS tracking register at the facility. Out of 325 verification interviews, 30 U-5 cards indicated that a DBS test had been completed. Of these 30 tests, 87% (26 tests) were verified by the research team with three of the four unverified tests coming from a single facility. Average Monthly Number of Maternal Retests The average monthly number of maternal retests conducted was measured using data from the Boston University PMTCT Integration Programme (BUPIP) monthly report. This outcome included all retests that were recorded, regardless of the test result. These data were verified with the evaluation HIV activity sheets. Verification was only possible for intervention facilities, as HIV activity sheets were not introduced in control facilities. Overall, the two sources had a correlation of 0.75 for the total number of tests done per month per facility. The average total number of tests per month per facility in intervention sources was very similar with the BU data source averaging 26.0 tests per month and the HIV activity sheet data source averaging 24.4 tests per month. ix Average Monthly Number of DPT1 Doses This outcome was measured using monthly figures of the number of DPT1 doses for each month from January 2012 to March 2014 that the facility reports to the DHO each month. These estimates included both outreach and static figures, since the facility does not disaggregate these. Figures were compared to the number of DPT1 doses that evaluation enumerators collected from each facility by counting the number of DPT1 immunizations recorded on each U-5 tally sheet for a given month. The research team compared these two figures and recollected tally sheet data if the two data sources differed by 10% or more. This process occurred in approximately 52% of cases. The research team then reconciled the resurvey data using pre-established data cleaning rules. DPT1 data were also verified using a data verification interview conducted with mothers or other caregivers who brought their infant to the first under-5 visit during the evaluation period. Of the 341 interviews conducted, 77% were conducted at facilities with mothers or other caregivers who had returned for 2 nd and 3 rd immunizations with their infants, while 23% were conducted in communities with mothers who were identified by CHWs. During each interview, the enumerator recorded the date of the DPT1 immunization, infant HIV status, and date of DBS test, if applicable, from the infant s U-5 card. These data collected were then compared to facility registers to check if the record existed at the facility and if the dates matched. Overall, DPT1 data for 3% of mothers (325 out of 10,435) were double-checked through data verification interviews, achieving 88% agreement. x ix There are several reasons for possible discrepancies. First, the activity sheets were a newly introduced tool and were inconsistently filled out during the beginning of the evaluation. Second, postpartum retests done at times other than U-5, although likely uncommon, may not have been recorded on HIV activity sheets. Overall, the BUPIP source was favored for its breadth in including all facilities (including control) as well as being the source of the baseline data. x There were direct matches for 62% of the data (202 out of 325 cases). Additionally, 26% of the data contained matched dates between the U-5 cards at the U-5 tally sheets but did not have a matching U-5 number. Often there are cases where the U-5 number is recorded as a dash on the tally sheet for first visits if the card number has not yet been assigned (as can be the case during first immunization visits). 8

16 Covariates Facility-level characteristics were primarily collected during the baseline period. A facility was characterized as rural if it was more than ten kilometers along the road away from the DHO, based on administrative reports. The average baseline number of mothers attending ANC was collected from monthly reports that facilities provide to the DHO. Additionally, the number of mothers appearing for 1 st ANC visit each month was also collected from these reports. The number of HIV positive mothers anticipated at U-5 was calculated by multiplying the average baseline number of mothers attending first ANC visit by the % of women attending ANC that are living with HIV, as supplied by the BUPIP database. Finally, the actual distance between the facility and the DHO was measured using GPS coordinates. STATISTICAL METHODS All major analyses were conducted using facility-level comparisons between Control, Simple Intervention, and Comprehensive Intervention facilities. Exploratory Analysis The main outcomes and covariates were examined visually to understand data structures, distributions, and trends over time. Line graphs of the three main outcomes (number of DBS tests, number of maternal retests, and number of DPT1 doses) over time were constructed using three month running averages for smoothing. Additionally, facility-specific baseline and intervention period data were compared for all outcome variables to examine trends and to identify potential outliers. Regression Analysis Data for the main outcomes consisted of repeated monthly measures for each facility from both the baseline and the intervention period. Multivariate linear regression models accounting for clustering at the facility level were used. Standard errors were estimated using bootstrapping methods. The isolated effect of each intervention on each of the primary outcomes was estimated by including interaction terms between a factor variable for the intervention groups and the time period (baseline versus intervention period). Additional covariates were included in the regression models to control for confounders and to increase precision. Covariates were chosen using Akaike Information Criterion (AIC). The regression model for the average number of DBS tests administered also adjusted for district and urban/rural characteristics since these are the variables on which the initial sample was stratified, as well as time (using splines to account for linear trends), a binary indicator for a period of a national stockout, the number of DPT1 doses provided in the same month, and the number of mothers living with HIV anticipated in that month. Because the DBS testing results were particularly affected by outliers, the primary regression for this outcome used a log transformation of the outcome. The regression model for the average number of maternal retests administered adjusted for district and urban/rural characteristics, time, the monthly average number of mothers going to first ANC, the distance between the facility and the DHO, and the number of DPT1 doses provided in the same month. Finally, the regression model for the average number of DPT1 doses provided each month adjusted for the average number of mothers going to first ANC, time, and the average distance from the DHO. Since the Comprehensive intervention alone was operationally integrated with immunization services, the analysis first assessed if the Comprehensive Intervention resulted in a difference in the 9

17 change between baseline and the intervention period compared to the combined change experienced in both the Control and Simple Intervention facilities. It then assessed if the combined intervention arms experienced a statistically significantly different change than the Control facilities. Since these data were collected as repeated measures from the same facility and the outcomes were measured as count data, we conducted the same regression analyses on the outcomes using generalized estimating equations (GEEs) with a negative binomial family, log link function, and exchangeable working correlation matrix as a sensitivity analysis. The multi-level approach that this model uses accounts for the high correlation between outcome measures from the same facility. The negative binomial distribution was used to account for the non-normal distribution of the data and the fact that the value of the outcome cannot be negative. GEE results are presented in appendices. Secondary Analysis Secondary analyses were conducted for DBS outcomes at the patient level, rather than the facility level, to test for additional evidence of intervention effects. We used linear regression to test for differences in the age of an infant at their first test between intervention arms. We also created several graphs that demonstrate bottlenecks across intervention arms, in the delivery of samples to labs and the processing once they arrive. Exit Interviews and Qualitative Analysis Exit interview data used Pearson s chi-squared test to test for differences in proportions across the three evaluation arms. Facility staff interviews and focus group discussions were analyzed using a thematic analysis approach. Each focus group discussion was coded according to common themes by two separate evaluation staff members. xi All statistical analyses were done using Stata version 12 (Stata Corp LP, College Station, Texas, US). RESULTS Using baseline data that were directly collected from the facilities, the monthly averages of DBS tests, DPT1 doses administered, first ANC visits, six week retests, and total retests were similar across the evaluation groups. xi Inter-rater agreement was calculated using the kappa statistic with an expected percent agreement of 25%. The overall kappa was 0.63, meaning there was 63% agreement beyond what is expected by chance alone. The kappa for themes that were specifically related to HIV testing and the integration of services was The majority of disagreement was due to the fact that one coder did not include as much as information as the other, as opposed to the two coders reaching dissonant conclusions. Therefore, by using both sets of codes for analysis, a fuller representation of the focus group discussions was achieved. 10

18 Table 2. Summary of Sample by Intervention Arm Control (N=20) Simple (N=20) Comprehensive (N=20) N (%) N (%) N (%) Total N Facility Characteristics No ART 14 (70%) 15 (75%) 15 (75%) 44 Static ART Site 2 (10%) 3 (15%) 4 (20%) 9 Mobile ART Site 4 (20%) 2 (10%) 1 (5%) 7 Strata Choma Rural 8 (40%) 8 (40%) 8 (40%) 24 Choma Urban 2 (10%) 2 (10%) 2 (10%) 6 Livingstone Urban 3 (15%) 3 (15%) 3 (15%) 9 Monze Rural 6 (30%) 6 (30%) 6 (30%) 18 Monze Urban 1 (5%) 1 (5%) 1 (5%) 3 Mean (SD) Mean (SD) Mean (SD) P-value * Baseline Monthly Averages DBS tests 3.97 (7.05) 3.74 (6.98) 4.30 (8.08) 0.89 DPT1 doses (23.43) (17.42) (26.75) 0.99 First ANC Visits (25.04) (21.44) 29.9 (21.44) Week Retests 3.29 (3.73) 1.18 (1.81) 3.04 (5.31) 0.14 Total Retests (15.64) 6.74 (9.38) (27.16) 0.19 * The intervention arms were regressed as indicator variables on each outcome. After the regression, an F-test was used to test for equality between the three evaluation arms. At the time of sampling, all samples with p-values of less than.90 for DBS, DPT, and ANC averages were removed from consideration. Since that time, more complete data is available, explaining the 0.89 p-value for the DBS tests at the final sample. Infant DBS Testing The change in the average number of infant DBS tests administered per month from baseline to the intervention period was examined and compared between evaluation arms. The baseline period for the DBS analysis started January 1 st, 2012 (month = 0) and ended July 31 st, 2013 (month = 19). During this period, a total of 4,529 DBS samples were collected. Data from August and September 2013 were excluded from the analysis as this was when the interventions were being piloted. Differences between Baseline and Intervention Periods The average number of tests in the study sample at baseline was 4.0 tests per month. The average number of DBS tests administered per month per facility increased by 0.43 tests over baseline in Simple Intervention facilities and by 1.19 tests over baseline in Comprehensive Intervention facilities. The average number of DBS tests administered per month decreased by 0.49 tests in Control facilities (Table 3). While the Control group s monthly DBS testing numbers decreased between the baseline and the intervention period (-0.49), the Simple Intervention group s monthly testing numbers increased between baseline and the intervention period (0.43), with a total difference of 0.92 more tests per month over time as compared to the Control group. Similarly, the difference in the average monthly DBS testing numbers between the baseline and the intervention periods among facilities in the Comprehensive Intervention group was 1.78 tests greater than the difference experienced among facilities in the Control group. 11

19 Table 3. Average Number of DBS Tests Administered per Month per Facility, by Group Control Simple Comprehensive N=20 N=20 N=20 Mean (SD) Mean (SD) Mean (SD) Baseline Average 3.96 (5.31) 3.75 (6.19) 4.30 (6.51) Intervention Period Average 3.48 (2.75) 4.18 (6.18) 5.49 (8.27) Difference (3.44) 0.43 (2.14) 1.19 (3.69) The average number of DBS tests during baseline was plotted against the average number of DBS tests during the intervention period for each facility, with the size of the circle weighted by the expected number of HIV-positive mothers to appear for U-5 visits (Figure ). Figure suggests that a few large Comprehensive Intervention and Control facilities are driving a large proportion of the overall changes observed in the aggregate baseline and intervention periods. Figure 2. Baseline versus Intervention Period Monthly Averages of DBS Tests for All Facilities Regression Results The linear regression using a log-transformed outcome of the average monthly number of DBS tests revealed that Simple Intervention resulted in a 17% [90% CI: -7%, 46%, P-value = 0.26] greater change in average number of monthly DBS tests over time as compared to the change experienced in Control facilities (Table 4). The Comprehensive Intervention experienced a 10% [90% CI: -10%, 12

20 36%, P-value = 0.43] greater change in the average monthly number of DBS tests between baseline and the intervention period as compared to the change experienced in Control facilities. Neither of these differences was statistically significant. Table 4. Linear Regression Results of Intervention Arm on Logged Outcome (Number of DBS Tests per Facility per Month) Covariates e B P-value [90% CI] Intervention Arm Control Ref Simple [0.72,1.08] Comprehensive [0.83,1.21] Time Period Baseline Ref Intervention period [0.59,1.00] Intervention Impacts: Simple v Control [0.93,1.46] Comprehensive v Control [0.90,1.36] # of Monthly DPT1 Immunizations [1.00,1.01] District & Urban/Rural Stratum Urban Choma Ref Rural Choma [0.69,1.49] Urban Livingstone [1.12,3.07] Urban Monze [0.88,3.17] Rural Monze [0.97,2.08] Time [1.01,1.12] Time [0.90,1.01] Time [0.89,1.03] Time [0.99,1.29] Time [0.98,1.07] Time (Intervention Pd) [0.97,1.05] Known National Stock out of DBS Kits [0.79,1.10] Number of HIV+ Mothers Anticipated < 0.01 [1.06,1.19] Constant [0.73,1.64] Additional linear regressions using a log transformation were run to assess intervention impacts on first DBS tests and second DBS tests. No significant intervention effects were detected results are presented in Appendix E. Two additional post hoc linear regressions without a log transformation were run (Table 5). One regression included all of the facilities, and the other excluded two large outlier facilities (one Comprehensive Intervention facility and one Control facility). The regression that included all of the facilities estimated that the Simple Intervention resulted in an additional 1.01 tests per facility per month (90% CI: -0.11, 2.13, P-value = 0.14) over the change experienced in the Control group, and the Comprehensive Intervention resulted in an additional 1.54 tests (90% CI: -0.08, 3.16, P-value = 0.12) over the change experienced in the Control group. The regression that excluded the outliers 13

21 estimated a much more modest impact for both interventions, indicating that results seen in the aggregate difference-in-difference measures are largely driven by the two outlier facilities. Table 5: Multivariate Linear Regression Results With and Without Outliers for Number of DBS Tests per Facility per Month Covariates Full Model Excludes Outliers Coeff P-value [90% CI] Coeff P-value [90% CI] Intervention Arm Control Ref Ref Simple [-1.19,0.88] [-1.11,0.29] Comprehensive [-0.38,1.80] [0.10,1.88] Time Period Baseline Ref Ref Intervention period [-2.89,-0.12] [-2.95,0.18] Intervention Impacts: Simple v Control [-0.11,2.13] [-0.72,1.33] Comprehensive v Control [-0.08,3.16] [-0.49,1.01] # of Monthly DPT1 Immunizations < 0.01 [0.02,0.06] < 0.01 [0.01,0.05] District & Urban/Rural Stratum Urban Choma Ref Ref Rural Choma [-0.72,2.21] [-1.50,0.87] Urban Livingstone [0.63,5.14] [-1.06,2.37] Urban Monze [1.00,9.10] [-1.81,4.74] Rural Monze [0.44,3.11] [-0.39,1.78] Time [-0.01,0.45] [-0.10,0.39] Time [-0.36,0.21] [-0.39,0.28] Time [-0.86,0.10] [-0.41,0.17] Time [-0.02,1.78] [-0.24,0.84] Time [-0.46,0.27] [-0.24,0.35] Time (Intervention Pd) [-0.13,0.24] [0.05,0.25] Known National Stock out of DBS Kits [-1.53,0.72] [-1.82,-0.04] Number of HIV+ Mothers Anticipated < 0.01 [0.81,1.32] < 0.01 [1.09,1.66] Constant [-4.61,-0.76] [-3.03,-0.03] Finally, a GEE model is presented in Appendix E. The GEE results are similar to those obtained from the linear regression with the log transformation. Time Trends A line graph using a three month running average of the number of DBS test kits administered reflects the trends over the baseline and the intervention periods (Figure 3). There is an observable drop in DBS testing between time = 9 (September 2012) and time = 13 (January 2013) as the result of a national shortage of DBS test kits. This temporary drop is controlled for in the analysis. As the regression results also demonstrate, the Simple and Comprehensive Intervention arms experienced an increase in the number of DBS tests administered during the intervention period, though the trend observed in the Comprehensive Intervention line was largely a result of one facility. Similarly, the decrease that is seen in the Control arm is also heavily influenced by one facility. 14

22 Figure 3: Changes in Average # of DBS Tests over Time, by Intervention Arm Maternal HIV Retests This section examines changes in postpartum retests for mothers at the first immunization visit (six weeks) as well as the total number of retests. The six-week test is emphasized because it is a high volume touch point between new mothers and health facilities early in an infant s life. The total number of retests gives a more holistic view of the volume of HIV services being delivered at U-5. The baseline period for the maternal HIV retest analysis was from January 1, 2013 (month = 0) to July 31 st, 2013 (month = 7) xiii. The analysis excluded data from the training and piloting period of September and October In total there were 8,712 postpartum retests done in the evaluation period and 5,055 done at baseline. Differences between Baseline and Intervention Periods In all arms, the average number of six week maternal retests and total maternal retests increased from the baseline period to the intervention period (Figure 4), with the increase greater in the intervention arms than in the control arm. The average number of six week retests increased from 1.2 to 8.9 in Simple Intervention facilities, from 3.0 to 12.0 in the Comprehensive Intervention Facilities, and from 3.3 to 6.5 in Control Facilities. Similarly, the total number of retests per month nearly quadrupled in Simple Intervention facilities and nearly doubled in the Comprehensive Intervention facilities, while the total number of retests only increased from 14.2 to 17.7 in Control facilities. xiii Data before January 1, 2013 was not available. 15

23 Figure 4. The Average Number of Maternal Retests During Baseline and Intervention Period per Facility Baseline Intervention Period wk Total 6 wk Total 6 wk Total Control Simple Comprehensive Figure 5 shows that almost all treatment facilities experienced gains over time while the changes for Control facilities are mixed. This graph excludes one large facility. It does not appear that outliers are driving the observed trends in retests. Figure 5. Baseline versus Intervention Period Monthly Averages of Total Postpartum Retests for All Facilities 16

24 Regression Results Two multivariate linear regressions were run to assess the impact of the interventions on six week retests and on total number of retests (Table 6). These results confirm that both the Simple and Comprehensive Interventions resulted in an increased number of postpartum retests. The impact of the Simple Intervention on six week tests is estimated to be a difference between baseline and the intervention period that is 4.60 tests (90% CI: 2.19, 7.01, P-value< 0.01) higher than that experience in the Control group. The impact of the Comprehensive Intervention on six week tests is estimated to be a difference that is 5.76 tests (90% CI: 2.63, 8.91, P-value <0.01) higher than that experienced in the Control group. The impacts of both interventions on the total number of retests are similar. The impact of the Simple Intervention on total retests is estimated to be a difference that is (90% CI: 7.12, 24.41, P-value < 0.01) tests greater than the difference experienced in the Control group. The impact of the Comprehensive Intervention on total retests resulted in a difference that is (90% CI: 1.52, 20.33, P-value = 0.06) tests greater than the difference experienced among Control facilities. These effects are similar in a GEE regression with the same parameters (see Appendix F). Table 6. Multivariate Linear Regression Results for 6 Week Retests and Total Number of Retests (6 wk, 6 mo, 9 mo, 12 mo, 18 mo) Covariates Coeff # of 6 Wk Retests # of Total Retests P- value [90% CI] Coeff P- value [90% CI] Intervention Arm Control Ref Ref Simple [-3.77,-0.67] [-15.33,0.35] Comprehensive [-1.92,1.91] [-8.64,11.75] Time Period Baseline Ref Ref Intervention period < 0.01 [3.05,8.89] [-6.36,9.69] Intervention Impacts: Simple v Control < 0.01 [2.19,7.01] < 0.01 [7.12,24.41] Comprehensive v Control < 0.01 [2.63,8.91] [1.52,20.33] # of Monthly DPT1 Immunizations [0.01,0.08] [0.04,0.36] District & Urban/Rural Stratum Urban Choma Ref Ref Rural Choma [-2.38,3.41] [-10.85,13.80] Urban Livingstone [-6.55,-1.68] [-16.62,3.94] Urban Monze [-9.36,11.51] [-30.33,69.10] Rural Monze [-2.15,2.83] [-6.80,11.05] Time < [-0.10,0.63] [-1.54,2.05] Time [-0.60,0.29] [-0.20,2.26] Time (Intervention Pd) [-1.14,-0.16] [-1.91,0.47] Number of 1st ANC Visits [0.04,0.16] [0.09,0.56] Distance to DHO [-0.10,0.01] [-0.24,0.09] Constant [-9.16,2.36] [-44.47,33.70] 17

25 Time Trends A line graph of the average total number of maternal retests per month suggests that both intervention arms experienced strong initial gains that tapered off over time (Figure 6). Figure 6. Average Total Maternal Retests per Month, by Treatment Group A linear regression was used to test for any time trends in the impact of the intervention (Appendix F). The regression found that the estimated impact of the Simple Intervention on all retests decreased from nearly 25 additional tests per month in the first three months to an additional 10 tests per month increase in the second half of the intervention period. With an average of 6.7 retests done per month at baseline, these differences represent large changes. The drop is similar in the Comprehensive Intervention arm for the total number of tests. The effect on the number of six week tests is more stable in the Comprehensive Intervention arm, which averaged approximately six additional tests per facility per month (a 200% increase over baseline) throughout the intervention period. Out of 8,712 retests completed during the intervention period, 94 were positive for an average positive rate of 1.1% (90% Binomial CI: 0.87%, 1.3%). xiv There were no material differences in the positive rate between the baseline and intervention periods. Immunization While the goal of the two interventions was to increase maternal and infant HIV testing rates, it was also important to assess if they affected immunization rates. The baseline period for this analysis ran from January 1, 2012 to September 30, The months between August and November 2012 xiv Binomial confidence intervals used as the positive rate is bounded at 0. Data source: BUPIP Monthly Facility Reports 18

26 when all facilities experienced a sharp drop in immunization numbers - were excluded from the analysis. A total of 10,435 doses of DPT1 were administered across the 60 evaluation facilities during the intervention period between October 1, 2013 and March 31, Differences between Baseline and Intervention Period The average facility-based difference between the baseline and intervention periods in the number of monthly DPT1 doses administered is 1.9 (SD 6.0) in the Control arm, compared to 0.9 (SD 4.1) and 2.0 (SD 5.6) in the Simple and Comprehensive Intervention arms (Table 7). Table 7. Average Number of DPT1 Doses Administered per Month per Facility Control Simple Comprehensive Mean SD Mean SD Mean SD Baseline Average (23.43) (17.42) (26.75) Intervention Period Average (23.25) (18.00) (29.05) Difference 1.93 (5.97) 0.86 (4.08) 2.01 (5.56) Regression Results A multivariate linear regression confirms that the Comprehensive Intervention did not have a significant effect on DPT1 doses (Table 8). The regression estimated that the average change in number of DPT1 doses provided per month was approximately 0.86 doses higher [90% CI: -1.40, 3.12] in Comprehensive Intervention facilities compared to the combined average change in the Simple Intervention and Control facilities. This difference was not statistically significant (p-value 0.53), indicating that the intervention did not impact immunization rates overall. Table 8. Multivariate Linear Regression Results for the Montly Average Number of DPT1 Doses Comparing Comprehensive Group to Combined Control & Simple Groups 19 Covariates Coeff P-value [90% CI] Intervention Arm Control & Simple Ref Comprehensive [-3.62,1.89] Time Period Baseline Ref Endline [0.02,2.52] Intervention Impacts: Comprehensive v Control & Simple [-1.40,3.12] District & Urban/Rural Stratum Urban Choma Ref Rural Choma [-5.97,4.91] Urban Livingstone [-5.16,1.82] Urban Monze [-13.30,13.93] Rural Monze [-3.55,4.41] Baseline Avg # of ANC Visits < 0.01 [0.79,1.01] Distance from DHO [-0.08,0.10] Constant [-4.15,4.59]

27 A similar multivariate linear regression was run to estimate the combined impact of the Comprehensive and Simple Interventions versus Control on the monthly number of DPT1 doses administered. No significant differences were detected (Table 9). Table 9. Intervention Group Comparisons for Difference in Change in Number of DPT1 Doses between Baseline and Intervention Period Using Linear Regression Models* Control Simple Simple Comprehensive Simple + Comprehensive [90% CI: -3.66, 1.16] P-value = [90% CI: -2.22, 2.67] P-value = [90% CI: -3.59, 2.56] P-value = [90% CI: -0.75, 3.71] P-value = 0.28 *Models were adjusted for average distance from the district health office (DHO), the average number of ANC visits, and a variable that combined both district and urban/rural characteristics. Comparisons demonstrate the row category versus the column category. Finally, a GEE regression was run adjusting for the same covariates to obtain the incidence rate ratios for the Comprehensive intervention compared to the Simple Intervention and control facilities None of these comparisons were statistically significant. This regression model is included in Appendix G. Time Trends A line graph using running three-month averages for smoothing was created to examine DPT1 rates across the three intervention groups (Figure 7). Trends were similar across time, including during the intervention period. Figure 7. Average DPT1 Doses per Month, by Treatment Group 20

28 Secondary Analyses Age at First DBS Test For DBS outcomes, individual data was available from the central DBS lab that contained dates on each stage of the DBS process, including: date of birth, date DBS sample was collected, date DBS sample arrived at lab, and date test result was entered in database (and returned to facility). The average age at which infants received their first test was examined to see if the intervention resulted in earlier testing of infants (Figure 8). Over time children were being tested earlier in all facilities, with the average age for a first DBS test dropping from four months to three months of age between January 2012 and January No significant differences (regression not shown) between the study arms were found. Figure 8. Average Age of Infants at First Test in Months, by Intervention Arm Time for DBS Sample Processing There were several challenges, independent of the interventions, with the courier service between the health facilities and central labs throughout the intervention period. These challenges often led to long delays meaning that on average it took several months for DBS samples to be processed by the central labs after they were collected from an infant. Figure 9 depicts the average time elapsed between DBS sample collection and sample processing by the central labs. 21

29 Figure 9. Time from DBS Sample Collection to Lab Processing Test kit supplies Throughout the intervention period, there were many instances undersupplied facilities, particularly for DBS test kits. Table 11 below shows the number of restocking visits made to facilities by commodity by intervention arm throughout the study. It is of note that the two large outliers in the DBS analysis are both in Choma district which saw the biggest challenges in terms of DBS stock. The large Control facility that saw a major drop in tests was stocked out of DBS for at least two months. Conversely, the large facility across town receiving the Comprehensive Intervention was resupplied with DBS four times. Table 11. Restock Instances per Commodity per Month by Intervention Arm Comprehensive Facilities Simple Facilities Determine Unigold DBS Determine Unigold DBS Total Sep Oct Nov Dec Jan Feb Mar Total

30 In both intervention arms, but particularly the Comprehensive Intervention group, facilities were more likely to run out of DBS than either rapid test commodity. It also appears that supply chain problems were more acute at the beginning of the intervention period before tapering off over time. Anecdotal evidence from district pharmacies suggested continued undersupply of DBS at the district level which led to local stock outs. Exit Surveys There were 426 exit surveys administered with mothers or other caregivers attending U-5 services at 51 facilities. Generally, mothers expected to receive similar services across the three arms (Figure 10). Mothers attending Comprehensive Intervention facilities had a statistically significantly higher expectation of receiving HIV services. However, only 14.8% (n=27) of mothers attending Comprehensive Intervention facilities were expecting HIV services, suggesting that efforts to publicize the Comprehensive Intervention in these catchment areas may not have reached the majority of mothers. Figure 10. Services Expected at Six-Week U-5 Clinic The percentages of mothers who reported receiving HIV services differed across study arms (Figure 11). In Control facilities, 11% of mothers reported receiving HIV services, compared to 46% in the Simple Intervention facilities and 45% in the Comprehensive Intervention facilities. 23

31 % of Mothers Receiving Service Figure 11. Services Received at Six Week U-5 Services ** % ** *** * Control Simple Comprehensive P-values were calculated using Pearson s chi-squared test: ***p-value < 0.01; ** p-value < 0.05; * p-value < 0.1 ^Other services expected included family planning services and provision of ITNs % Five out of the six mothers who reported not receiving immunizations were from the same facility at a time when there was a stockout of needles In addition to being asked open ended questions about what services they received, mothers were specifically asked if they had been offered an HIV test. The percentage of mothers who were offered an HIV test was statistically significantly different across intervention versus control arms with at least 60% of mothers in either the Simple Intervention or Comprehensive Intervention arms reporting being offered an HIV test compared to 28% in the Control arm (Table 10). This figure is higher than the percentages of women who reported receiving HIV services for the open-ended question about services received. This is unlikely a result of women being offered, but declining testing, since facilities anecdotally reported few to no cases of refused tests. Possible explanations may be that mothers may be reluctant to report receiving HIV services or that recall may be imperfect when mothers are asked open -ended question about services received. Of those who were offered an HIV test, prior knowledge that they would be offered a test was similar across treatment arms. Likelihood to attend the U-5 services conditional upon prior knowledge that an HIV test was to be given was not statistically significantly different across evaluation arms in either the actual scenario (among women who reported knowing that they knew they would be offered a test) or the hypothetical scenario (among women who reported not knowing that they would be offered a test). More women reported being more likely to attend U-5 services if they expected an HIV test than those who reported being less likely to attend U-5 services. 24

32 Table 10. Mothers' Knowledge of HIV Testing at Six Week U-5 Services Control Simple Comprehensive N (%) N (%) N (%) P-Value Offered HIV test Not offered 94 (71.76) 44 (38.94) 59 (32.42) < 0.01 Offered 37 (28.24) 69 (61.06) 123 (67.58) Knowledge of HIV test offer* 0.84 Did not know 21 (56.76) 36 (52.17) 63 (51.22) Knew 16 (43.24) 33 (47.83) 60 (48.78) Effect of knowledge of HIV test on visit attendance^ 0.13 Less Likely (1.67) More Likely 15 (93.75) 20 (60.61) 44 (73.33) Just as Likely 1 (6.25) 13 (39.39) 15 (25.00) Hypothetical effect of knowledge of HIV test on visit attendance % 0.26 Less Likely 6 (5.22) 3 (3.75) 8 (6.56) More Likely 84 (73.04) 50 (62.5) 88 (72.13) Just as Likely 25 (21.74) 27 (33.75) 26 (21.31) P-values were calculated using Pearson's Chi-Squared Test for difference in proportions *Only asked among mothers who received an offer for an HIV test ^Only asked among mothers who knew that they would be offered an HIV test % Only asked among mothers who did not know that they would be offered an HIV test Staff Interviews Facility staff at 20 intervention facilities (ten Simple Intervention and ten Comprehensive Intervention facilities) were asked to identify the greatest challenges to providing U-5 services. The most consistent response was under-staffing, cited by 13 out of 20 facility staff. Staff at four Comprehensive Intervention facilities and one Simple Intervention facility mentioned stock outs of HIV test kits and immunizations as a challenge. Only one staff member at a Simple Intervention facility indicated that there were no challenges. When asked about challenges specific to delivery of HIV services at U-5 clinic, common responses across both Simple and Comprehensive Intervention facilities were, again, staff and supply shortages. Three respondents from Comprehensive Intervention facilities cited that mothers sometimes refuse to be tested. One Comprehensive Intervention facility staff member cited that mothers sometimes feel uncomfortable about HIV testing and fear others finding out their status. She also indicated that mothers who are asked to be tested at the six week visit often feel stigmatized. Simple Intervention facility staff more frequently cited poor understanding by mothers of the importance of testing or a lack of knowledge of when testing should be done. Staff at two Simple Intervention facilities mentioned that a lack of space contributed to low levels of privacy and confidentiality when providing these services at U-5 clinic. Finally, another staff member at a Simple Intervention facility identified the high number of activities delivered at U-5 clinic as a challenge. 25

33 Staff at Simple and Comprehensive Intervention facilities were asked to comment on how mothers react to HIV services at U-5 clinic compared to HIV services offered at other times. Over half of the interviewees (seven from Comprehensive Intervention facilities and four from Simple Intervention facilities) indicated that mothers either did not respond differently or responded more positively. Two of these respondents referred to de-sensitization to testing during ANC services as a reason that mothers did not respond any differently while another mentioned that the information that the facility provides about the importance of testing makes mothers more accepting of HIV services at U- 5 clinic. One staff member from a Comprehensive Intervention facility said, [Mothers] do not react differently [at U-5 clinic than at other times], because they are already aware about HIV retest from the ANC time. Ten other interviewees indicated that mothers do have a different reaction to HIV services when offered at U-5 clinic compared to other times. Some of the reasons for this different reaction included fear and stigma, the newness of these services being integrated with U-5 clinic, and the burden associated with adding another service onto regular U-5 services. One staff member from a Simple Intervention facility said that some mothers have two U-5 cards so as to hide their HIV status. Another staff member from a Simple Intervention facility said, Mothers are reluctant [to get tested], because they think that they only come for vaccines at under-5, not HIV testing. Staff members from Comprehensive Intervention facilities were also asked a set of questions about the Comprehensive Intervention itself. All of the staff responded positively about the intervention, claiming that the intervention improved HIV testing for mothers and infants. All but one respondent indicated that the intervention improved clinic flow for U-5 services. One staff member highlighted that the intervention provided staff with the time to screen mothers adequately so that they would know who was due for retesting. Additionally, it allowed more time for health education. Seven of ten Comprehensive Intervention facility respondents indicated that the intervention created extra work. The three who said there was only a minimal amount of additional work indicated that the assistance of lay counselors and an effective division of labor was important to minimize any additional work. Nine out of ten respondents said that the intervention helped them do their job better and that they would continue with implementation of the intervention if given the chance. The staff member at the other facility indicated that it was already regularly doing EID and postnatal retests during U-5 services, so the intervention had little impact on the way in which they did their jobs. Focus Group Discussions Sixteen focus group discussions were conducted in the catchment area of eight facilities six groups included mothers who had attended outreach services, while ten groups included mothers who had attended static services. Perceptions of Health Services The majority of participants had positive things to say about the health services in their area. Several women noted that the benefits of the health services in their area were provision of immunizations, HIV testing to enable women to know their status, and provision of ITNs. Women who felt negatively about the health services in their area cited shortages of vaccines and other medicines and staff absence, tardiness, or lack of motivation. One woman in a static group said, The health services should be improved. There are few staff members, and they need to improve on record keeping. We spend a lot of time at the clinic because of lack of staff. There are not enough medicines at the facility. Mostly we are given pain killers, and for the rest, we are given a prescription that requires us to buy medicines. Static and Outreach Services The majority of participants from both static and outreach services reported bringing their children to U-5 services to receive immunizations and growth-monitoring. In general, the majority of women in both groups seemed to think that static services were better. Commonly cited reasons for this were that the stock of vaccines and other medications was more consistent at static services, there 26

34 are more services available in the event that something should be found wrong with the child, and static services are faster. One woman said that she preferred static services because HIV testing services are available there. The main advantage of outreach services was proximity and convenience. In two separate focus groups, participants indicated that the facility assigned them to attend outreach services. One woman relayed that mothers were divided into groups and told whether or not to go to static or outreach for U-5 clinic. Attitudes about HIV Services Being Offered at U-5 Many participants from the outreach groups indicated that they were not being tested at outreach (there were a few exceptions) and HIV was not often discussed at outreach. Women in the static services group were much more likely to report that they had received a health talk about HIV and HIV testing services during their last U-5 visit. The vast majority of women from both groups expressed positive opinions about HIV testing. Women recognized the importance of knowing their status, as that knowledge enables them to take better care of themselves and of their child. One woman from a static group said, We want to be retested early so that we know our status than waiting until we get too sick. Another woman from an outreach group indicated, Others don t want to be tested, but if you are breastfeeding and retested positive, you are advised on how to take care of your baby and yourself. Women in the outreach groups were specifically asked if they felt that mothers desire to avoid testing would drive them to choose outreach over static. Some women acknowledged that this might happen, but the majority of participants indicated that testing also does occur periodically at outreach. One woman said, Even at outreach we are tested for HIV. When due [for a retest], they keep our [U-5] card aside until you get tested. We always get tested. If not, it becomes difficult to be attended to if your card shows that you were not retested. DISCUSSION This evaluation found that both the Simple Intervention and the Comprehensive Intervention improved maternal HIV retesting, but did not have a significant impact on infant DBS testing. The interventions did not adversely affect immunizations. Interventions Increase HIV Maternal Retesting This evaluation found that both interventions yielded a statistically significant increase in retests for mothers, but no statistically significant difference in impacts between the two interventions was detected. The clearest possible driver of increased number of retests likely to be that facilities never stocked out and always had testing supplies when seeking to administer a test. However, many other factors could have driven this impact. Women who were already tested at ANC may not have recognized the benefit of another postnatal test without further counseling. The integration of services, in the case of the Comprehensive Intervention facilities, or the explicit instruction to provide these tests by the district in the case of both the Simple and Comprehensive Interventions, provided a platform for facility staff to educate mothers on the importance of retesting. Another driver of improved maternal retest rates could be the context in which mothers are being asked to get tested. HIV testing at routine services can serve to normalize HIV testing, thus reducing some of the fear and stigma. These results are reinforced by other qualitative studies, which have found similar results. 15,16 This normalization, however, can take time, indicating that the interplay 27

35 between the normalization of HIV services and the need to address fears and to ensure confidentiality can be a complex aspect of integrating services. Our qualitative results indicated that many mothers thought about the benefits of HIV testing in terms of their children s well-being, as well as their own health. An additional driver of increased maternal retest rates was possibly stemming from being in an environment where mothers are thinking about the health and well-being of their children. This can serve as an added motivator for mothers to learn their status. Both the Simple and Comprehensive Interventions included a pep talk at the beginning of the intervention period which reinforced testing guidelines by district health staff. During this talk, several facilities mentioned that they either do not prioritize retesting of mothers or have not done postpartum retesting at all in the past. As not all facilities stocked out of retesting kits, the pep talk likely played an important part in the initial increase in testing. Another factor could be that prior to the intervention, some facilities may have felt that test kits are better reserved for ANC when they are more likely to result in a positive test. Thus, in an environment of perpetual under supply, facilities may deprioritize postpartum testing. By guaranteeing a backup supply, this constraint was removed. No Detectable Impact on DBS Testing Our evaluation identified a small but non-significant increases in the number of DBS tests administered for both the Simple and Comprehensive Interventions. There are a number of reasons why impacts on DBS testing appear more muted. Two spillover effects may have caused an under-estimate of the true impact. At the district level, DBS kits may have been more likely to have been allocated to Control facilities where supplies were not reinforced via other channels. Also, facility to facility transfer of DBS kits may have occurred whereby kits allocated to intervention facilities were ultimately used at Control facilities. Other potential reasons include the low positivity rate of maternal testing, and higher overall baseline DBS testing rates. Finally, the study was not powered to detect a smaller impact on DBS testing of the magnitude that was ultimately estimated. No Detectable Impact on Immunizations Our results indicated that the intervention did not have an impact on DPT1 immunizations. Prior to the evaluation, the research team considered two ways in which immunization rates could have been negatively affected. The first was that women might have been less likely to bring their infants to U-5 services if they feared receiving an HIV test. Some of the focus group participants also mentioned this possibility. However, our data suggest that this was not the case. By using levels analysis, we would have been able to detect a drop-off in the number of DPT1 doses administered. Additionally, our qualitative methods reinforced that women generally have positive attitudes about testing. As mentioned before, those who might have been initially reluctant may find HIV testing more acceptable as it is routinized along with other maternal and child healthcare services. Other studies have found high acceptance of integration of HIV testing and immunizations. Rollins et al conducted a study at three health facilities in KwaZulu Natal, South Africa, where all mothers who brought infants in for immunizations were also offered HIV testing of infants. In that study, of the 646 mothers who brought their infants for immunizations, 90.4% agreed to the HIV testing and 56.8% of those mothers returned for results

36 The research team also considered that the interventions might negatively impact immunizations by creating more work for the staff. Many staff operating in low-resource areas are already overburdened, and adding additional services could add stress. High staff turnover or inadequate training can also lead to confusion about what services are meant to be provided at a given point in time. 23 Additionally, mothers have often cited increased wait times as a drawback for integration of services. 15, 16 Our interviews with facility staff did indicate that the Comprehensive Intervention created more work for them. The staff, however, recognized the importance of the changes and noted that despite the increased workload, they would like to continue the intervention rather than return to the way things were before. HIV Test Kit Supplies The evaluation highlighted the challenge of maintaining a consistent supply of HIV tests and the effect that stock outs can have on provision of services. Throughout the intervention period, there were many instances undersupplied facilities, particularly for DBS test kits. Facilities were more likely to run out of DBS than either rapid test commodity. It also appears that supply chain problems were more acute at the beginning of the intervention period before tapering off over time. Anecdotal evidence from district pharmacies suggested continued undersupply of DBS at the district level which led to local stock outs. Integration of Services Our results indicate that HIV testing services and immunization services can be coupled without negatively affecting immunization services The context in which these services are integrated, however, is critical. It is important that integration of services does not simply mean that nurses are asked to provide more services in the same amount of time without considerations of optimizing efficiency. Effective screening tools that allow staff to quickly determine who needs what services is an important component of successful integration The increased retesting rates achieved in this evaluation were largely carried out by PMTCT volunteers, Environmental Health Technicians (EHTs), and classified employees who have undergone training (i.e. Facility cleaners, watchmen, etc). Furthermore, transitioning to new operational models and retraining staff takes time to take root. Integration efforts should be closely monitored at the beginning and subject to random checks later to guarantee longevity of effects. Lack of regular retraining and follow-up is one possible explanation for the downward trend in testing during the second half of the implementation period. The component of both the Simple and the Comprehensive Interventions that guaranteed testing supplies was a critical component of the interventions. Stock outs can be a challenge, especially when integrating services, as a stock out in one commodity can serve to undermine both services. 15 The staff that we interviewed also cited stock outs as significant challenges to providing under-five services, and mothers in the focus groups mentioned that this is often frustrating. Evaluation Strengths This evaluation had several strengths. The constrained randomization approach ensured comparability between the study arms despite having relatively few facilities per arm, allowing for a more rigorous comparison between groups and greater attribution to the interventions. An evaluation that utilized measures from a single or even a few time periods could have been highly sensitive to temporary stock outs or other external shocks that affected some treatment arms disproportionally. The repeated measures from the baseline time period, however, enabled us to make estimates that were representative of the general levels of service delivery for both HIV testing and immunization prior to implementation. 29

37 We also measured testing of both mothers and infants, where previous studies looking at the integration of HIV testing and immunization services focused solely on the infant. 17, 18, 22 Finally, we used multiple sources to verify all administrative data collected for all outcome variables. Evaluation Limitations This evaluation also had limitations. Spillover effects may have impacted our results. Resupplying intervention facilities directly may have alleviated district-level supply shortages and allowed Control facilities to receive more testing supplies than they would have otherwise. As a result, the measured impact on maternal retests and DBS tests conducted of the two interventions may be underestimated. This evaluation did not attempt to estimate the proportion of transferred patients that would have remained at their original facility if the Comprehensive Intervention was universally implemented in all facilities. Consequently, results may reflect mothers who attended other facilities or outreach services to either avoid HIV testing or to take advantage of reinforced stock at one facility. There are some limits to the data quality due to the study s reliance on administrative data. The HIV activity sheet was only used in intervention facilities, limiting our ability to validate the number of retests done in Control facilities. Data for all outcomes in the intervention period were validated, but such validation was not conducted for the baseline period. Baseline data that separated static and outreach immunizations were not available, making it impossible to assess if the interventions affected the proportion of mothers attending static versus outreach U-5 clinics. It is possible that the interventions, which were focused on static settings, could have led mothers to attend outreach services instead. Mothers did not emphasize this during focus groups, even when they were probed for information on the topic, though the possibility cannot be ruled out. Most importantly, the total number of infants immunized did not change, suggesting this was not an important limitation. The exit survey revealed that fewer women than expected knew about testing services prior to their U-5 visit. While the study team utilized multiple channels for publicizing the intervention, these messages may not have reached the majority of mothers. This may be an indication that some of the synergies of combining services were not fully realized. A constrained randomization approach like the one utilized can be overly restrictive by limiting the number of possible assignments that fit the pre-specified criteria. Add implication of which Finally, the facilities in the sample are all supported by the ZCAHRD BUPIP program which has helped routinize the provision of a broad range of HIV services in these facilities. The results of this study will be most applicable to similar facilities that are already supported by implementing partners of the Ministry of Health. RECOMMENDATIONS Three important recommendations emerge from this evaluation to improve Zambia s performance along the PMTCT cascade. Reinforcing HIV test supply chains should be a top priority, as many lives might be lost due to supply chain failures. The large impact of the Simple Intervention on maternal retesting 30

38 rates indicates that solving HIV commodity supply chain challenges can yield important health benefits and should be prioritized. The cost per additional HIV-positive mother identified was $1,192 USD for the Simple Intervention, but at-scale implementation featuring systemic supply chain solutions could reduce the cost to below $100 USD per HIVpositive mother identified (see Appendix C). Similar challenges were observed in the transportation of DBS samples to the central lab. On average, it took 2-3 months for DBS samples to be received by the central lab. Due to the vulnerability of infants to HIV, this time lag indicates that many HIV-positive infants will die before receiving their test results and initiating treatment. Overall, this study makes clear that solving critical supply chain failures will save lives in a cost-effective manner. Integration of HIV testing and immunization services can be implemented without harming immunization uptake in areas with high acceptance of HIV testing. However, sustained monitoring and operational support is recommended if such integration is to yield benefits over time. This evaluation revealed that such integration did not affect immunization rates, indicating that the risks in areas with high acceptance of HIV testing are low. However, the evaluation did not statistically confirm any additional effect of integration on maternal or DBS HIV testing beyond that of the Simple Intervention. Since MoH guidelines already stipulate maternal retesting every three months and DBS testing of HIV-exposed infants at six weeks, many health facilities already use the six-week U-5 visit to conduct HIV testing activities. Thus, the primary advantage of formally recommending integration maternal retesting and DBS testing with the six-week U-5 visit in MoH guidelines would be to further routinize integration of such testing in the same way maternal HIV testing is routinized in ANC. The muted impact of integration in this evaluation reveals that routinizing such integration is likely to require more support than the six week training and piloting period utilized in this study. In particular, continuous data collection and oversight of HIV testing activities is likely required to enact a sustained improvement over time. Simple improvements can be made via improved coordination between EPI and HIV policy counterparts. Simple, low-cost actions can further improve coordination between ministries and health service provision, including: o Updating HIV fields on U-5 cards to match guidelines. Currently there is no place to record postpartum retest results or the second DBS test result. o Integrating HIV services into other U-5 data collection tools such as tally sheets and increase size of facility monitoring tools to improve legibility. o Aligning postpartum retesting time periods with the immunization schedule rather than time since last test. Opportunities include: 6 weeks (DPT, OPV, PCV, RV), 9 months (Measles 1), and 18 months (Measles 2) o Joint messaging and sensitization between PMTCT and immunization programming 31

39 ACKNOWLEDGEMENTS The research team would like to thank and acknowledge the following personnel whose participation made this research possible: the Southern Province Provincial Medical Office; the District Health Offices in Choma, Livingstone, and Monze; the staff and volunteers at the 60 health facilities, and the mothers and their infants who attended under-5 clinic during the evaluation, participated in focus groups, and participated in interviews. 32

40 SOURCES 1. Goverment of the Republic of Zambia & National AIDS Council. Zambia Country Report: Monitoring the Declaration of Commitment on HIV and AIDS and the Universal Access. (2014). at < ries/zmb_narrative_report_2014.pdf> 2. Joint United Nations Programme on HIV/AIDS (UNAIDS). Global Report: UNAIDS report on the global AIDS epidemic (2013). at < 3/UNAIDS_Global_Report_2013_en.pdf> 3. UNAIDS. AIDS by the Numbers. (2013). 4. Joint United Nations Programme on HIV/AIDS (UNAIDS). Countdown to Zero: Global plan towards the elimination of new HIV infections among children by 2015 and keeping their mothers alive (UNAIDS, 2011). 5. United Nations Children s Fund. Towards an AIDS-Free Generation - Children and AIDS: Sixth Stocktaking Report. (UNICEF, 2013). 6. Newell, M.-L. et al. Mortality of infected and uninfected infants born to HIV-infected mothers in Africa: a pooled analysis. The Lancet 364, (2004). 7. Violari, A. et al. Early antiretroviral therapy and mortality among HIV-infected infants. N. Engl. J. Med. 359, (2008). 8. Phiri, C. & Laboratory Services, Zambian Ministry of Health. (2012). 9. WHO Department of Immunization, Vaccines and Biologicals & UNICEF Programme Division. GIVS: Global Immunization Vistion and Strategy (WHO and UNICEF, 2005). at < 33

41 10. Wallace, A. S., Ryman, T. K. & Dietz, V. Experiences integrating delivery of maternal and child health services with childhood immunization programs: systematic review update. J. Infect. Dis. 205 Suppl 1, S6 19 (2012). 11. Clements, C. J., Nshimirimanda, D. & Gasasira, A. Using immunization delivery strategies to accelerate progress in Africa towards achieving the Millennium Development Goals. Vaccine 26, (2008). 12. Ryman, T. K. et al. Integration of routine vaccination and hygiene interventions: a comparison of 2 strategies in Kenya. J. Infect. Dis. 205 Suppl 1, S65 76 (2012). 13. WHO. Summary of WHO Position Papers - Recommended Routine Immunizations for Children (Table 2). (2014). at < 14. WHO & UNICEF. Zambia: WHO and UNICEF estimates of immunization coverage: 2012 revision. (2012). at < 15. Ryman, T. K. et al. Community and health worker perceptions and preferences regarding integration of other health services with routine vaccinations: four case studies. J. Infect. Dis. 205 Suppl 1, S49 55 (2012). 16. Wallace, A. et al. Qualitative assessment of the integration of HIV services with infant routine immunization visits in Tanzania. J. Acquir. Immune Defic. Syndr , e8 e14 (2014). 17. McCollum, E. D. et al. Superior uptake and outcomes of early infant diagnosis of HIV services at an immunization clinic versus an under-five general pediatric clinic in Malawi. J. Acquir. Immune Defic. Syndr , e (2012). 18. Goodson, J. L. et al. Evaluation of using routine infant immunization visits to identify and follow-up HIV-exposed infants and their mothers in Tanzania. J. Acquir. Immune Defic. Syndr , e9 e15 (2013). 19. Central Statistical Office Census of Population and Housing: Population Summary Report. (2012). 34

42 20. World Bank. World Development Indicators - Zambia. at < 21. Central Statistical Office (CSO), Ministry of Health of Zambia, Tropical Diseases Research Centre, University of Zambia & Macro International Inc. Zambia Demographic and Health Survey (CSO and Macro International Inc, 2009). 22. Rollins, N., Mzolo, S., Moodley, T., Esterhuizen, T. & van Rooyen, H. Universal HIV testing of infants at immunization clinics: an acceptable and feasible approach for early infant diagnosis in high HIV prevalence settings. AIDS Lond. Engl. 23, (2009). 23. Partapuri, T., Steinglass, R. & Sequeira, J. Integrated Delivery of Health Services During Outreach Visits: A Literature Review of Program Experience Through a Routine Immunization Lens. J. Infect. Dis. 205, S20 S27 (2012). 24. Khatun, J. & Roy, N. C. Missed opportunities for reproductive and child health services of clients in urban NGO clinics of Bangladesh. Matern. Child Health J. 10, (2006). 25. Horwood, C. et al. Prevention of mother to child transmission of HIV (PMTCT) programme in KwaZulu-Natal, South Africa: an evaluation of PMTCT implementation and integration into routine maternal, child and women s health services. Trop. Med. Int. Health 15, (2010). 35

43 APPENDIX A TIMELINE OF ACTIVITIES Year Month Develop protocol Submit IRB Receive IRB approval Develop operational plan Orient district health staff Train health facility staff Intervention piloting Evaluation period Data collection/entry Interim analysis on EPI Final data analysis ` APPENDIX B RANDOMIZATION APPROACH This evaluation used a stratified and constrained randomization approach. Six hospitals and primary care units affiliated with hospitals were excluded from the initial sampling frame of all governmentrun facilities in Livingstone, Monze, and Choma districts. xv The 77 remaining facilities were stratified based on district and on urban / rural divide. xvi Within each stratum, facilities were randomly selected semi-proportionately, with a slight intentional oversampling of urban facilities. After randomly selecting 60 facilities into the study, these facilities were ranked by their average number of immunizations per month xvii and divided into 20 sets of three facilities (ex. the three facilities with the three highest DPT1 volumes in group one, the three facilities with the fourth through sixth highest DPT1 volumes in group two, etc). Within each set of three, a facility was randomly assigned to one of the three study arms. A series of F-tests was administered to each candidate sample to assess balance across intervention arms on the number of DBS tests, number of DPT1 immunizations, and number of first ANC visits. If a p-value of less than 0.90 was measured for any of the three variables, the assignment was discarded. This randomization procedure was done 500 times. The remaining randomization assignments were then tested for external validity against the overall sample frame on the same three variables, discarding randomization assignments with p- values of less than 0.20 for any of the variables. Out of 500 randomization assignments generated xv Hospitals and Hospital Affiliated Health Centers (HAHCs, primary care facilities housed inside hospitals) were excluded from the sample because the catchment areas of the population they serve overlap with other facilities making community sensitization activities difficult. Health facilities or health posts without a full time nurse or midwife trained to do DBS testing were excluded from the sampling frame. This information was collected during pre-evaluation exploratory visits. xvi Rural was defined as more than 10 km along the road away from the District Health Office. These data were based on administrative reports and have since been confirmed by analysis using GPS coordinates and Google Maps. There were no facilities in the rural Livingstone stratum. xvii DPT1 data from January 2012 to March 2013 used for sampling was provided by the MoCDMCH, the most recent data available at time of sampling in June

44 and tested, 16 passed all of the tests imposed. The final randomization assignment was then randomly selected from these 16. APPENDIX C SIMPLE INTERVENTION CE MODEL A model based on study findings was developed to estimate the cost-effectiveness of the Simple Intervention to identify additional HIV-positive mothers. It is estimated that the Simple Intervention requires $1,192 USD for each additional HIV-positive mother identified. This figure should be viewed as an upper-bound cost, since costs in an at-scale, non-study context are likely to be far lower. If stock out issues can be solved at minimal cost without requiring additional transport to facilities, cost per additional HIV-positive mother identified is $394. Likewise, if reinforcement of existing HIV testing guidelines can be integrated into other pre-planned training activities, or if emphasis of HIV testing is already high, then it would cost $878 USD per additional HIV-positive mother identified. If both supply chain and knowledge issues could be addressed at minimal cost, then the cost per additional HIV-positive mother identified could be as low as $24 USD. Model inputs and cost estimates are summarized in Table 12 below. Table 12: Simple Intervention Cost-Effectiveness Model Inputs Input Description Value Unit Notes Number retests per clinic per month (Control) 18 Doses Study data Simple Intervention impact on retests 83% Study data Percent retests that are positive 1.15% Study data Number of facilities in sample 60 Facilities Study data Cost of Determine antibody test kit $ 0.70 USD Study data Cost of Unigold test kit (confirmatory) $ 1.50 USD Study data Staff time to administer tests 0.1 hours Estimate Cost of one staff person hour $ 2 USD Assumes lay counselor is administering tests Guideline reinforcement sessions (per 6 months) 1 Sessions Study data Resupply visits (per 6 months) 2.65 Visits Study data Facility trip cost $ 52 USD Assumes motorcycle. 60 kms / trip. $0.50 / km, $15 / trip for driver, 15% overhead Number of facility staff for reinforcement per clinic 1.5 People Study data Facility staff per diem (includes transport) $ 30 USD Study data DHO staff per diem $ 30 USD Study data Venue for reinforcement session $ 150 USD Estimate Facilities per reinforcement session 20 Facilities Study data Costs per HIV Positive Mother Identified Value Unit Full Simple Intervention $ 1,192 USD Including test administration and reinforcement only $ 394 USD Including test administration and resupply transport only $ 878 USD Including test administration only $ 24 USD 37

45 APPENDIX D POWER CALCULATIONS NOTE: The original power calculations assumed a cluster randomized design with binary, personlevel outcomes. The immunization calculation drove the overall sample size determination since it had the smallest policy-relevant effect size. Relevant minimum detectable effect sizes (alpha = 0.1, power = 0.80) of the original calculations without covariate adjustment were: o o o 0.10 effect on DPT1 of Comprehensive versus Simple + Control facilities 0.74 effect on retests of one Treatment versus Control 0.74 effect on DBS tests of one Treatment versus Control The original design required the matching of individuals across multiple health facility registers. When this proved infeasible, the analytical approach was adjusted to become a levels-based, difference-in-difference analysis. The power calculations without covariate adjustment with updated baseline numbers and analysis reveal similar power (Table 13). Table 13: Study Power Calculations xviii Outcome Number of DBS Tests Number of Immunizations Number of Retests Alpha Power 80% 80% 80% # Treatment Facilities # Control Facilities Mean Treatment (# of Tests / Doses) Mean Control (# of Tests / Doses) SD Treatment SD Control from baseline # Baseline Measures # Endline Measures Correlation: Baseline to Baseline Correlation: Baseline to Endline Correlation: Endline to Endline Obs / Cluster (Months of data) Intra Cluster Correlation xviii These power calculations are updated from the original calculations which required tracking of individual patients which proved infeasible. 38

46 MDES xix 66% 13% 69% xix The MDES figures represent upper bounds as the calculation does not account for covariates outside of baseline rates that increase power. 39

47 APPENDIX E ADDITIONAL DBS REGRESSION RESULTS Table 14: Linear Regression Results for Log of 1 st and 2 nd DBS Tests / Facility / Month Covariates 1st DBS test 2nd DBS test e^b P-value [90% CI] e^b P-value [90% CI] Intervention Arm Control Ref Ref Simple [0.80,1.09] [0.84,1.14] Comprehensive [0.92,1.21] [0.97,1.22] Time Period Baseline Ref Ref Intervention period [0.64,1.03] [0.69,1.08] Intervention Impacts: Simple v Control [0.93,1.35] [0.87,1.24] Comprehensive v Control [0.87,1.26] [0.88,1.29] # of Monthly DPT1 Immunizations < 0.01 [1.00,1.01] [1.00,1.01] District & Urban/Rural Stratum Urban Choma Ref Ref Rural Choma [0.73,1.29] [0.84,1.46] Urban Livingstone [1.05,2.25] [1.03,2.17] Urban Monze [0.86,2.66] [0.99,2.43] Rural Monze [0.89,1.56] [1.02,1.78] Time [0.95,1.03] < 0.01 [1.06,1.14] Time [0.96,1.02] [0.93,1.03] Time [0.91,1.06] [0.93,1.01] Time [0.99,1.26] [0.98,1.14] Time [0.97,1.04] [0.97,1.05] Time (Intervention Pd) [1.00,1.08] [0.97,1.02] Known National Stock out of DBS Kits [0.81,1.18] [0.81,1.00] Number of HIV+ Mothers Anticipated < 0.01 [1.06,1.13] < 0.01 [1.05,1.14] Constant [0.81,1.63] [0.50,0.89] 40

48 Table 15: GEE Regression Results for Number of DBS Tests per Facility per Month Covariates IRR P-value [90% CI] Intervention Arm Control Ref Simple [0.58,1.12] Comprehensive [0.70,1.37] Time Period Baseline Ref Intervention period [0.60,1.21] Intervention Impacts: Simple v Control [0.86,1.56] Comprehensive v Control [0.76,1.44] # of Monthly DPT1 Immunizations [1.00,1.01] District & Urban/Rural Stratum Urban Choma Ref Rural Choma [0.41,1.68] Urban Livingstone [0.84,4.05] Urban Monze [0.52,3.51] Rural Monze [0.68,2.58] Time [0.99,1.19] Time [0.83,0.99] Time [0.89,1.11] Time [0.98,1.45] Time [0.91,1.03] Time (Intervention Pd) [0.98,1.07] Known National Stock out of DBS Kits [0.60,1.03] Number of HIV+ Mothers Anticipated < 0.01 [1.09,1.27] Constant [0.45,2.19] 41

49 APPENDIX F ADDITIONAL RETEST REGRESSION RESULTS Table 16: GEE Regression Results for 6 Week Retests and Total Number of Retests (6 Wk, 6 Mo, 9 Mo, 12 Mo, 18 Mo) Covariates IRR* # of 6 Wk Retests # of Total Retests P- value [90% CI] IRR* P- value [90% CI] Intervention Arm Control Ref Ref Simple [0.16,0.62] [0.21,0.78] Comprehensive [0.33,1.14] [0.38,1.27] Time Period Baseline Ref Ref Intervention period [0.79,7.02] [0.52,2.15] Intervention Impacts: Simple v Control < 0.01 [2.08,8.07] < 0.01 [1.91,6.45] Comprehensive v Control [1.40,3.98] [1.00,2.69] # of Monthly DPT1 Immunizations [1.00,1.02] [1.00,1.02] District & Urban/Rural Stratum Urban Choma Ref Ref Rural Choma [0.39,1.46] [0.36,1.46] Urban Livingstone < 0.01 [0.17,0.63] [0.21,1.20] Urban Monze [0.43,2.34] [0.37,3.43] Rural Monze [0.46,1.20] [0.39,1.75] Time < [0.83,1.20] [0.82,1.10] Time [0.84,1.23] [0.95,1.22] Time (Intervention Pd) [0.91,1.02] [0.94,1.04] Number of 1st ANC Visits [1.00,1.02] [1.00,1.02] Distance to DHO [0.99,1.01] [0.99,1.01] Constant [0.15,61.41] [2.07,251.50] *Incidence Rate Ratio 42

50 Table 17: Regression Investigation Intervention Period Time Trends in Maternal Retest Uptake Covariates Coeff # of 6 Wk Retests # of Total Retests P- value [90% CI] Coeff P- value [90% CI] Intervention Arm Control Ref Ref Simple [-3.55,-0.66] [-14.95,0.01] Comprehensive [-2.61,2.14] [-11.71,13.63] Time Period Baseline Ref Ref Oct ' [-1.16,4.27] [-9.65,4.94] Nov ' [0.29,10.03] [-4.89,15.08] Dec ' [0.09,5.12] [-6.24,5.13] Jan ' [0.56,6.85] [-2.25,19.23] Feb ' [-0.05,3.36] [-3.48,7.87] Mar ' [1.09,8.53] [1.03,15.26] Intervention Impacts: Simple v Control (Oct) [1.77,9.84] < 0.01 [12.55,37.29] Simple v Control (Nov) [-0.68,10.39] [1.51,27.84] Simple v Control (Dec) < 0.01 [4.03,11.98] < 0.01 [13.67,36.57] Simple v Control (Jan) [-1.42,6.73] [-3.40,24.85] Simple v Control (Feb) [0.44,6.47] [0.21,18.34] Simple v Control (Mar) [-2.60,7.01] [-3.13,19.37] Comprehensive v Control (Oct) [4.04,19.63] [5.40,42.78] Comprehensive v Control (Nov) [-1.67,11.54] [-2.14,25.73] Comprehensive v Control (Dec) [1.17,11.30] [-3.25,21.93] Comprehensive v Control (Jan) [1.67,9.70] [-6.36,15.94] Comprehensive v Control (Feb) < 0.01 [2.77,9.10] [-0.40,18.58] Comprehensive v Control (Mar) [-4.87,3.94] [-4.07,12.06] Constant < 0.01 [1.97,4.62] < 0.01 [7.28,21.13] 43

51 APPENDIX G ADDITIONAL DPT1 REGRESSION RESULTS Table 18: GEE Regression Results for Monthly Average of DPT1 Doses Covariates IRR* P-value [90% CI] Intervention Arm Control & Simple Ref Comprehensive [0.74,0.99] Time Period Baseline Ref Endline [1.00,1.10] Intervention Impacts: Comprehensive v Control & Simple [0.92,1.10] District & Urban/Rural Stratum Urban Choma Ref Rural Choma [0.82,1.56] Urban Livingstone [0.77,1.57] Urban Monze [0.48,1.23] Rural Monze [0.89,1.57] Baseline Avg # of ANC Visits < 0.01 [1.02,1.03] Distance from DHO [1.00,1.01] Constant < 0.01 [6.23,11.83] *Incidence Rate Ratio 44