Supplementary webappendix

Transcription

1 Supplementary webappendix This webappendix formed part of the original submission and has been peer reviewed. We post it as supplied by the authors. Supplement to: Watt JP, Wolfson LJ, O Brien KL, et al. Burden of disease caused by Haemophilus influenzae type b in children younger than 5 years: global estimates. Lancet 2009; 374:

2 Web appendix: Methods to estimate the global burden of disease due to Haemophilus influenzae type b and Streptococcus pneumoniae in children less than 5 years of age L J Wolfson, K L O'Brien, J P Watt, E Henkle, M D Deloria-Knoll, N McCall, E Lee, K Mulholland, O S Levine, and T Cherian for the Hib and Pneumococcal Global Burden of Disease Study Team World Health Organization, Geneva, Switzerland (L J Wolfson, T Cherian); GAVI s PneumoADIP, Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA (K L O'Brien, E Henkle, M D Deloria-Knoll, N McCall, E Lee, O S Levine); Hib Initiative, Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA (J P Watt); and London School of Hygiene and Tropical Medicine, London, UK (K Mulholland) Correspondence: Dr Lara J. Wolfson, Office of the Assistant Director-General, Health Security and Environment, World Health Organization, 20 Avenue Appia, CH-1211 Geneva 27 Switzerland wolfsonl@who.int Current affiliations: E Lee: New York City Department of Health and Mental Hygiene, Bureau of Communicable Disease 1

3 Abstract Haemophilus influenzae type b (Hib) and Streptococcus pneumoniae (Spn) are important causes of illness in children under 5, together accounting for between 10% and 20% of all under 5 deaths occurring outside the neonatal period. For public health decision makers at national, regional, and global levels to make effective public health decisions regarding newly available vaccines, as well as other control measures available to combat the burden of these illnesses, credible country specific estimates of disease burden are needed. A comprehensive, systematic effort to estimate, by country, the number of severe illness cases and deaths in children 1 to 59 months of age due to Hib and Spn was undertaken as a partnership between the World Health Organization (WHO) and the Global Alliance for Vaccines and Immunization s (GAVI's) PneumoADIP and Hib Initiative. Disease burden was estimated for the three main severe syndromes caused by these two organisms: pneumonia, meningitis and non-pneumonia, non-meningitis invasive disease. These methods use data gathered from a comprehensive literature review which identified over 15,000 articles in multiple languages, which were systematically assessed for quality. The results from more than 245 studies from 75 countries were included in the mathematical models used to estimate disease burden. This paper describes the methods, and discusses limitations in the data. Although resulting burden figures may be underestimates in some countries, due to the quality and availability of data, the methods described in this paper provide guidance in prioritizing future data collection efforts and allow for the estimation of burden prevented through vaccination. Keywords: modeling, disease burden, Haemophilus influenzae type b, Streptococcus pneumoniae; child mortality; vaccines; pneumonia; meningitis; meta-analysis. 2

4 Introduction Haemophilus influenzae type b (Hib) and Streptococcus pneumoniae (Spn, pneumococcus) are major causes of bacterial meningitis, pneumonia and other serious syndromes among young children, especially in the developing world. Credible estimates of global, regional, and country burden of Spn and Hib are needed to understand the relative importance of these diseases; the potential for their control through public health interventions such as vaccination; and for decision-making related to vaccine development. The process of generating such estimates also serves to identify key gaps in available data that allows for prioritizing research activities to generate more precise estimates in the future. Ideally, measures of disease burden would be based on empirical (not modeled) data from reliable disease surveillance, and cause-specific vital registration systems. However, given the inadequacy of current diagnostic tools; challenges in case ascertainment; and vital registration systems which currently do not provide reliable and timely information in the majority of countries that constitute most of the disease burden 1, 2 modeling approaches are used to estimate levels and trends in disease burden. The World Health Organization (WHO) Hib and Pneumococcal Global Burden of Disease project aimed to estimate cases and deaths from Hib and Spn pneumonia, meningitis, and invasive nonpneumonia/non-meningitis (NPNM) among children 1-59 months of age at country level in the year 2000, allowing aggregation upwards to regional and global levels. This paper describes the methods developed for that estimation process. The estimates themselves are presented in detail in two companion papers: O'Brien et al 3 (Spn), and Watt et al 4 (Hib). Methods The methods and estimates were developed by a working group through an interactive process that included two reviews by an independent Expert Review Panel, with revisions based on their comments and suggestions. This process was designed to comply with new guidelines instituted by WHO to ensure that 1, 2, 5 official estimates are transparent and clearly documented. In general, several approaches can be considered for estimation of any of the three disease syndromes (pneumonia, meningitis, and NPNM): a proportional approach that starts with a mortality or morbidity envelope of the clinical syndrome and apportions those attributable to Hib or Spn to derive cause specific deaths or cases; an incidence-based approach that starts with clinical disease incidence, either etiology-specific or for an entire syndrome (applying etiological fractions when the starting point is the incidence of a clinical syndrome) to derive cause specific cases, and then estimates the number of deaths by applying a case-fatality ratio; or for at least one disease syndrome (if not two), the disease burden could be estimated by looking at the relative occurrence of one syndrome relative to another (i.e. triangulation). The availability and quality of data obtained from a comprehensive literature review 6 was used as a basis for the various modeling approaches used. For pneumonia, disease burden was estimated as an etiologic fraction of all-cause pneumonia cases and deaths. For meningitis, an incidence-based approach was used wherein incidence and case-fatality rates were derived from the literature. For serious nonpneumonia/non-meningitis invasive disease syndromes, incidence and case fatality rates were estimated indirectly based on the reported relationship between NPNM and meningitis cases and deaths. The overall structure of how different parameters fit together in estimating disease burden is given in Appendix Equation 1. Relative Risk of Disease in Human Immunodeficiency Virus (HIV) Infected Children An important consideration in estimates for all three syndromes is to adjust for the impact of the increased risk of disease incidence (i.e. cases) in children infected with HIV (See Appendix Equation 3). The relative risk of Hib and Spn disease in HIV-infected compared with HIV-negative children is estimated from a meta-analysis of available studies (Table 1) 7-11 ; the relative risk is not significantly different across syndromes. The majority of published studies were conducted in South Africa. Because of lack of other data we assumed the relative risk from the meta-analysis is generalizable globally. Literature Review A systematic review of published literature from 1980 to October 2005 containing data on Hib and Spn invasive disease was conducted, searching 5 global databases and 4 regional databases. 6 Other sources of data included contacts with investigators of unpublished studies and WHO consultation reports using the Hib Rapid Assessment Tool. 12 Additional studies meeting inclusion criteria but published between October 2005 and October 2007, were included as identified through the Expert Review Panel and country 3

5 consultation. The list of citations included in the final analysis are provided here (Appendix Pneumococcal Citations, page 30 and Appendix Hib Citations, page 38). Of 15,099 references initially identified, 1899 were selected for full text review based on screening by trained individuals, all of whom had graduate training in medicine and/or epidemiology (see Figure 1). 6 A standardized questionnaire was completed by two separate reviewers for those studies which fulfilled further inclusion criteria; discrepancies between the two reviewers were resolved by an adjudicator. Quality Assessment of Literature Quality assessments for studies reporting incidence, age distribution, or case fatality ratio (CFR), were based on two subjective criteria, applied by two independent abstractors: the likelihood of the study missing cases and the reliability of the diagnostic methods for laboratory isolation. Papers were then classified into three categories: "A" papers where both reviewers judged both criteria to have been met; "B" papers where only one reviewer judged each criterion to have been met; "C" papers where both reviewers judged that neither criterion had been met or data was not available to allow a judgment. Category C studies, as well as all included studies from Asia and Africa, underwent a third quality assessment before a decision was made about inclusion in the final data set for meningitis. Only A and B studies were used in estimating meningitis incidence; studies from all categories were included for CFR. For meningitis incidence rates, out of 110 studies evaluated for Hib, 51 were excluded: 35 for problems with case ascertainment, 5 for problems with diagnostic methods, and 11 for both. 90 studies were evaluated for Spn, and 39 were excluded: 34 for problems with case ascertainment, 2 for problems with diagnostic methods, and 3 for both. Pneumonia Ascertaining pneumonia cases and deaths attributable to Hib or Spn is challenging, largely because of lack of standardized case definitions, difficulty in establishing the microbial etiology of pneumonia and differences in case ascertainment Any estimates of etiology-specific pneumonia cases and deaths must fit plausibly within envelopes of established estimates of under-5 pneumonia incidence 16 and pneumonia mortality 17 (see Appendix Equation 4). To estimate both cases and deaths using either a proportional mortality or an incidence based approach requires knowing Hib and Spn CFR, which are not generally available; the existing data are among hospitalized children and may not be representative of Hib and Spn pneumonia CFR occurring in the community; furthermore laboratory methods to determine bacterial etiology of pneumonia lack in sensitivity and specificity, depending on the body fluid that is assessed. Attributable Fractions of Pneumonia from Hib and Spn trials The best available data on proportional etiology of pneumonia is derived from vaccine probe studies 18, wherein a vaccine with a known efficacy against microbiologically-confirmed disease is administered to one part of a study population and not the other, and the burden estimated based on disease reduction in the vaccinated group. These were considered to be most representative of pneumonia occurring in the community and more likely to identify cases of Hib and Spn than conventional microbiological methods. There are three definitions of pneumonia that have been used in Phase III clinical trials of Hib and Spn conjugate vaccines: clinical pneumonia and clinical severe pneumonia, defined according to WHO's Integrated Management of Childhood Illness (IMCI) case management guidelines 19 ; and a standardized definition of radiological pneumonia. 20 The clinical definitions are meant for case management and err on the side of greater sensitivity at the expense of specificity for pneumococcal pneumonia. The various case definitions in the conjugate vaccine trials were evaluated to identify those that matched most closely the case definition used in studies to estimate all-cause pneumonia cases and deaths. Regardless of the pneumonia endpoint used, the proportion attributable to Hib disease is obtained by dividing the point estimate of vaccine efficacy against that endpoint by the overall efficacy against invasive disease. For the proportion attributable to Spn, the calculation is more complex; the point estimate of vaccine efficacy against the endpoint is first divided by the overall efficacy against invasive disease caused by the pneumococcal vaccine serotypes, and then by the proportion of pneumococcal disease attributable to vaccine serotypes in the population where the trial was conducted. The proportion of pneumonia attributable to Spn is then corrected for the effect of Hib vaccination in the trial populations. 4

6 This correction is needed since all PCV trials were done in the presence of Hib vaccine but the all-cause pneumonia case and death estimates are for a period when Hib vaccine was not in widespread use. The estimated proportion of each pneumonia endpoint that can be attributed to Hib or Spn is based on a random-effects meta-analysis (on the log scale, using the relative risk) of the results from Hib and Spn conjugate vaccine efficacy trials (Table 2) Random effects meta-analyses were used throughout the modeling process because fixed effects models do not incorporate between study variability and except in rare instances produce biased results. 39 Dual Approach: Proportional Estimates of Pneumonia Cases and Deaths The number of cases of Hib and Spn pneumonia is derived by applying the meta-analysis estimate of the proportion of clinical pneumonia cases 16 that are due to Hib and Spn to all-cause pneumonia cases in children under 5 years of age, adjusting for the higher incidence rates among HIV-infected children, and reductions due to use of Hib vaccine in the year The number of Hib and Spn pneumonia deaths is similarly estimated by applying the proportion of pneumonia deaths attributable to Hib or Spn to overall estimates of pneumonia deaths in children 1-59 months of age. This results in a difference in the age groups used to estimate cases (0-59 months) and deaths (1-59 months); the consequence is that the derived case-fatality rates will underestimate the true CFR. We assumed that bacterial pneumonia is typically associated with alveolar consolidation and if allowed to progress, at the time of death the pathology would be of sufficient magnitude to be visible on a chest radiograph. Thus, we have assumed that the proportion of pneumonia deaths that are caused by Hib/Spn is approximated by the proportion of pneumonia cases with alveolar consolidation on chest x-ray (CXR +) that are caused by Hib/Spn. This assumption is also supported by the increase in the proportion of cases caused by bacterial pathogens as pneumonia severity increases. For this approximation to hold, there are two assumptions that must be accepted. All bacterial pneumonia deaths are CXR (+), and the CFR of Hib or Spn CXR (+) cases is equivalent to the CFR of non-hib and non-spn CXR (+) cases. In high access to care settings, the case fatality ratio of Hib/Spn pneumonia is likely to be significantly lower than non- Hib/non-Spn pneumonia, whereas in areas of the world with less access to quality medical care the CFR of Hib/Spn pneumonia is likely to exceed that of the non-hib/non-spn pneumonia. The direction of bias therefore is to overestimate the Hib/Spn deaths in high access to care settings and underestimate them in low access to care settings, the latter where the majority of global pneumonia deaths occur. A benefit of this dual proportional approach to estimate cases and deaths separately is that while the etiology-specific incidence and case-fatality rates are not estimated directly, it is possible to derive them using the resulting estimates of Hib/Spn cases and deaths (see Appendix Table 1). We assumed that Hib pneumonia deaths only occur in children up to two years of age (discussed further in the Hib results paper 4 ), but that Spn pneumonia deaths occur throughout the first five years of life. We estimate separately deaths that occur in children co-infected with either Hib or Spn and HIV by effectively applying the derived Hib or Spn case-fatality ratio among HIV-negative children to Hib/Spn pneumonia cases occurring among HIV infected children. Uncertainty The key source of uncertainty in the pneumonia model is the between-study variability in the vaccine efficacy estimates. A jack-knife analysis 40, in which one study is omitted from the analysis at a time, illustrates the range of results that could be found (Table 3). For Hib, the Lombok trial pulls the overall estimate downwards and for Spn, the Gambia trial significantly pushes the meta-analysis upwards. Because we have assumed that all Hib pneumonia deaths occur only among children less than two years of age, the upper limit uncertainty bounds 3, 4 reflect the possibility that there may be deaths in older children. Meningitis For meningitis, where country-specific studies of Hib and Spn incidence and case fatality rates are available, an incidence-based approach was used. Estimated country-specific rates are combined with population figures, Hib vaccine coverage, access to care estimates and HIV prevalence data to estimate cases and deaths (Appendix Equation 1). In obtaining country-specific estimates, if there were several estimates from multiple studies for a country, the estimates were summarized by a random effects meta-analysis This allows a study's contribution to the estimate to reflect both the (relative) sample size of the study (within-study variability) as well as similarity with other studies (between-study variability). When there were only two studies in a group, the contribution of each study to the meta-analysis was based only on the within-study variability. 5

7 As data are not available for each model parameter in every country, imputation 41-43, or extrapolating the known information to settings where information is unknown, is necessary. Meningitis reported incidence rates are correlated with child mortality 6 ; they also may vary across geography, for example as seen by comparing reported Hib meningitis incidence rates in low mortality settings in Asia ( to ), to the United States, ( to ). To ensure that the data used for each country is as local as possible, very narrow geographic/mortality strata neighborhoods were defined, and gradually those neighborhoods were hierarchically expanded if data was not available within the narrow definition - a form of nearest-neighbor hot-deck imputation Countries were grouped according to under-5 mortality strata 48 ("low", <30 deaths per 1000 live births; "medium", 30-<75 deaths per 1000 live births; "high", 75-<150 deaths per 1000 live births; "very high", >150 deaths per 1000 live births), geographic region (based on 21 geographic subregions using 6, 49 United Nations definitions, plus a stratification in or out of the meningitis belt in Africa). Our algorithm for obtaining parameter estimates uses country-specific data wherever possible. So, provided that the literature review yielded one or more studies that are "representative" of the country (the overall mortality setting in which the study was done is akin to the child mortality levels in the year 2000; the study was not done in a special population; and the study met other minimal quality criteria), we used that study (or a meta-analysis of studies)--to obtain a country-specific point estimate (Figure 2) of both incidence rates (assuming 0% HIV prevalence among children 1-59 months), and case fatality rates among children with good access to care. The majority of studies with incidence and CFR information were among children <5 years of age. As some studies provided the data only for those <1 or <2 years of age, we inferred the overall <5 rates using multipliers derived from studies which reported rates for either/or <1, <2, and <5 years of age (Appendix Equation 2). We were able to include an additional 6 incidence studies for Hib, and 11 studies for Spn, via the meta-analysis of the proportion of cases <1 and <2 to obtain overall <5 rates. Table 4 shows the estimated age distribution of cases by mortality strata. Disease rates may be underestimated if children with invasive Hib or Spn disease do not reach facilities where case ascertainment takes place. Some studies collected data to estimate the proportion of cases missed and provided adjusted incidence rates, or data allowing quantitative adjustment of reported rates. Wherever this information was provided directly by study authors, the adjusted rates were used. Similarly, case fatality rates in those who do not have access to care are expected to be much higher than those reported in studies where treatment was provided. To determine the proportion of children with good access, country specific estimates derived from the Multiple Indicator Cluster Surveys (MICS) 50 were used. The proportion of children less than five years with suspected pneumonia in the past two weeks who were taken to a health care provider from these surveys were used as a proxy for access to care for meningitis. This may be an overestimate of access to care for meningitis since meningitis cases progress more rapidly than pneumonia cases and there is a more narrow time window for accessing care. The direction of bias is therefore to underestimate the CFR for meningitis through use of the MICS data as we have done. For countries where no MICS data was available, regional estimates were used, if available, or diphtheria/tetanus/pertussis vaccine (DTP3) coverage 51 was used in the absence of such data. For countries in Latin America where no other data was available, and for industrialized countries, 100% access to care was assumed. Country access to care ranged from 14% to 100%, with a mean of 74% and a median of 78%. Countries in Africa had the lowest mean access to care (49%). In the proportion with no access to care, a 90% meningitis CFR was assumed. This value was based on estimates of meningococcal meningitis CFR from the pre-antibiotic era. Uncertainty The available data -- both studies that contribute to the estimation of rates, and the reliance on imputation and extrapolation to obtain country-specific estimates - imply a large degree of uncertainty, the most important aspects of which are driven by the choice of conceptual approach and the inclusion/exclusion of data. The uncertainty values for the incidence and case fatality rates of meningitis are generated from estimates of sampling error (obtained through Taylor-series approximations ) or of non-sampling error. The latter was obtained first by running 16 separate models using different input combinations of quality scores (A only, A+B, A+B if no A, and A+B+C studies), with and without adjusting for HIV or the within-study incidence adjustment, and then a jackknife analysis leaving out one study at a time. We report uncertainty of the country-specific estimates according to the algorithm shown in Table 5. 6

8 Non-pneumonia/non-meningitis invasive disease Both Spn and Hib are known to cause invasive syndromes other than meningitis and pneumonia. These non-pneumonia/non-meningitis (NPNM) invasive disease syndromes include, among others, bacteraemia/sepsis, cellulitis, septic arthritis, osteomyelitis, epiglottitis, peritonitis, and pericarditis. For Hib, these syndromes are less common than either meningitis or pneumonia. For Spn, the bacteraemia/sepsis syndrome may be an important component of disease burden, both because of its relatively high incidence and resulting mortality in developing countries. Data on the incidence of these syndromes was very limited; however many studies reported on the relative proportion of meningitis and NPNM cases among invasive disease syndromes. The burden of NPNM is thus based on establishing the ratio of NPNM to meningitis cases, separately for very high/high and medium/low mortality strata, and multiplying this ratio by the estimated number of meningitis cases for each country. Rates of Spn NPNM syndromes are affected very significantly by the clinical threshold for case detection as Spn causes non-focal bacteremia. For example, if the clinical practice includes liberal policies on obtaining blood cultures from all febrile children, the incidence rates would be higher, but CFRs would be lower. NPNM cases due to Spn were separated into non-severe cases (e.g., those managed in the outpatient setting), and severe cases (i.e. hospitalized cases) based on the clinical case detection methods described in the study; we assumed that no mortality resulted from non-severe Spn cases. The number of deaths caused by NPNM was calculated by multiplying the severe NPNM cases by an appropriate CFR. A meta-analysis of the relationship between the CFR for NPNM syndromes and that for meningitis was used to obtain a multiplier which was applied to the country-specific estimate of meningitis CFR to obtain country-specific NPNM CFR. The meta-analysis multipliers for both cases and deaths are shown in Table 6. NPNM uncertainty bounds were calculated by applying the NPNM: meningitis case and CFR ratios to the meningitis upper and lower bounds. Accounting for Vaccine Impact The estimates of disease incidence should only be applied to the population at risk for the disease to calculate the number of cases, and hence, any of the population protected from disease by vaccination - either directly or indirectly - must be accounted for when estimating the number of cases in a country. As a result of our literature review approach, our meningitis incidence measures were those occurring in the absence of Hib and pneumococcal vaccine but Hib conjugate vaccine was in widespread use within some countries in the year 2000; pneumococcal conjugate vaccine (PCV) was extremely limited. To account for Hib conjugate vaccine coverage in our estimation of Hib meningitis cases and deaths we used the WHO-UNICEF estimates of Hib vaccine coverage 55 by year and calculated the proportion of the under-5 population in each country that had received 3 doses of Hib vaccine. To this, estimates of direct and indirect protection were applied, which allowed for estimation of the impact of immunization on Hib disease. Direct Effects of Vaccination To measure the direct effects of Hib vaccination, we rely on measures of vaccine efficacy (VE) against invasive disease reported from clinical trials. Seven studies 11, 26, report the efficacy of Hib vaccine, with point estimates ranging from 94% to 99.29% (62%-100%). Four studies reported the direct protection against all-serotype invasive pneumococcal disease 35, 36, 61, (personal communication, Steven Black), ranging from 42% to 88.7% (-28%-95.3%). Formal random-effects meta-analysis yields respective results of 96% (+/-2%) for Hib and 63% (+/- 28%) for Spn. Indirect Protection For the Spn conjugate vaccines, data collection is too geographically limited to fully estimate any indirect impact. For the Hib vaccine, in addition to a recent review paper 62, there are 6 studies from "mature" immunization programmes (i.e., Hib vaccine in use for several years) in the United States, Denmark, Israel, and the Gambia, and these are used to model the indirect effects of Hib immunization. There are a further 7 additional studies from programmes where the vaccine had only recently been 60, 64, 65, introduced. A least squares regression line 73 (R 2 =93%, Figure 3) was fit to the six data points from the mature programmes to obtain an equation linking population-level coverage among children under-5 to the reduction in the size of the population at risk, thus providing a way of accounting for herd immunity impact. 7

9 The results of the regression model, which are similar to those of an age-structured model based on the experience of Finland 68, 74, can be used to estimate, with uncertainty, the reduction in disease burden that are attributed to vaccine use. Country Consultation Per WHO guidelines, the country-specific disease burden estimates and methodology were communicated to WHO member states through the network of WHO regional and country offices. Respondents were asked to review the estimates and provide any additional quality data which might improve the accuracy of the estimates. Discussion These methods are likely conservative, and may underestimate the true disease burden. The reasons for this include: case ascertainment in most reported studies was limited to those who sought facility-based care; that study procedures (e.g. lumbar punctures, blood cultures) were not performed in all eligible subjects, especially in developing countries; inadequate specimen transport, handling, and laboratory techniques result in under-estimation of disease incidence; and antibiotic use before specimen collection lowers the ability to identify both Hib and Spn. Meningitis incidence may be as much as 10 times higher than estimated in some settings. A Hib vaccine probe study in Lombok, Indonesia found a culture confirmed meningitis incidence of 16/100,000 children <2 years but clinical meningitis cases prevented by the vaccine was 158/100, Publication bias is always a concern when conducting a literature review. 75 Most of the published studies used in the meningitis and NPNM analysis are descriptive in nature with no statistical tests to determine publication worthiness. Negative findings (low incidence) are likely indicative of limitations in detection and the minimum criteria for incidence required a large number of person-years of observation to minimize parameter instability. Unpublished studies for Hib and Spn disease are unlikely to be of high enough quality to be used for the primary outcomes. However, to ensure completeness of the input data, unpublished national or representative surveillance data was sought during the country consultation process. The approach used in estimating disease burden included several innovative features. First and foremost is the use of the hierarchical nearest-neighbour hot-deck imputation approach to estimating country-specific rates, rather than a regression-model type approach. The choice to do this was driven by a number of factors, including transparency in showing how any individual country estimate is derived; recognition of the limitations of the data available for providing a complete set of covariates on which to do such modeling; and acknowledging that while such models are designed to perform well on average, they are not necessarily as valid for individual predictions, and may not be the best choice in doing countryspecific estimates. Additionally, this approach allows the identification of studies which had high impact on overall estimates, and what studies could be done in the future to equalize the availability of quality data across regions. 3, 4 As an example, no single paper impacted the global estimate of Spn meningitis cases by more than +/- 6%; and only 14 papers resulted in more than 1% variation. However, while all but 12 papers used in the Hib meningitis incidence estimates cause less than 1% variation, the end results are sensitive to two studies, both in Asia; one reducing overall meningitis cases by 27% 27, the other raising overall cases by 19% 76, highlighting the potential value of gathering additional data at various mortality strata in that region. However, since cases and deaths from meningitis account for only a small proportion of the total cases and deaths from the three syndromes, with pneumonia the major syndrome, additional meningitis incidence or CFR data will have little impact on country, regional or global Hib or Spn estimates. There are numerous ways in which the validity or the consistency of the estimates can be checked. One internal consistency check is to examine how the ratio of the Hib:Spn meningitis and pneumonia CFRs--which are estimated independently--vary together. Spn meningitis is usually associated with higher CFR than Hib meningitis; only a limited number of countries were found, all with lower child mortality rates, where the estimated meningitis CFR was higher for Hib than for Spn. A second consistency check also highlighted a quirk of the modeling process. While very little is known about the differential CFR between Hib and Spn pneumonias, the estimated relationship between the Hib and Spn pneumonia CFRs is driven by the meta-analysis estimates of the relative relationship between CXR+ efficacy and clinical pneumonia efficacy. Thus, despite the lack of hard clinical evidence to show any differential, these methods dictate that the Spn pneumonia CFR will generally be higher than that for Hib as the ratio of Spn:Hib clinical pneumonia efficacy is 1.6 and 1.7 for CXR+ pneumonia. Countries where the estimated pneumonia CFRs are greater for Hib than for Spn are ones with high HIV prevalence, 8

10 which can alter the relationship between Hib and Spn incidence. A final consistency check was to compare the estimated CFR and incidence rates by syndrome with overall under 5 mortality levels, anticipating that both would show a relatively high degree of correlation; this plausibility check appears to be met (data unpublished). This exercise highlighted the importance of standardized data collection and reporting of surveillance data to facilitate burden estimation as well as study comparability, especially for diseases where the organisms cause multiple disease syndromes, each of which in turn can be caused by multiple pathogens and where disease incidence varies significantly within narrow age ranges. The development of such standardized data collection and reporting will be among the tasks that need to be undertaken as a consequence of this study. Although the utility of random effects meta-analysis (used throughout the modeling process) has been criticized 77, it was determined to be the best way to combine information into a single input parameter and estimate uncertainty. Parameter uncertainty as generated by the meta-analyses is just one small part of the overall uncertainty of the estimates and the uncertainty bounds represent both sampling and nonsampling error. Quantifying non-sampling error in global disease burden exercises has long been a particular limitation, and the approach provided in this paper provides at a minimum a valuable starting point for discussion. It particularly highlights the uncertainty that results not just from limited sample sizes and population variability, but from the paucity of data that is usually available to estimate disease burden. While high-quality data collection efforts in regions of the world where data is weakest might improve the representativeness of the input data in the model, they are unlikely to significantly change the estimated numbers of cases and deaths, since such data collection would be limited to meningitis and NPNM, which form only a fraction of the total burden. Economic and ethical constraints make further vaccine probe studies to inform the pneumonia estimates unlikely, although one could argue from a purely statistical standpoint that validation of the outlier results of the pneumonia clinical trials, which significantly influenced the estimated proportion of pneumonia deaths attributable to each organism, may yield important information. Quantifying disease burden in neonatal populations, in adult populations, and the impact that vaccine use has on disease burden in different population settings are important additional areas of work now being undertaken. The methods offered here represent an important step in providing a rigorous, methodologically sound approach to estimating the burden of these two diseases in children under five and provide a guide as to how such estimates might be approached in the future. 9

11 Table 1. Relative risk of pneumococcal infection associated with HIV infection in children Study site Years of Surveillance Age Group Disease Outcome SP Meta-Analysis of Relative Risk: ( ) Relative risk compared with HIVuninfected (95% CI) Soweto <2 years IPD 36.9 (21.8, 64.5) Johannesburg <12 years IPD 41.7 (26.5,65.6) Soweto months Bacteremic pneumococcal severe lower respiratory tract infection 42.9 (20.7, 90.2) Soweto <12 years Pneumococcal Meningitis 40.4 (17.7,92.2) Hib Meta-Analysis of Relative Risk: 7.38 ( ) Soweto <12 years Hib Meningitis 1.74 (0.23, 13.2) and 3.20 (0.74, 14.2) for <=1 and <=2 years of age, respectively Soweto months Bacteremic Hib lower respiratory tract infection 21.4 (9.4, 48.4) Soweto <12 months Invasive Hib disease 5.9 (2.7, 12.6) 10

12 Table 2. Summary of per-protocol vaccine efficacy against pneumonia, vaccine efficacy against invasive disease, and the proportion of invasive pneumococcal disease (IPD) cases caused by vaccine serotypes from Hib and Spn conjugate vaccine trials Conjugate Vaccine Trial* Clinical Pneumonia Severe Pneumonia Vaccine Efficacy Pneumonia with alveolar consolidation on chest x-ray Invasive Disease** % vaccine serotype (including 6A) among IPD Cases Hib Gambia (-4.1, 18.2) 9.4 (-7.5, 23.7) 22.4 (1.9, 38.6) 95 (67, 100) Lombok (-0.2, 7.7) 0.15 (-8.9,8.4) -12 (-35.7, 8.3) 84.1 (-31,100) Bangladesh 28 NA NA 32 (8, 50) 86 (-8, 100) Chile NA 13 (-7, 30) (-7, 43) 91.7 (64.8,100) SP NCKP 4.3 (-3.5,11.5) 31 NA 30.3 (10.7, 45.7) (82.7, 99.9) % (83.3, 98.5) 33 South Africa 10.8 (0.6,20) (4, 27) (4, 40) (32, 98) % (75.7, 100) 35 Gambia 36 7 (1, 12) 12 (-9, 29) 37 (25, 48) 77 (51,90) 64.9% (54, 75.7) Philippines 0.1 (-9.4,8.7) 37 NA 22.9 (-1.1,41.2) (32.0, 99.9) 80.8% (71.8, 89.7) 38 NA: not available; -- : not applicable; * All studies are randomized clinical trials except the Hib Bangladesh study, which is a case-control study with systematic vaccine allocation, and therefore included in the analysis; ** VE meningitis reported for Lombok and Bangladesh studies; Hospital controls were used in the analysis; Northern California Kaiser Permanente (USA); HIV (-) used in the analysis; Derived as average vaccine efficacy of the other three studies since not available directly from trial. 11

13 Table 3. Meta-analysis and jack-knife analysis of Hib and Spn conjugate vaccine efficacy trials for various pneumonia efficacy endpoints, showing the variability that could result from omitting any single trial, and the weight of that trial in the meta-analysis estimate. Conjugate Vaccine Trial Clinical Pneumonia Severe Clinical Pneumonia Chest x-ray confirmed Pneumonia (Used as proxy for mortality) Estimate Interval Weight Estimate Interval Weight Estimate Interval Weight Hib Meta-Analysis 5% (1%, 9%) 5.8% (-3%, 14%) 21.3% (3%, 36%) Without Gambia 4.5% (1%, 8%) 12% 5.4% (-9%, 18%) 27% 19.5% (-12%, 42%) 30% Without Lombok 8.1% (-2%, 17%) 88% 11.6% * (0%, 22%) 53% 28.0% (19%, 36%) 20% Without Bangladesh % (-7%, 32%) 26% Without Chile % (-6%, 12%) 20% 19.8% (-7%, 40%) 24% SP Meta-Analysis 8% (2%, 14%) 21.2% (12%, 29%) 35.8% (16%, 51%) Without NCKP 10% (2%, 17%) 38% % (11%, 58%) 25% Without South Africa 6.4% (0%, 13%) 22% 22.6% (0%, 40%) 90% 38.9% (12%, 58%) 25% Without Gambia 6.2% (0%, 12%) 24% 21.1% (14%, 28%) 10% 26.2% (19%, 33%) 25% Without Philippines 9.7% (3%, 16%) 17% % (13%, 57%) 24% -- : not applicable; * Trials that, when omitted, have a high impact are italicized. 12

14 Table 4. Estimated proportion of Hib and Spn meningitis cases under 1 and under 2 years of age Mortality Strata Low Medium High or Very High Hib Spn Under 1 Under 2: Under 1 Under 2 51% (2%) 82% (2%) 63% (3%) 83% (1%) N=38, n=20 N=34, n=19 N=28, n=15 N=28, n=19 70% (4%) 88% (2%) N=14, n=10 N=12, n=7 73% (3%) 85% (3%) 78% (5%) 96% (1%) N=19, n=11 N=11, n=7 N=13, n=10 N=3, n=3 *Estimated standard error is shown in parentheses; N=number of studies; n=number of countries. 13

15 Table 5: Uncertainty bounds for meningitis incidence and case-fatality rates Lower Bound The minimum of the following: (a) minimal estimate from the 16 models (b) the 2.5 th percentile from the estimates calculated using the jack-knife analysis OR, if the minimum is not different from the point estimate, (c) the 2.5 th percentile based on sampling error only Upper Bound The maximum of the following: (a) maximal estimates from the 16 model (b) 97.5 th percentile from estimates calculated using the jack-knife analysis OR, if the maximum is not different from the point estimate, (c) the 97.5 th percentile based on sampling error only Table 6. Key parameters for the Hib and Spn non-pneumonia/non-meningitis (NPNM) models Etiology Severity of NPNM Disease Mortality Strata Meningitis Incidence Multiplier Meningitis Incidence Multiplier SE Number of Studies Number of Countries Meningitis CFR Multiplier Meningitis CFR Multiplier SE Hib Severe Very Low Medium High Very High Spn Severe Non-Severe Very Low Medium High Very High Very Low Medium High Very High *Assumed that non-severe SP had a CFR of NA NA 14

16 Figure 1. Summary of references identified and studies abstracted for the Hib and Spn invasive disease literature review. [reproduced from 6 ] 15,099 Hib & Spn references identified from database search and title/abstract screened + 32 HibRATs* + 12 references identified from gray literature search and country consultation process 1331 duplicate citations 3,285 met criteria for full text review 54 unable to obtain full text 1,899 undergo full text review 1,000 screened out for lack of applicable data 216 fail to meet primary outcome criteria, partial data abstracted 899 enter data abstraction process 352 excluded for meeting at least one exclusion criteria *Hib rapid assessment test. 332 articles (336 studies) meet criteria for at least one primary outcome & enter analysis database. 15

17 Figure 2. Schematic of deriving country-specific Hib or Spn meningitis parameter estimates from country, regional or global data. Representative data available for country? N Y 1. Country Estimate Employ metaanalysis of country data as point estimate Data available for same subregion, N Y 2. Subregion x Mortality Estimate Employ meta-analysis of all studies in subregion, mortality Data available for same midregion, N Data available for same region, N Y Y 3. Midregion x Mortality Estimate Employ meta-analysis of all studies in mid region, mortality 5. Derived estimate: For incidence: Global ratios of meta-analysis estimates by mortality strata were calculated. Appropriate ratio applied to the nearest available region, mortality stratum within the same region For CFR : Global mortality strata 4. Region x Mortality Estimate Employ meta-analysis of all studies in region, 16

18 Figure 3. Evaluation of the indirect effect of Hib vaccine through the relationship between the proportion of children immunized with Hib vaccine and the percent reduction in Hib cases. [Observed data are shown in X (two studies overlap); solid line is fitted equation; and dashed lines are 95% prediction intervals; Hib3 coverage : percent of eligible children who received 3 doses of Hib conjugate vaccine; SE for fitted equation is.035 (intercept) and.058 (ln(x)]. % Reduction in U5 Cases 100% 90% 80% 70% 60% 50% 40% 30% y = Ln(x) % R 2 = % 0% 10% 30% 50% 70% 90% Hib3 Coverage 17

19 Appendix Equations 1: Basic Structure of the Model Despite the different approaches used to actually derive the various rates, as described in the main body of the paper, ultimately, we have for each country a set of incidence and case-fatality rates, by syndrome (pneumonia, meningitis, and NPNM). Appendix Table (below) Appendix Table. Source of data for model equations Shaded cells represent parameters derived from estimates (non-shaded cells) Cases Deaths Pneumonia Meningitis NPNM (A) Derived from metaanalysis Population at risk x of vaccine efficacy incidence rate studies and applied to estimates of clinical pneumonia incidence among children 0-59 months of age (B) Derived from metaanalysis of vaccine efficacy studies and applied to estimates of pneumonia mortality in children 1-59 months of age Cases x CFR Incidence Rate Cases/Population at risk* (C) Meta-analysis of published studies adjusted for HIV prevalence Case Fatality Ratio Deaths / Cases (in presence of vaccination) ** (D) Meta-analysis of published studies adjusted for access to care (E) Meta-analysis of ratio of NPNM: Meningitis x Meningitis Cases Cases x CFR Cases/Population at risk* (F) Meta-analysis of ratio of NPNM: Meningitis CFR x Meningitis CFR * under-5 population not protected directly or indirectly by Hib vaccination ** because the numerator is deaths in children 1-59 months, and the denominator cases in children 0-59 months, this will be an underestimate. Using the derived parameters from the table, to obtain estimates of cases and deaths, these are generally combined as follows (this can also be applied to obtain estimates of mortality for years other than the year 2000). Define the following variables: Variable Description Pop < 5 Population under 5 in the target year HIV 2000 HIV prevalence (as a %) in children under 5 in the year 2000 HIV Year HIV prevalence (as a %) in children under 5 in the target year RR HIV Relative Risk of Hib (7.39) or Spn (40.96) in HIV+ populations Effective Coverage The proportion of children under 5 who are protected directly or indirectly by vaccination 18

20 Then point estimates can be obtained as follows, given the incidence and CFR rates. Incidence Cases (HIV-) Year = Pop< 5 (1 Effective Coverage) ( 1 HIVYear ) 1 + HIV2000[ RRHIV 1] Incidence rate in HIV- population RRHIV Incidence Cases (HIV+) Year = Pop< 5 (1 Effective Coverage) HIVYear 1 + HIV2000[ RRHIV 1] Cases = Cases (HIV-) + Cases (HIV+) Year Year Year Deaths (HIV-) = Cases(HIV-) CFR Year Deaths (HIV+) = Cases(HIV+) CFR Deaths Year Year = Year Year Deaths (HIV-) Year + Deaths (HIV+) Year Incidence rate in HIV+population To get rough approximations of an upper or a lower bound, the upper and lower bounds of each parameter in the above equations can be appropriately included. Specifically, however, each set of parameters, where derived as part of the analysis described in this paper, is given below: (A) Pneumonia Cases Define the following variables: Variable Description Pop 2000 Population in the year 2000 HIV 2000 HIV prevalence (as a %) in children under 5 in the year 2000 RR HIV Relative Risk of Hib (7.39) or Spn (40.96) in HIV+ populations Effective Coverage The proportion of children under 5 who are protected directly or indirectly by vaccination ARI Incidence Overall population incidence of ARI in the year 2000 P ˆCases, Disease 2000 Meta-analysis estimate of the proportion of cases attributable to the disease (either Hib or SP) Then the overall number of cases of either Hib or SP can be calculated as: Cases = ˆ P (1 Effective Coverage) 1 + HIV ( RR 1 ARI Incidence Pop ( ) Disease, Year Cases, Disease 2000 HIV This was done separately for children aged 0-11 and months of age. 19

21 (B) Pneumonia Deaths Define the following variables: Variable Description Pop 2000 Population in the year 2000 HIV 2000 HIV prevalence (as a %) in children under 5 in the year 2000 RR HIV Relative Risk of Hib (7.39) or Spn (40.96) in HIV+ populations Effective Coverage The proportion of children under 5 who are protected directly or indirectly by vaccination ARI Deaths The derived number of ARI deaths in the year 2000 among HIV-negative P ˆCases, Disease 2000,HIV children in the absence of vaccination. See Appendix Equation 4 Meta-analysis estimate of the proportion of deaths attributable to the disease (either Hib or SP) Then the overall number of deaths of either Hib or SP can be calculated, separately for those with and without HIV, as follows: Deaths, Deaths Disease, Year HIV, = P ˆCases Disease ( 1 EffectiveCoverage) (HIV2000RRHIV ) DeathsDisease,Year,HIV (1 HIV ) Disease,Year,HIV + = This was done separately for children aged 1-11 and months of age 2000 ARI Deaths 2000,HIV (C) Meningitis Cases Define the following variables: Variable Description Pop 2000 Population in the year 2000 HIV 2000 HIV prevalence (as a %) in children under 5 in the year 2000 RR HIV Relative Risk of Hib (7.39) or Spn (40.96) in HIV+ populations Effective Coverage The proportion of children under 5 who are protected directly or indirectly by vaccination Incidence Hot-deck meta-analysis estimate of the meningitis incidence in the HIVpopulation, adjusted for country-specific HIV prevalence. See Figure 2 ( ) Cases (HIV-) Pop (1 Effective Coverage) 1 HIV = HIV2000[ RRHIV 1] Cases (HIV+) Pop (1 Effective Coverage) HIV Cases HIV = HIV2000[ RRHIV 1] = Cases (HIV-) + Cases (HIV+) RR Incidence Incidence This was done separately for children aged 0-11 and months of age. 20

22 (D) Meningitis Deaths Using the Cases derived in (C), further define the following variables: Variable CFR Description Hot-deck meta-analysis estimate of the meningitis CFR (assumed to be the same for HIV+ and HIV-). See Figure 2 Deaths (HIV-) = Cases(HIV-) CFR Deaths (HIV+) = Cases(HIV+) CFR Deaths = Deaths (HIV-) + Deaths (HIV+) (E) and (F) Non-Pneumonia, Non-Meningitis Cases and Deaths Using the estimates of Cases and Deaths for meningitis for (C) and (D) above, define the following variables: Variable Description NPNM Multiplier Cases Meta-analysis estimate of the multiplier of NPNM cases to Severity Mortality Strata meningitis cases for each mortality strata, and in the case of SP, separately for severe and non-severe cases. See Table 6 NPNM Multiplier CFR Meta-analysis estimate of the multiplier of NPNM CFR to the meningitis CFR for either all Hib cases, or severe SP cases. See Table 6 Then case estimates are calculated by mortality strata as follows, using the country-specific estimates of meningitis cases. NPNM Cases (Hib) = Meningitis Cases(Hib) NPNM Multiplier Cases Mortality Strata NPNM Cases (SP) = Meningitis Cases(SP) NPNM Multiplier Cases Mortality Strata Severe Hib Mortality Strata Severe SP Mortality Strata + Meningitis Cases(SP) NPNM Multiplier Cases Non Severe SP Mortality Strata Country-specific estimates of meningitis CFR, as applied in (D), are then applied for deriving deaths: NPNM Deaths (Hib) NPNM Deaths (SP) = = Meningitis Cases(Hib) NPNM Multiplier Cases Meningitis CFR(Hib) NPNM Multiplier CFR Meningitis Cases(SP) NPNM Multiplier Cases Meningitis CFR(SP) NPNM Multiplier CFRSP Severe Hib Mortality Strata Hib Severe SP Mortality Strata These equations are applied separately to the HIV + and HIV - estimates of meningitis cases in order to get separate estimates of NPNM cases and deaths in the HIV+ and HIV- populations. 21