Appendix 23 On application of methods for improving confidence in the outcome of surveillance for disease freedom Tom Murray 175
176
177
The Stochastic Scenario Tree Approach to Combining Multiple Sources of Evidence to Demonstrate Freedom from FMD (Information Paper) Tom Murray, Associate Professional Officer, FAO-EUFMD Commission The following paper is principally based on workshops attended by the Author in February 2006 and February 2007 on the use of complex sources of evidence to demonstrate freedom from disease. Introduction The OIE Terrestrial Animal Health Code provides guidelines for demonstration of disease freedom. Specific measures for Foot-and-Mouth Disease are described in Chapter 2.2.10 and surveillance guidelines in appendix 3.8.7. Appendix 3.8.1.5 provides for the use of multiple sources of evidence to demonstrate disease freedom. Paragraph 4 states that The methodology used to combine the evidence from multiple data sources should be scientifically valid, and fully documented including references to published material 1 Traditionally, national disease status has been determined using structured cross-sectional surveys, which are generally difficult and expensive to implement. On-going surveillance may also be assessed by expert panels, but there are no accepted methods for quantifying either confidence in the surveillance process, or the probability of national disease freedom demonstrated thereby. 2 Demonstration of disease freedom presents extra difficulties in regions with inadequate technology or resources to apply conventional surveillance systems and in vaccinated populations where design prevalence would be quite low, clinical signs may be less evident and serological tests must be able to distinguish between the vaccinated and exposed animal. Further, the confidence in the disease freedom will depend on the likelihood that incursions of infection occur without detection by the surveillance system, for example when the area has land borders or is in close proximity to an endemic area. Provision is made in EU Council Directive 2003/85/EC for the use of vaccination in the face of any future outbreaks of FMD in European countries. Non Structural Protein (NSP) tests have been developed to distinguish the vaccinated animal from the exposed animal and desktop simulation exercises/workshops are currently being conducted to consider the implications associated with the use of these tests to demonstrate FMD freedom after eradication of FMD, when vaccination has been used as a control measure. An important topic discussed at the first of these exercises was the question of how to deal with small numbers of test positive animals. It was concluded at this workshop that these positives should be dealt with in the context of the overall surveillance system, including epidemiological, clinical and serological examinations. The scenario tree approach is a means of combining various sources of evidence in a quantitative manner and could prove useful when dealing with small numbers of antibody test positive animals. This concept was originally developed in relation to regions where limitations in technology or communications may inhibit the use of conventional surveillance systems and where multiple sources of data which are already available would provide more compelling evidence of disease freedom. Outline of how quantitative methods are used to combine surveillance system components into one overall surveillance system and its evaluation The aim of the system is to combine the various surveillance system components, including passive/routine surveillance and active surveys; for example random serosurveillance, non 1 OIE. Terrestrial Animal Health Code. 2006. http://www.oie.int/eng/normes/mcode/a_summry.htm 2 Cameron A, et al. Demonstrating freedom from disease using multiple complex data sources. A proposed standardised methodology and case study. Open Session of the EUFMD Standing Technical Committee. Bern. 16-19 September 2003. 178
comprehensive serosurveillance 3 compulsory notification, veterinary farm visits, abattoir inspections and other data sources (Figure 1), to produce a quantitative value for our confidence in disease freedom given that none of the surveillance system components detected disease. This can also be described as the negative predictive value of the overall system. The probability of a new incursion of disease, within a given time period, is also incorporated into the model and variations in relative risks between different regions and animal groups are accounted for. Figure 1: Combining Surveillance System Components (SSC) into a Surveillance System Surveillance System SSC1 Random Serosurveillance SSC2 Veterinary inspections SSC3 Compulsory Notification SSC 4... SSC5... Bayesian inference is used to combine the prior probability of disease/circulation and survey results to calculate the posterior probability of disease/circulation. The original prior probability may be based on risk assessments, expert opinion any other relevant data. To combine surveillance system components, where there is a lack of independence between surveillance system components, the posterior probability of disease calculated from the first surveillance system component is used as the prior probability for the next surveillance system component and so on. Where there is complete independence between surveillance system components then the original prior is used. Further calculations are required for dealing with varying levels of independence. For the following surveillance programme the prior probability can be estimated using the posterior probability from the previous survey and the likelihood of a new incursion over the relevant time period. Many of the inputs and outputs of these models are in the form of distributions (stochastic) rather than point data, as this more accurately reflects the variability and uncertainty associated with most biological data. A more detailed guide to the development of a scenario tree for an individual surveillance system component, the combination of surveillance system components and how the probability of disease incursion is accounted for are available at the Ausvet website: http://www.ausvet.com.au/freedom Situations where this work can be relevant to the work of the EUFMD Commission Substantiation of FMD freedom in vaccinated populations and how to deal with small numbers of positive results to serosurveillance surveys are currently major topics of discussion within Europe. The quantitative use of other sources of evidence combined with serosurveillance could assist in finding a solution to this problem which is repeatable and which allows comparison between different surveillance systems used in different regions. This work can be used to support disease surveillance in countries bordering Europe. Countries which are vaccinating to control disease will eventually want to declare freedom with vaccination, followed by a declaration of disease freedom without vaccination one year after vaccination programmes cease. The use of the scenario tree approach can identify other sources of data which would, if combined with serosurveillance, improve our overall confidence in disease freedom. The increased risk associated with the movement of animals during festivals such as the Kurban festival can be factored into the work. The Thrace region of Western Turkey is an area of particular importance to the EUFMD Commission, as it acts as a buffer zone between non vaccinating countries in South-East Europe and Eastern Turkey (Anatolia). In 2007 there have been sporadic outbreaks of FMD Serotype A and Serotype O in Thrace. Efforts are been made through vaccination and biosecurity measures to once again eradicate disease in this area. When eradication is achieved it will then be important that best use is made of all available sources of data used in demonstration of disease freedom and 3 Non comprehensive serosurveillance refers to any serosurveillance which is not random and therefore not fully representative of the population 179
that the overall surveillance system would ensure early detection should a new outbreak occur. Proposed technique for EUFMD Secretariat to implement the scenario tree approach to demonstrate disease freedom and to early detection of circulating virus The process should be an iterative and progressive activity where uncertainty associated with data is reduced with each survey (Figure 2). A workshop would be organised where the scenario tree would be constructed. A situation of major interest would be the Thrace region of Turkey, or possibly adjoining areas of Greece and Bulgaria. The participants would comprise of individuals with experience of the epidemiology of FMD in the specific region, knowledge of other sources of evidence which could be used to demonstrate freedom from FMD circulation and knowledge of the use of the scenario tree approach to demonstrate disease freedom. These individuals would include (for example) official veterinarians from Turkey, consultants hired by the EUFMD Commission in Thrace in recent years and members of the EUFMD research group. Figure 2: Iterative process for development of surveillance programme based on the stochastic scenario tree approach Evaluation of Results Evaluation of Surveillance System in Place Implementation of Surveillance Programme Identification of other data sources and strategies to reduce uncertainty Design of Surveillance Programme Modeling and Evaluation of other data sources in Scenario Tree 180
The aim of the process is to produce a structured and coordinated design for a most effective surveillance system for demonstration of disease freedom that incorporates the experience of recent surveys, technologies available and the experience of experts in the region. 1. Workshop a. The Negative Predictive Value (NPV) of the current surveillance systems in place should be evaluated b. Other sources of data available to include in the surveillance system are identified and ranked. Ideally these sources of data are already in place and the objective here is to quantify their value within an overall surveillance system. c. Rough trials are carried out of the other sources of evidence using the scenario tree approach. Risk assessments, literature and expert opinion can be used to estimate values of parameters inserted into the scenario tree. The inputs and outputs are often in the form of distributions (stochastic) rather than point values, as this is more reflective of the uncertainty and variability associated with biological processes. d. The other sources of data are re-ranked based on the sensitivity of the overall outputs to the various inputs and the degree of uncertainty associated with each input. Including data sources with the minimum uncertainty associated with them and reducing the uncertainty of the most significant data sources, helps to improve the overall model. e. Decisions made on what sources of evidence should be included in the overall model and proposals for reducing uncertainty associated with these inputs. 2. Carry out surveillance for the time period in question using surveillance system components decided on during the workshop. 3. When all results are in and given negative results, calculate the negative predictive value of the overall surveillance system and the probability of disease given the negative surveillance results 4. The overall surveillance system should be analysed to identify how the next survey can be improved and how uncertainty associated with various inputs can be reduced. 5. The posterior probability of disease is combined with the probability of re-entry of disease into the region to produce an estimate of the prior probability of disease for the next survey Conclusion The scenario tree approach can be used to gather all available sources of evidence in a coordinated approach to demonstrate disease freedom. This quantitative approach provides for a repeatability and comparability between different surveillance systems used for different regions and in different time periods. It can use expert opinion, risk assessments, current and historic data. The aim is to make best use of all sources of evidence already available and helps to provide a framework to bring together the knowledge and experience of experts in a coordinated and logical fashion. The time period of surveillance, the likelihood of new incursions of disease and the relative risks of disease between different strata (regions or animal groups) are all factored into this approach. References: OIE. Terrestrial Animal Health Code. 2006. http://www.oie.int/eng/normes/mcode/a_summry.htm Cameron A. Barford K. Martin T. Greiner M. Sergeant E. and other members of the International EpiLab research team on Disease Freedom. Demonstrating freedom from disease using multiple complex data sources. A proposed standardised methodology and case study. Open Session of the EUFMD Standing Technical Committee. Bern. 16-19 September 2003. EPIZONE, EUFMD (FAO) & EU Coordination Action FMD-CSF. Workshop on the design and interpretation of post Foot-and-Mouth Disease vaccination serosurveillance by NSP tests Part I (dense cattle-pig countries). January 23-25, 2007 181
Willeberg P. (2003) Note on Animal Health Surveillance. Impressions from an OIE Technical Consultation in Ft Collins, Colorado 17-21 March 2003. Appendix14. pp 148-151. Thirty-fifth General Session of the European Commission for the Control of Foot-and-Mouth Disease http://www.ausvet.com.au/freedom/ COUNCIL DIRECTIVE 2003/85/EC of 29 September 2003 on Community measures for the control of foot-and-mouth disease. Acknowledgements The methodology for the analysis of complex surveillance systems using stochastic scenario tree modelling was developed principally by Angus Cameron, Tony Martin, Jenny Hutchison, Evan Sergeant and Nigel Perkins. Dr Keith Sumption, EUFMD Commission. 182