Good afternoon. It is fortuitous that I am the final presenter at

Transcription

1 [Session: Modeling and Risk Assessment] A Predictive Modeling and Decision-Making Tool to Facilitate Government and Industry Response to an Intentional Contamination of the Food Supply DR. ANDREW JAINE BT SAFETY, LLC Good afternoon. It is fortuitous that I am the final presenter at this session, as the presentations started off with Col. Hoffman describing a method for evaluates the risk associated with a food contamination incident as the product of three components: the threat to the food target, the vulnerability of the food, and the consequences of any ensuing incident. The previous speakers addressed various aspects of the threat and vulnerability of the target, and in this presentation I will describe a system for evaluating and managing the last of these three components: the consequences of food contamination incidents. This system is a predictive model designed to help government and industry improve their ability to respond to food contamination events. We started developing it about 4 years ago, soon after 9/11, when one of my partners in BTSafety became concerned about how a bioterrorism attack on the food supply could affect the several produce companies that he owns. After looking into the available approaches to reduce the vulnerability of these companies to this type of risk, he decided that no effective approaches were available and so we started designing an approach. Fairly early in that process we realized that we needed some detailed information about the agents that are the most likely to be involved in a food event, so we approached some areas of the federal government for this information. Those initial conversations led to them becoming interested in what we were doing, and which eventually led us to start working with them to develop a system based on those designs. Over the last 4 y this development has grown considerably; now there are a wide range of people from different backgrounds involved: from the scientific community, the Food and Drug Administration, Homeland Security (through the National Center for Food Protection and Defense ), and also other agencies like the Centers for Disease Control, the USDA, from industry, and many others. And much of the work is now funded in part by the FDA and in part by Homeland Security though the National Center for Food Protection and Defense, although some is still privately funded. As I said, the intent of the system is to enable those involved in responding to food contamination events to understand the potential consequences of these types of incidents and then to manage and improve the speed and effectiveness of their responses. The main thing that differentiates our system from other models is that it puts all the factors that influence the consequences of a food event together into a single model that models the entire evolution of a food event, starting at the point of food production (for example the farm) and following it through to the conclusion and an evaluation of the ultimate consequences for example, the impact that it has on all of the various affected stakeholders, like consumers, the Public Health System, industry, and so on. When we started to study the consequences of food contamination incidents we realized fairly quickly that the magnitude and type of consequences of an incident may vary greatly depending on many of the specific incident characteristics, such as the type of food product, the nature of the contaminating agent, how much product is contaminated, and so on. For example, the impact of an E. coli O157:H7 contamination of lettuce may well be very different from the impact of an E. coli O157:H7 contamination of ground beef. So to get the required accuracy in each different event scenario we cannot develop a single model that accommodates all foods and all agents, but instead have to model each combination of agent and food in which we are interested separately. So our model must be very focused and specific. We are not trying to provide a model for every possible food event that may break out; we are trying to answer specific, well defined questions. But there are many such questions, because there are many foods of interest produce, seafood, meat products, etc., and also many agents of interest, E. coli O157:H7, Clostridium botulinum toxin, Salmonella, Enterobacter sakazakii, and so on. So there are an enormous number of possible combinations. And to develop unique models for even a small subset of these would be prohibitively time-consuming and expensive. To overcome this we are developing a generalized system that will accommodate all types of event involving any types of agent and food. We do this by decomposing each event into the entire set of distinct phases that have a significant affect on the event s consequences, and then decompose each phase into its distinct characteristics. It is probably easiest to explain this by drawing a parallel with building a house. There are, of course, an enormous number of different types of houses, with different appearances, contents, prices, etc. However, almost all houses share the same basic parts. They all have bathrooms, bedrooms, closets, living rooms, and so on. And, while many of these rooms may look quite different, the things that go into each room are largely the same. For example: all rooms have floors, walls, and so on. There are some things that are specific to any given type of room, for example, bathrooms have faucets, baths, mirrors, etc., whereas dining rooms don t. But even then most of the items that are unique to a specific type of room are the same for all rooms of that type. So, if you were a house builder, how would these similarities affect the way you would build a new house? First, and most importantly, you must decide on the objectives for the house. Do you want to build single-family houses which would be Proceedings of the Institute of Food Technologists First Annual Food Protection and Defense Conference

2 A predictive modeling and decision-making tool... appropriate for many different types of family, or specific, custom homes, for example a home for a person who loves open spaces and wants the whole house to be one big room? The way that you approach your project will be quite different depending on these choices. And you have very similar considerations when you start to build a computerized modeling system; you have to accurately define your objectives. So when we started building this predictive modeling system, which we call the Consequence Management System (CMS), the first thing that we did was to establish a very specific set of objectives for it. The most important of these objectives is that the CMS must be able to accurately predict the ultimate consequences of any type of food contamination incident, including the effects of any interventions, and to present those predictions in a form that is easily understood by the average user. So our objective, to use our home construction analogy, was to build a model that is the equivalent of the multi-purpose, single family home ; not the equivalent of a custom home that would only meet the needs of one buyer. And, continuing with this analogy, when you have established the objectives for the home, then the next step is to develop a detailed architectural plan for the construction of the home that will ensure that the development will meet your defined objectives and that all of the components will integrate seamlessly into the whole. By developing the plans first you can get an accurate estimate of the tools and building materials you will need, and also, if you are planning to build several houses, or a subdivision, then you can get additional benefits and save a lot of money by using this planning process to find the ways in which the homes that you want to build are similar, and then capitalizing on these similarities by mass-purchasing or mass-producing those modules. This will enable you to get economies of scale while still making sure that each home is unique in the ways that are most important to the buyer: color, number of rooms, size of the dining room, décor, or whatever. Similarly, after defining our objectives for the CMS we then designed an overall architectural plan for how to develop it to meet those needs. And because we wanted to be able to model all types of event, we created an architecture that enables us to get all the possible economies of scale by taking advantage of the similarities between events. We do this by decomposing all different types of food event into a set of phases, where each phase represents a step in the evolution of the event that has a significant impact on the consequences; and then integrating all these phases together into a single model of the entire event. We have currently identified fourteen such phases, for example: food sourcing, food distribution; agent characteristics; agent-food interaction; agent dose response, and so on, however this list is still increasing. Back to our home construction analogy: when you have finished your plans and you know exactly the design for the home or homes that you want to build, then the next thing that you have to do is collect together the resources that you need for the construction. To build a home the resources are a crew and the right set of tools. Some of these resources are for specific purposes, for example if you are going to have tile floors then you need a tiler and a set of tiling tools, to install the plumbing you need a plumber and a set of plumbing tools; etc. And then you need general resources, like laborers and general use tools like hammers and screwdrivers. However, while you may need many different types of resource, the resources required for each room of the house, and for each different house, generally remain the same. For example, due to aesthetic or other differences you may use a different type of tile in different rooms, but the tiling resources will apply all the tile, regardless of type. Similarly the plumbing resources are used for all the plumbing in the house; and the general use resources are used throughout the house. So you do not need to put together different sets of resources for different rooms, or even for different houses. Instead, if you get the right resources to start with you can use them over and over for different rooms in the same house and for other, houses. So if you are going to build a subdivision you get really good resources, as they will last. In our case, as I said earlier, we decompose food events into phases, which in our analogy loosely correspond to the various rooms of a house. And in the same way that the various rooms have different sets of components, most of which can be built with similar resources, so, similarly, the various phases of an event can have different sets of characteristics, but most of these characteristics can be modeled with similar resources. So, as a part of our planning for CMS we identified all the different types of model construction resources that we need to build the various event phases, and, like with our house construction, sometimes unique resources are required to model a specific characteristic of a phase, but in most cases the resources are applicable across all types of phases. We then built these resources into CMS. So, the CMS is like a collection of resources that enable you to model the various phases of food events, some of these enable you to model how a food is sourced, others enable you to model how it is distributed; others help you model agents, how an agent interacts with a human to make them ill, how the Public Health System will respond when an ill person enters the system, and so on. But I expect that by now you are thinking that this sounds a very complicated system, and this is true of some of the internal workings of CMS. It has to be, because the internal workings of some phases of the evolution of a food outbreak are pretty complicated. However, you may recall that I said that one of our fundamental objectives is to be able to present the results in a form that are easily understood by the average user. So, while we work very hard to make sure that that the internal workings of the system are a science-based, accurate reflection of the real life evolution, we try equally hard to take all of that complexity and implement it in an interface that shows the effects of the evolving simulation to the user in an impactful, real-time, and visual way. Those of you who were at Dr. Offutt s presentation this morning may recall that she said you can put up lots and lots of data, but if you give them a map to look at, they like it and understand it. We agree with that completely. So, one of the ways in which we show the results of our simulations is as a map of the United States that changes in real time to show how the simulated event progresses geographically over time. And to show the temporal progress of various stages of the event, the CMS output also has a set of bar charts that depict the number of cases that are present at each stage of the event at each point in the simulation. To implement all of the required complexity in a system that runs in real time requires a huge amount of flexibility from our modeling environment, and unfortunately none of the existing off-the-shelf modeling toolkits provides that amount of flexibility, so we are developing CMS in original program code. In this way we can ensure that can support all types of event, from small, unintentional events like the illness that results when a bird flies over a field and deposits some E-coli O157:H7 on several heads of lettuce, up to large, intentional events, like one that would occur if bioterrorists crop dusted a field with some nasty agent. And CMS can model any number of different phases for each event, and any of those phases can be modeled using any of a wide range of possible characteristics, each of which is available to any phase. And because CMS is written in original program code, when we run into a unique characteristic that we have not seen before we can build it into the CMS ourselves. And we expected this to happen frequently, in part because we are constantly moving on to new foods and agents that we make new demands, but also because the current knowledge of many existing bioterrorism agents is still changing rapidly, which often requires new capabilities. Because we expected this, we have structured CMS specifically to make sure that it is easy to incorporate these types of change. So, with this design any new scenario that does not involve a totally different type of characteristic can be modeled by simply adding Proceedings of the Institute of Food Technologists First Annual Food Protection and Defense Conference

3 A predictive modeling and decision-making tool... the new data to the system, and any that requires new characteristics can be modeled by adding the required new characteristics to CMS. Finally, to go back to our home construction analogy one last time, when you have set the objectives, developed the plans, put together the required development resources and you are ready to start actually building the house, then you need the building materials. And in most cases the purchaser has very specific likes and dislikes about the color, size, texture, etc. that they want, so many of the materials must be selected specifically for each house. But again, if you are building a subdivision, you can save a lot of cost and time by giving the purchaser some alternatives to select from, and then buying the materials in bulk. Similarly, our building materials are the data that we collect about each agent, product, etc. And we find that, to meet our objective of getting the sort of accuracy that we need in the predictions of the ultimate consequences of each incident, we must model each agent/food combination individually. So, while we decompose all events into a set of similar phases, we then have to collect the data and models to inform each of these phases separately for each combination of agent and product. But in many cases we can get economies of scale, because the same food manufacturer that makes food X also makes food Y, so when we go in to talk to them, we can collect at the same time the information for all the products that they produce that are of interest to us. But accurately modeling a food incident often requires a lot of data. For example, for each selected food we must collect detailed data to build a model of the entire movement of the food through its distribution chain, both geographically and temporally, from production to consumption, that is, where does the food go and how long does it take to get there? These data must be very specific. So take, for example, lettuce sourcing. Lettuce is, of course, a seasonal product. So if you buy your lettuce in summer it will probably come from Salinas, California. If you buy it in the fall it could still come from Salinas, but also could come from Huron, California. However, if you buy it in the winter it will probably come from Yuma, Arizona because Yuma has the largest winter production. So we must collect data from all those sources. And a similar level of data is required about each of the stages of the distribution system, such as the movement of the product from the field to the retail store, the characteristics of how it is received and handled in the back room of the store before it comes onto the shelf, how long it takes the consumer to go in and buy it and take it home, and so on. Next we need consumption information, and this includes items such as how long it takes the consumer to eat the product after they take it home, how much of the product they eat over what period of time, how that consumption varies with demographic profile, and so on. Next, a similar level of data collection is required for each agent that we need to support, such as how consumers in each demographic segment would respond to consumption of various levels of the agent, what levels of contamination of the agent in the food could reasonably be encountered, and so on. Then we need to collect data to inform models of how many of the consumers of the contaminated product would get ill, how ill they would get, and what they would do about that illness, for example, would they seek medical attention? Then we need to collect data to inform models of how the various patients would interact with the Public Health system, by modeling the range of likely actions on the part of the medical system (which of course depends on the type of contaminating agent, the extent of the contamination, and so on). We call this phase Public Health Response and it is one of the areas in which we are investing a considerable amount of time. Once an event happens and people around the country start getting ill; it describes how long we would expect it to take for our Public Health System to receive reports of the various illnesses, then to pick out the related illnesses from the noise of all the other illnesses that are occurring at the same time, so that they can relate these various illnesses together and identify the outbreak, then to get the information and the authority they need to decide on and initiate appropriate response. In this area we are working very closely with a group from the National Center for Food Protection and Defense led by Dr. Don Schaffner to build a model of that entire process to be integrated with the CMS. And so we collect data and build models to follow the event through all of its phases until we reach its ultimate conclusion, and at that point we model the impact of the event on consumers, the Public Health system, and the economic impact on industry and the country, etc. However, hopefully, before that ultimate conclusion, somewhere along this evolution the information about the unfolding event will stimulate someone to initiate some mitigating intervention. For example, the Public Health System receives enough reports to enable them to recognize that an outbreak is actually happening and they decide to intervene. Or affected consumers call an 800-number hotline to the company that produced the product and the company recognizes the outbreak and decides to intervene, and so on. So, we also model this aspect of the event by collecting data to build models of the processes that lead up to the recognition of the outbreak, the various intervention alternatives that are available at each point in the simulation, and the probable effectiveness of each type of intervention. And, when we have collected together all these data and models, we use the resources built into the CMS to build the data into a comprehensive model of a wide range of various scenarios for potential events, for example events with the same agent and product but with different points of contamination in the food distribution chain, or with various levels of contamination, different quantities of contaminated food, different demographic subgroups, etc. So our development program has a very intensive set of data collection activities, and in those activities we try very hard to collect real hard data to inform the model wherever possible. Some phases of the event, for some foods and agents, are very rich in data. For example, we have data that we have sourced from lettuce industry on every shipment of lettuce made during the last four years by several lettuce companies, so the food distribution for that product can be modeled to a high degree of accuracy using these data. However, other phases, such as the characteristics of the response of the Public Health system, are less rich in data, so the existing data must be supplemented with statistical models. In other cases, like the dose/response curve for several potential bioterrorism agents, the data pathway is very sparse, so these parts of the CMS must be informed almost entirely by computerized models. So our data collection process is sometimes going out to various data sources and obtaining the available data, in other cases it means going out and collecting existing statistical models, and in yet other cases, where neither data nor models exist, we must build new statistical models. Occasionally during this data collection process we find that there are several different data sources for a particular characteristic of a phase, and in some cases these different data are in conflict, with some people believing one data set, and others believing other data sets. In these situations our job, we believe, is not to select between those data and models, but to provide a system that will accommodate all of those variations and let the user select which variant they prefer, and to change between variants, and even, if they prefer, to put in their own model. To achieve that, the CMS supports different, user selectable alternative models for most phases, and comes with a data entry system and a complete statistical modeling subsystem that enables experienced users to even write their own formulae for complex statistical distributions. So, there are large amounts of data and models to collect and build. But we are not trying to do this collection alone; we are working cooperatively through relationships with a large number of external entities. We are receiving considerable help from a number of groups at the National Center for Food Protection and Defense and from FDA, CDC Proceedings of the Institute of Food Technologists First Annual Food Protection and Defense Conference

4 A predictive modeling and decision-making tool... (who are holding expert elicitations at our request on some of the agents that we re working on), USDA, ERS, a large number of industry partners, and the scientific community. Plus, we are of course filling in the gaps with our own teams of experts in the various fields. And all of this is leading to CMS becoming a very data rich environment. So, you may wonder what a user sees in an actual simulation run with the CMS. Well, to start a simulation using the CMS, the user must first select the specific characteristics of the outbreak to be simulated, such as the food and agent involved, the point at which the product is contaminated, the quantity of product contaminated, the level of contamination, and so on. CMS helps in this selection by suggesting reasonable values. For example, if you select the point of contamination as the field, then it will suggest the quantity of product based on the average field size, but the user can then change that selection at will. As the simulation runs it clicks off the hours since the simulated contamination happened on an elapsed time counter, and at each of those simulated hours it shows the output in three distinct displays: The first display is a map of the United States that shows the geographic location of the product as it moves through the distribution channel. As the event moves through the different phases at a specific geographic location it shows the progress by displaying a symbol at the location that is colored to indicate the current state of the event there. So, for example, when the product first arrives at a retail store at the new location a colored symbol appears on the map at that location. As the food at that location then moves to the home the color changes, and the color continues to change to show when it first gets consumed, then when the first consumer at that location gets ill, and so on. The second display is a bar chart that shows the quantity of product at each stage of distribution at the specific point in time in the simulation. So, for example, bars on the chart represent the quantity of product that remains in the field, on the truck, at retail, and so on. Following consumption, the bars indicate the quantity of product consumed, the number of resulting illnesses, and so on. Then subsequent bars show the number of people who have received medical attention through the Public Health System, and mortality if that occurs. The third display mechanism is a set of textual displays that show the current state of epidemiologic knowledge surrounding the event and the current evaluation of the event impact. The first of these text displays shows each significant change in the known epidemiologic information about the event, such as when the first illness occurs, when the first report is made to the PHS, when the PHS would identify the agent, and so on. The second text display shows the system s evaluation of the total impact of the event at the current time using the user selected impact metric. The system will support a range of different impact metrics, including Quality Adjusted Life Years (the impact on the quality of life of each of the patients with distinct forms of disease presentation), the economic impact on various sectors of the economy, and so on, and the user can select which impact metric they prefer to see. But part of the power of the system is that, in addition to showing the consequences of the event itself, it also demonstrates the effects of implementing mitigating interventions, and the effects changes in the timing of those interventions. At any time during the simulation the user can click a button to indicate that they want to invoke an intervention, and in response CMS will display a list of the interventions that are feasible at that point in time. The feasible interventions are selected by CMS based on what epidemiologic information is known at the time the user requests the intervention. So if, for example, the user initiates an intervention prior to the time when the epidemiologic models indicate that the outbreak would be recognized by the Public Health System, then the system indicates that the only interventions that are available are those that would be appropriate if you have received some advance warning of a pending or ongoing outbreak (for example a phone-in threat). If that is not the case then the user can click a button to allow the simulation to proceed. If they initiate an intervention at a later time, for example when the epidemiologic models indicate that the outbreak would be recognized, and the agent identified, but not the product, then it will show a list of the interventions that are appropriate to that specific level of epidemiologic knowledge, and so on throughout the entire life-cycle of the outbreak. And so, throughout the entire event evolution, every hour the CMS shows graphically how many locations it estimates are involved; how widely they are geographically dispersed; the quantities of product or people at each stage of distribution, consumption or morbidity; and a summary of the total impact of the event to that point, in the user s chosen impact metric. And based on what they see, the user can select to invoke any interventions that may be available, and if they do then the evolution of the event is changed, to reflect the changes that would result from that intervention, and the impact would also change accordingly. So, CMS allows the user to assess the likely consequences of a food contamination event, and also to assess how changing the parameters of various phases of the event, for example by increasing the level of contamination of the agent, or by intervening in the event by making a public announcement, would impact the entire consequence of the event. So, how can this information be used? Well, I think Col. Hoffman articulated the answer to that far better than I could when he said that in order to evaluate the risk associated with a specific food event you must be able to evaluate the consequences of the event, and that is what CMS helps you do. So, the consequence evaluations that it provides are useful in advance of any event for planning, to evaluate what product/agent combinations are of greatest potential impact, and how we would expect to see those events evolve. But the CMS can also be useful during an event, because they give an understanding of how fast products move, how fast events evolve, and how quickly we have to intervene for the intervention to be effective, and so on. And, another obvious use of the CMS is in training in food safety and food defense, for table-top exercises and other similar activities. In conclusion, I have described the process that we are using to build the CMS, and that the process requires extensive cooperation from many sectors of government and industry. And I have described a little of the excellent cooperation that we are receiving from many areas of government including the FDA, the Dept. of Homeland Security, CDC, USDA, CDC the Public Health System, and many others. But probably one of the most important areas of CMS is the way that it characterizes the distribution system for a food, and often to do this we require the collection of confidential data from industry. To ensure that we can maintain a very high level of confidentiality of such data we have spent the last year and a half with our lawyers working with the lawyers for FDA, for the National Center for Food Protection and Defense and with lawyers representing various sectors of private industry to work out a contractual arrangement under which we can, as a private company, collect data from other private companies while fully protecting their interests. Under this agreement, when we gather confidential data it is totally anonymized, and all such data is incorporated into the CMS solely in aggregation with data from other sources through statistical averages that prevent the individual data source from being identifiable. Then, when we have completed our processing of the data the original data are destroyed. Further, since BTSafety is a private entity, we are not subject to discovery through the Freedom of Information act. So these contractual arrangements give us the ability to collect all the data that we need, while providing the companies that provide the data with the assurances that the confidentiality of their data is totally protected. Finally, our thanks to the many agencies, universities, companies and individuals who are collaborating with us on this project, and a special thanks to the Food and Drug Administration, to the Dept. of Homeland Security and to the National Center for Food Protection and Defense for providing funding for this work. Proceedings of the Institute of Food Technologists First Annual Food Protection and Defense Conference

5 Consequence Management System Andy Jaine, BTSafety, LLC

6 Consequence Management System (CMS) A predictive modeling tool to help enhance government and industry response to contaminations of the food system

7 What does CMS do? Models the entire evolution of food events Quantify the timing and consequences of the event Consumer exposure and outcome Impact on public health infrastructure Impact on the economy, business, public, etc. Effect of different interventions Containment and remediation Integrates and builds on the results of stage models

8 What can CMS be used for?... What-If scenario planning Quantify morbidity, mortality, economic impact Identify impacts on affected constituencies (consumers, general public, health care, food industry, government) nt) Assist in decision making and priority setting Facilitate allocation of resources Training Consequence assessment Helps weigh the cost/benefit of various policy and intervention decisions Illustrates the time frames that would maximize the effectiveness of policies and actions Food system, agents, crisis management Table top exercises

9 Structure of CMS A tool that enables modeling of the evolution of all types of food event temporally and geographically Data-centric - reflects real data and real prior incidents Visual - easily visualize and assess the impact of decisions Flexible - accommodates all reasonable scenarios Practical operates when some attributes are unknown or imprecise Extensible facilitates easy enhancement to include improved data and models as they become available

10 CMS System U.S. Patent

11 The Consequence Management System has resulted in a collaborative program involving FDA, DHS, USDA, CDC and EPA. Government recognizes a public/private partnership is critical for defense of the U.S. food supply. I applaud and encourage continued collaboration by the private sector in the development of the CMS. NCFPD Industry Workgroup July 12, 2005 Lester Crawford Former Commissioner of FDA July 11, 2005

12 Protection of Confidential Data Secure server installed at BT Safety used exclusively for data storage (no internet connection) Password protected and limited access Companies have the option of customized confidentiality agreements Data sanitized of all company references Company participation acknowledged only if permission is granted by the company

13 Data Confidentiality Original data destroyed or returned Whenever possible, aggregate data used in the model As a private entity, data collected by BTSafety is not discoverable through Freedom of Information

14 The project is a collaboration of a number of agencies, universities and companies

15 National Center for Food Protection and Defense Industry Workgroup

16 We gratefully acknowledge funding by the following organizations: