Population Health Forecasting Model for California Developing a Prototype and its Utility

Transcription

1 Population Health Forecasting Model for California Developing a Prototype and its Utility Prepared for: The California Endowment Jonathan E. Fielding, MD, MPH and Gerald Kominski, PhD UCLA School of Public Health, in collaboration with the Los Angeles Department of Health Services and the California Department of Health Services UCLA Health Forecasting Team (May 2004) 1

2 Executive Summary Second Report UCLA Health Forecasting Project The California Population Health Forecasting Model is designed to improve the health of all Californians by reducing health disparities and supporting better, evidence-based decisions by key local, county and state elected and appointed officials as well as community organizations, advocacy groups, health plans, and other public health practitioners. The need for a comprehensive health forecast is substantiated by the rapidly changing and unique sociodemographic population mix in California and the lack of evidence-based health outcome data on our diverse population. Due to these important socioeconomic trends and migration patterns, simple extrapolation from existing trends without incorporating inclusion of these factors is likely to lead to erroneous assumptions and decisions that can have major adverse effects on future health. The stages of forecasting include formulating the problem, obtaining information, selecting the appropriate method, implementing and evaluating the method, and using forecast results. Population health forecasting uses microsimulation techniques to model individual behavior, lifetime events, and ultimately health outcomes. The model considers the influence of countless events and decisions that occur either simultaneously or in congruence with one another, and simulates a large population such as a county, state, or country, in order to produce results that can help decision-makers in better understanding programmatic and policy options to improve health and reduce health disparities. The model is comprised of three building blocks or modules: the base population framework, risk factor-disease modules, and the forecasting module. Together, they will produce estimates of the distributional effects of changes in key health determinants such as population characteristics, individual behaviors like smoking, physical activity levels, and obesity as well as summary measures of health status such as disease incidence and mortality. Physical activity is being modeled as the first risk factor due to its impact on health outcomes and increasing importance in health-related decision-making. Potential users of the California Health Forecasting Model include individuals in various healthrelated fields who are interested in advocating for public health programs or anticipating the future impact of current decisions on health outcomes. The population health forecasting model will serve as a critical analytic tool to introduce timely and relevant information into policy debates at all levels of government as well as in the private sector. It has the potential to interject new and valuable information about our future health based on existing policies and programs; provide critical information regarding changes in health disparities among subpopulations; and even more importantly, gauge the future health impacts of possible changes in current programs and policies on both the whole population and for specific groups. This real world application of the model can enhance both the advocacy and planning process for designing and UCLA Health Forecasting Team (May 2004) 2

3 implementing strategies and interventions that will improve the health of a population at the level of a state, a county or a community.. Part 1: Description of the California Health Forecasting Model Section 1: Why Forecast? Introduction A forecast is a prediction about the future that accounts for complex events and interactions by modeling characteristics and behavior while taking dynamic variable into consideration. Forecasting provides a useful framework that mitigates, to some extent, the uncertainty of the future. It thus allows one to anticipate the future impact of current actions on certain outcomes. This project focuses on development of a model to forecast the effects of specific actions, alone or in combination, on the health of a population and key subgroups. Both public and private enterprises operate under conditions of uncertainty about the future, but are required to make decisions today based on expectations and beliefs about their future impacts. Forecasting helps to reduce, but not eliminate uncertainty. Accurate forecasts can help an organization make better decisions for the future by gaining insight into both current and future trends. Forecasts can answer three types of questions about future states: When might specific outcomes of interest occur under different scenarios? What are the qualitative characteristics of the outcomes based on current inputs and/or changes in those inputs? What will be the magnitude of some quantifiable outcome, such as mortality or income, in the future? As a society, we devote considerable resources to forecasting the economy, weather, agricultural yields, transportation systems, and technology growth all very difficult and error prone efforts. We invest in these analyses because the return to various members of society, individuals, government agencies, businesses, policy makers, is considered sufficient to justify the endeavor. Each of these groups uses results from the analyses regularly to evaluate alternatives and make better informed decisions. In the health sector, forecasting has been primarily used to project the impact of current or possible changes in federal policies on health care such as their effect on insurance coverage, utilization rates of specific procedures, federal Medicare and Medicaid costs as well as the number, type, and distribution of health care providers. Yet, until now, we have almost completely neglected to forecast our most precious asset health itself. Population health may be easier to forecast than most other issues and topics for which we estimate future trends and effects. The fact is that with current knowledge and computing power, we can forecast the health of our population and key subgroups of the population over several decades with a quite acceptable level of accuracy. Further, we can forecast the health of key subgroups of the population, such as women, older adults, African-Americans, Latinos or children. UCLA Health Forecasting Team (May 2004) 3

4 The development of a population health forecasting model has the potential to interject new and valuable information about our future health based on existing policies and programs. It will also provide critical information regarding changes in health disparities among sub-populations. Even more importantly, the model will allow us to gauge the future health impacts of possible changes in current programs and policies on both the whole population and for specific groups. Thus, the forecasting model will serve as a critical analytic tool to introduce timely and relevant information into policy debates at all levels of government as well as in the private sector. In this capacity, health forecasting can enhance advocacy efforts as well as the planning process for designing and implementing strategies and interventions that will improve the health of a population. Health Forecast for California We are interested in predicting future health outcomes such as mortality, disease incidence, and morbidity for the population of California. We also want to examine how future trends in health outcomes will differ for subgroups of the population. As specific examples, how will changes in rates of obesity affect the health of Latino women over age 65 living in Los Angeles County or African American men between age 25 and 35 years living in Alameda County? Currently, the evidence base is comprised of populations that do not reflect California s diversity. Thus, our health forecasting tool is necessary for advocacy and program planning as it accounts for changes over time in the demographics of California s population. The basis for the forecasting model is a very substantial data base on the health trends in populations and on characteristics and behaviors that ample epidemiologic evidence has shown to be causally related to health in populations. For example, African Americans have a lower life expectancy compared to whites and Latinos have a higher incidence of Type II diabetes than both African Americans and whites. Also, individuals with lower socioeconomic status (SES) are more likely to die prematurely than individuals with higher SES. Previous research also shows that several risk factors such as physical activity, smoking, alcohol consumption, and diet strongly influence health outcomes For example, individuals who do not engage in physical activity are more likely to develop a heart attack and those who smoke are more likely to experience many forms of cancer as well as cardiovascular disease than nonsmokers. There is also a strong base of evidence from reviews of epidemiologic studies that specific interventions alter the health outcomes in the overall population and subgroups over time but and the distribution of these risk factors is not the same across the population but changes over time and with age. In addition to predicting future trends in health outcomes and disease burdens for the entire population, the California Health Forecast Project uses a microsimulation modeling approach capable of forecasting future health for specific subgroups of the population. Finally, by changing the levels of certain sociodemographic characteristics and risk factors, which is particularly important given California s dynamic population mix and changing characteristics, we are able to see their effects on future health outcomes. UCLA Health Forecasting Team (May 2004) 4

5 Section 2: Overview of Forecasting and Microsimulation Stages of Forecasting 1. Formulate Problem Figure 1 outlines the stages involved in any forecasting effort. The first step involves by formulating the problem or research question(s) of interest such as: What will the population of California s health status be in 2020? How will health outcomes vary for different groups of people living in California such as Latinos, Angelenos, or persons over age 65 years in 2020? How will health status differ among California adults if their average level of physical activity increases by 20% between now and 2020? 2. Obtain Information Once the problem has been formulated, it is necessary to collect relevant information needed to answer the pertinent questions. Both qualitative and quantitative information should be included if appropriate. For example, to answer the questions posed above, one might need the following information: Historical demographic data on the population of California, Information on population trends in health status over time, Information about key risk factors that reproducibly affect health outcomes, Information about the relationships among demographic characteristics, risk factors, and health outcomes, and Information to project future values of these variables 3. Select Method The next step is to select the appropriate method for forecasting. A forecasting model is a model developed to produce forecasts (or estimates in unknown situations) [and] may draw upon a variety of measurement models for estimates of key parameters [or] true values of some unknown population value (Armstrong, 2001). Two major types of forecasting, qualitative and quantitative, exist in a broad sense. Qualitative forecasts specify the expected direction of the variable of interest whereas quantitative methods estimate both the direction and the magnitude. Quantitative approaches to forecasting are classified based on the type of data and techniques used to analyze the data. When selecting a quantitative approach, one should make several assessments including the level of knowledge about relations between variables; amount of change involved; type of data available; need for policy analysis; and the extent of knowledge regarding the particular domain (e.g., health) (see Figure 2). Aggregate Models One set of quantitative models uses aggregate level data to produce aggregate level forecasts, which do not provide information about the distribution of risk factors or outcomes (Table 1). Naïve Models UCLA Health Forecasting Team (May 2004) 5

6 The simplest of these approaches is the naïve model, which assumes that future values of the outcome variable will be based on current and historical values (generally the most recent value) without incorporating additional information or conducting any type of analysis. Naïve models generate quick and easy results that typically assess the short-term usually measured in days, weeks, months, or a single year. They require minimal data and are easy to implement. For example, the base rate a typical or average behavior for a given population can be measured with cross-sectional data and serve as a naïve model, which has been referred to as the default forecast. One example of a successful naïve model might be using this year s number of registered nurses employed in a certain area to predict the number of nurses available in the same area the following year. Without accounting for other factors such as nurse recruitment programs and new nursing school graduates or considering external issues that might affect a nurse s decision to work, this approach would be considered naïve. Some observers have argued that naïve models can sometimes outperform other qualitative and quantitative methods in predicting short term future values of certain variables such as the stock market, gross domestic product (GDP), and inflation (Sherden, 1998). However, naïve models are often of limited usefulness for policymakers. The primary limitation is that naïve models do not produce accurate forecasts when variables are expected to change over time. By definition, they will miss trends such as cyclical or seasonal effects in the data. Further, naïve models do not provide any insight into alternative factors that may induce change in future outcomes. In sum, naïve models are adequate for simple decisions with relatively minor consequences where the cost of conducting a formal forecast would outweigh any potential benefits. When little change is anticipated in the future, and the use of complex forecasting methods is not warranted, policymakers may be inclined to use a naïve approach or rely on their own unaided judgments. Extrapolation Models Some demographers and other researchers have extended the naïve model to consider recent trends in an outcome variable when predicting future values. Numerous researchers have published extensively regarding different methods of demographic forecasting, which include time-series analysis and extrapolation (Lee and Carter, 1992). Extrapolation models allow for patterns and trends (e.g., seasonal or cyclical) by using historical values of a series or cross-sectional data that are assumed to be generalizable in the future. In general, extrapolation is used for decisions about trends that are relatively stable and for which much change is not expected in the future. For example, extrapolation would probably not be useful for assessing the effects of a new intervention or the implications of a change in policy. In assessing the health of populations, there are many known factors of influence and a significant body of literature on specific policies and programs that can alter these factors of influence. Cell-Based Models The cell-based model examines associations between risk factors and outcomes without making explicit assumptions regarding causality. Typically, cell-based models employ cross-sectional data and therefore do not allow for a temporal component per se, but do prove to be useful for UCLA Health Forecasting Team (May 2004) 6

7 following aggregate-level measures over time. This approach to modeling is used often in health-related research, where observational data is readily available and the effects of several independent variables (e.g., individual characteristics and behaviors, interventions, and diseases) on a particular health outcome are of interest. For example, The World Health Report 2002 highlights the relation between twenty risk factors and the global and regional burden of disease (GBD) (World Health Organization, 2002). The report includes predictions about the burden of diseases in different regions of the world that can be attributed to different risk factors. Their analysis creates a hypothetical situation where the disease or risk factor is reduced or eliminated and compares the results to the current situation or future projections based on current levels. There is no consideration of the effectiveness of proposed interventions. Instead, the authors assume that rates for risk factors are set to the lowest possible levels in the alternative scenario. Although it represents a different type of analysis, the GBD makes an important methodological contribution to our work, as it standardizes risk factors and outcomes to allow for comparisons across different factors. Combined Extrapolation and Cell-Based Models Extrapolation techniques and cell-based analyses can be combined to produce aggregate level forecasts that incorporate a temporal component while assessing relationships among variables. Kenneth Manton at Duke University s Center for Demographic Studies has used this approach in publishing extensively about the impact of reductions in disability in the elderly and the implications for the health care system. He developed two health forecasting models: one model was designed to analyze discrete health state changes using population and vital statistics data whereas the other describes both discrete and continuous changes using data from longitudinal community populations (Manton, Stallard, and Singer, 1992). The first model supplements aggregate data with multiple inputs from scientific experts while the second uses relatively information-rich measurements. Both models can be modified by expert judgment to deal with simulations of multiple possible interventions (Manton, Stallard and Singer, 1992). Population Microsimulation Models Microsimulation models are distinct from the models mentioned above in that they do not examine aggregate level data. Rather, microsimulation techniques operate using unit-level data such as household information, as well as joint distributions of risk factors and outcomes (Figure 2). Doing so allows the technique to consider the composition of the population and subgroups while assessing simultaneous changes in multiple factors including behaviors and programs or policies in order to make conclusions based on outcomes of interest in simulated populations. Population microsimulation models typically assess program or policy changes in light of social and economic conditions. In fact, simple microsimulation models have been used to predict the impact of policy and behavioral changes on specific aspects of the health of populations for more than two decades, but have considered only a small set of risk factors and diseases applied to static populations. Microsimulation models are flexible and can easily incorporate different assumptions and new research findings in an efficient manner. This quality is particularly valuable for assessing different scenarios based on competing programs or alternative interventions in an effort to meet decision maker requests. UCLA Health Forecasting Team (May 2004) 7

8 There are two major types of microsimulation models static and dynamic that are widely used to model potential impacts of social policies (Citro, 1991). Static models operate using cross-sectional databases that provide a snapshot of the population at a point in time (Citro, 1991). In contrast, dynamic models operate using longitudinal databases that contain individual histories (Citro, 1991). In the late 1980 s, Milton Weinstein at Harvard developed the Coronary Heart Disease Policy Model, a static microsimulation model of policy and technological advances on the incidence, prevalence, and mortality from coronary heart disease, and changes in the cost of health care. Lee Goldman at UCSF, Department of General Internal Medicine, continues to use this discretetime, state-transition model to project future trends and assess the impact of interventions. He used the CHD Policy Model to estimate the effects of investments designed to change coronary risk factors between 1981 and 1990 on the incidence, prevalence, mortality and costs of CHD during this period and projected the impact of the interventions through 2015 using effects of risk factor reductions that were estimated for Observed changes in risk factors between 1981 and 1990 resulted in a reduction of CHD deaths by approximately 430,000 and overall deaths by approximately 740,000, with an estimated cost effectiveness of $44,000 per year of life saved. Given that much of the benefit from risk factor reductions is delayed, estimated reductions for the extended, 35-year period of 1981 to 2015 were 3.6 million CHD deaths and 1.2 million non-chd deaths, which reduced the cost to about $5,400 per year of life saved (Goldman et al, 2001). More recently, researchers at Statistics Canada have developed a dynamic or continuoustime model Population Health Model (POHEM) that combined new and existing models to assess the impact of different policy interventions or technologies on the health of the Canadian population. Infectious Disease Models Infectious disease models include forecasting the progression and impact of certain infectious diseases on future health outcomes and at a very simple level can be modeled in the framework previously discussed. Similar techniques that might be used in this situation would include modeling known risk factors. For example, many of the world's infectious diseases are known to be highly sensitive to long-term climate and short-term weather changes. The application of environmental data to the study of disease provides the opportunity to forecast the risk of disease outbreaks or epidemics. In fact, existing global systems for epidemic preparedness focus on disease surveillance using either expert knowledge or statistical modeling of disease activity and thresholds to identify time periods and areas of risk (Myers et al., 2000). A 2001 report published by the American Academy of Microbiology summarized the state-of-the-field with regard to modeling the relations between climate and human health through changes in vector-born and infectious diseases. Because accurate disease forecasting models would markedly improve epidemic prevention and control capabilities, more sophisticated models have been developed to consider individual interactions and contact patterns that can affect outcomes. Researchers at the University of Michigan have developed computer programs to project the incidence rates of infectious disease based on complex patterns of contact, mode of transmission, incubation period, and vectors of disease transmission (Koopman et al, 2002). UCLA Health Forecasting Team (May 2004) 8

9 Model Selection Overview In general, quantitative methods are preferable to qualitative techniques when sufficient data are available. In general, quantitative models are used for complex problems, such as health. Also, econometric models typically perform better for long-term forecasting, whereas extrapolation models are often employed for short-term forecasting projects. When large changes in the outcome measure are expected, causal approaches are typically preferred to naïve approaches. The choice of forecasting method depends on four key issues: 1) forecast accuracy needed, 2) complexity of variable relationships, 3) time allowed for analysis, and 4) balance of forecasting costs relative to benefits. When choosing which forecasting method to use, each of these issues should be considered. Other criteria usually employed in the selection process include convenience, market popularity, structured judgment, statistical analytic needs, relative track records, and experience with alternative approaches from published research. While myriad forecasting methods have been employed for various research problems, the appropriate method depends on the specific situation and study objectives (Figure 3). Selecting a single forecasting model can prove to be difficult, therefore it may be useful to combine methods or try multiple approaches. 4. Implement and Evaluate Method The chosen forecasting method(s) will then be implemented to generate projections (i.e., outputs). Implementation requires securing appropriate resources such as a research team, software, and necessary data sets. Implementing more complex methods such as population microsimulation also requires sufficient time for the construction of each model component to ensure that the model will run smoothly. The next critical step is to evaluate the forecast in the situation it will be used by asking a variety of questions about model inputs, assumptions and outputs. Possible questions might include: Do reasonable alternatives exist for the base assumptions? Do the underlying assumptions about rate of change based on changing inputs make sense? Are the data and methods valid and reliable? Are the outputs sufficiently robust to answer the original research question(s) and can the analysis be easily replicated? Forecast evaluation should employ standard scientific principles that would apply to any other academic study. For example, a good evaluation should begin with tests of inputs and outputs followed by a comparison of the method used to reasonable alternatives. Testing inputs is particularly important for causal modeling in which model improvement will allow for a better assessment of policy changes. Testing outputs on the other hand is clearly important for assessing uncertainty but also for ensuring that the appropriate model was chosen. One method of testing inputs involves testing underlying assumptions with objective measures such as published results. In the absence of objective information, subjective measures such as expert opinion can be used. A method for identifying unreasonable assumptions is testing the UCLA Health Forecasting Team (May 2004) 9

10 construct validity of important model parameters and/or relationships between the parameters as well as other inputs. The next evaluation step should be testing the data and study methods. Attention to data validity and reliability can greatly impact forecast results and conclusions drawn from them. In general, the more important the event (i.e., intervention or policy change) under study, the more important is data testing. In keeping with good scientific practice, all data used in the forecast should be fully disclosed and accessible where appropriate. Ideally the completed model should be validated by predicting the most recent health outcomes, for year T, by creating forecasts based on prior periods, t=0 T-1, and check if there is reasonable predictive value. This requires that the model be developed excluding the most recent years of data, and check if the truncated model can predict the outcomes for these years. Making forecast data available to other researchers and entities will allow outputs to be replicated. Conducting direct replications of the forecast that generate the same or similar results is an obvious reliability check. Other methods of assessing reliability might include comparing parameter results from different forecasts, applying the same forecast methods to similar data, or extending the study to other situations. In addition, output assessment can include examining face validity or the reasonableness of forecast results and using appropriate error measurement. 5. Use Forecast Results If the evaluation identifies gaps in knowledge or deficiencies in the selected method, the decision should be reassessed, a new method should be selected and the process should occur again. Once a thorough evaluation provides evidence that the forecast methods are scientifically sound, model results can be used. In this final step, forecasters must work to ensure that the model gains acceptance among decision-makers, who will in turn use projections from the forecast in the decision-making process. The extent of model use is directly related to the level of confidence placed in the forecast results, which is in large part determined by the level of uncertainty in the forecast. One method of dealing with uncertainty in population forecasts is the variant approach which uses a combination of assumptions to generate a forecast range bounded by high and low values. Another approach is to use statistical techniques to generate probability distributions. Finally, population forecasters can combine more simple statistical approaches with expert opinion to account for uncommon events that are not likely to be captured in trend data. In sum, several approaches to ensure wide spread use of forecast results might include: Providing a range of estimates using prediction intervals for all projections so that policymakers have a greater comfort level with the accuracy of results, Monitoring and reporting forecast accuracy over time, Updating the model periodically to ensure that the most recent data is included and current assumptions hold. Actively disseminating overall results and specific outputs relevant to timely policy challenges. UCLA Health Forecasting Team (May 2004) 10

11 Table 1: Summary of Characteristics of Different Approaches to Model Data Approach Description and Organization of Data Types of Questions Answered Outputs Aggregate Models Naïve Extrapolation Cell-Based Aggregate-level outcome data from previous and current periods Assumes the current value of the outcome measure is held constant in the future (i.e., no change). Aggregate-level, outcome data Autoregressive Integrated Moving Average Model ARIMA time series, and other extrapolation techniques are used to model expected changes in the future values of an outcome based on historic trends Aggregate-level (typically crosssectional) data on all risk factors and outcomes Models associations between a series of risk factors and an outcome variable What will be the future value of the outcome variable in the near term if nothing changes? What is our best guess of the future value of an outcome measure based on historic trends? Which factors are associated with the outcome variable? What is the predicted value of an outcome measure given a certain set of risk factors? Projected mean values for outcome variable in future (generally short term) time periods Projected mean values for outcome variable in future time periods Parametric or non-parametric estimates of independent variables at aggregate level Predicted mean value of outcome variable in aggregate given specific values of independent variables Examples of Type of Analysis Planning Budgeting Planning Budgeting Demographic models Actuarial models Macroeconomic models Evaluations of Interventions Social Epidemiology Biostatistics Advantages Minimal data requirements Often sufficient if little change is expected during forecast period Useful for outcome variable with distinct patterns that are repeated over time Limited data requirements Fairly accurate in projecting trends over long time periods Allows assessment of associations between risk factors and outcome variables Disadvantages Does not consider factors associated with changes in outcome variable (i.e., provides no insight into risk factors) Uses population means and assumes an underlying distribution of the data No consideration of factors associated with changes in outcome variable (i.e., provides no insight into these factors Uses population means and assumes an underlying distribution of the data Does not consider impact of time on outcome Uses population means and assumes an underlying distribution of the data Greater data requirements No causal inference UCLA Health Forecasting Team (May 2004) 11

12 Approach Combined Extrapolation and Cell- Based Description and Organization of Data Aggregate-level, data on all risk factors and outcome Models relations between a series of risk factors and an outcome variable and then uses ARIMA, time series, and other extrapolation techniques to incorporate time Types of Questions Answered Given the predicted values for a set of risk factors and historical trends, what is the best guess of the value of an outcome in the future? Outputs Projected mean values for outcome variable in future time periods Predicted mean value of outcome variable in aggregate based on values of independent variables Examples of Type of Analysis Planning Forecasting Evaluation Advantages Allows assessment of relations between risk factors and outcome variables Fairly accurate in projecting trends over long time periods Disadvantages Uses population means and assumes an underlying distribution of the data Significant data requirements Joint distributions of risk factors and outcome measure are required Assumes proportional and constant hazard function No causal inference Microsimulation Discrete Time Continuous Time Individual-level data on all risk factors and outcomes Models a sample population of individuals prospectively to project future distributions of risk factors and outcomes Individual-level data on all risk factors and outcomes Models a sample population of individuals prospectively to project future distributions of risk factors and outcomes What will the future distribution of risk factors and outcomes be for a sample population in the absence of policy changes? How will individuals be affected by policy changes? What will the future distribution of risk factors and outcomes be for a sample population in the absence of policy changes? How will individuals be affected by policy changes? Projected distribution of population according to outcome variable in future time period Projected distribution population according to outcome variable in future time period Planning Forecasting Evaluation Planning Forecasting Evaluation Provides distribution of individuals according to risk factors and outcome variable Models changes in multiple risk factors in a single time period Accommodates age-time interactions, non-proportional and variant hazards and threshold effects Allows for causal inference Provides distribution of individuals according to risk factors and outcome variable Models changes in multiple risk factors simultaneously Accommodates age-time interactions non-proportional and variant hazards and threshold effects Allows for causal inference Extensive data requirements Joint distributions of risk factors and outcome measure are required Less precise than continuous time microsimulation models Forecasts might not be more accurate than simple models Extensive data requirements Joint distributions of risk factors and outcome measure are required Forecasts might not be more accurate than simple models UCLA Health Forecasting Team (May 2004) 12

13 Section 3. UCLA Population Health Forecasting UCLA Population health forecasting uses microsimulation techniques to model individual behavior, lifetime events, and ultimately health outcomes. The model considers the influence of countless events and decisions that occur either simultaneously or in congruence with one another. The model simulates a large population such as a county, state, or country, in order to produce results that can help make conclusions. Through the consultation process with the advisory panel and the team s forecasting methods research, several important methodological challenges related to population health forecasting have been identified: Long-term methods: for many health conditions, long-term forecasts are needed to model associations between risk factors and final health outcomes because of the long induction periods for many chronic diseases. For example, some health benefits of reductions in smoking initiation rates may only start to be seen twenty or more years later. However, it is also clear that changes in some risk factors have very rapid health benefits for example smoking cessation reduces heart attack rates within the first year. Multivariate approach: The complexity of the factors known to influence health outcomes argues for multivariate approaches including multiple risk factors and multiple health outcomes for health forecasting. Distributional effects: Although policymakers and planners may be interested in knowing whether the population s health will improve or decline globally, they are also interested in identifying the underlying causes of these changes and highlighting distributional effects within subgroups according to race/ethnicity, gender, age, and location. Data Requirements: In order for the final model to be able to produce detailed distributions of the population, risk factors, disease prevalence, and mortality rates by age, gender, race/ethnicity, and county, extensive data are required for each of the building blocks. Each of the building blocks requires distinctly different techniques so each module can be developed independently. However, the form and content of each building block is highly dependent on the others, so it is important to coordinate and integrate the various components and variables in the models. The natural order is to first build the core microsimulation framework while considering the diseases, risk factors, and interventions that will be included in later modules. These factors must be incorporated into the core framework from the outset. Subsequently, the disease module and then the forecasting module will be developed. Base Population Module A base population module for California includes socioeconomic and demographic information and computer programming to perform microsimulations of samples that describe the California population. It provides the underlying framework for the population health forecasting model UCLA Health Forecasting Team (May 2004) 13

14 and contains a demographic profile of California s population. This allows for synthetic replications, which will enable analysis of intervention effects. This component does not require assumptions on causal relations or structural forms of disease developments, but rather allows these relations to be modeled through the incorporation of disease modules. Risk Factor - Disease Modules Risk-factor disease modules are smaller models that describe linkages between 1) various risk factors, environmental conditions, and population characteristics and 2) morbidity and mortality. These components are crucial to understand health trends in California s population. Various diseases and risk factors are first modeled separately and then linked together through the microsimulation framework, given that the risk factor or disease information can be linked to the variables that are available in the simulation. This approach allows the use of research and conceptual models from around the globe, as long as the associated effects are not specific to one locale or a specific population segment. For example, models of heart disease are not particular to a location within the state and could thus serve as useful tools. However local data are required to model the impact of smog on respiratory diseases in Los Angeles County thus ruling them out for inclusion in the model. Forecasting Module Initially, the microsimulation model needs to use current data on the population of California. In order to make statements about the future, it is also necessary to make projections regarding the model assumptions and population data. These projections may be created using aggregate models of the parameters of interest, such as migration, fertility, and economic developments. These projections could be based on any combination of techniques, such as econometric models, time series models, expert judgments, or the Delphi method. It is likely that judgmental time series forecasting will be principally used to project variables with large expected changes (Webby, O Connor, Lawrence, 2001). Section 4: Specifications of The California Health Forecasting Model Our model is comprised of three components: a descriptive population framework, risk factordisease modules, and a forecasting module (Figure 4). Descriptive Population Framework Our preferred framework for forecasting is similar to two existing microsimulation models developed by Statistics Canada: 1) LifePath is a dynamic longitudinal microsimulation model of individuals and families. Microsimulation models extend multistate, disaggregated models using individual level data (Ahlburg, 2001). Using behavioral equations based on historical data, it creates statistically representative samples consisting of complete lifetimes of individuals. It is used for analyzing and developing government policies, particularly those requiring evaluation at the individual or family level, with an essentially UCLA Health Forecasting Team (May 2004) 14

15 longitudinal component. It can also be used to analyze a variety of societal issues of a longitudinal nature such as intergenerational equity or time allocation over entire lifetimes. 2) POHEM is an extension of LifePath that models health and disease. Using equations and sub-models developed at Statistics Canada and drawn from the medical literature, the model simulates representative populations and allows the rational comparison of competing health intervention alternatives, in a framework that captures the effects of disease interactions. More information on these models can be found at Both models include overlapping birth cohorts, and fully simulated person characteristics, replicating actual aggregate numbers. In developing our microsimulation models, one hundred years of vital statistics will be required to simulate a complete cohort. In addition, the model requires detailed mortality data (life tables), immigration and birth statistics. Furthermore, we include information on gender, education, employment, income, marital status and geographic location. Finally, the sample cohort needs to be stratified by race and ethnicity so that racial and ethnic differences can be modeled. Risk Factor Disease Modules These modules provide the link between population characteristics and health outcomes and are based on analyses of data sets linking individuals behavior with morbidity and mortality outcomes. Our primary focus is on characteristics such as socioeconomic factors (i.e., income and education) and behavioral factors. Health-related behaviors will include physical activity and diet/nutrition as well as risk factors such as smoking and alcohol intake, cholesterol level, and blood pressure. Health outcomes will include those diseases such as diabetes, cancer and coronary heart disease (CHD) that are both leading causes of mortality and affected by public health interventions. Making the connection between risk factors and health outcomes require the use of widely accepted, prominent modeling approaches (e.g., Weinstein model for CHD) in order to increase public acceptance of the population forecasting model results. By leveraging existing models, there will be no need to conduct additional analyses. In fact, the ideal method of combining literature sources would be to conduct extensive meta-analyses. However, such an approach is also outside the scope of this project. Rather our approach will be to use the results of published meta-analyses and, where such information is not available, to be somewhat less systematic while still accounting for important methodological concerns such as study design. The Forecasting Module This final module enables us to forecast health outcomes and project the results of interventions into the future. Without the forecasting module, we would be able to predict current outcomes under counterfactual conditions, but we would not be able to project future outcomes. Using a variety of forecasting techniques, we will be able to estimate the impact of various interventions not only in today s world, but also in tomorrow s world. In this module, we will take full UCLA Health Forecasting Team (May 2004) 15

16 advantage of the growing results of systematic meta-analyses of interventions designed to improve health at the population level. The principal source of this data is the Guide to Community Preventive Services but other sources, such as the Cochrane Collaboration, will also be accessed. A mixture of quantitative and qualitative forecasting techniques will be used to develop projections for the relevant variables. As mentioned above, projecting a population's future health status requires considerable data. The process has been facilitated through the collection of trend data and projections of the demographic characteristics of California s population from the Department of Finance. Data Sources Table 2 includes a list of data sources by module for each type of information required for the population health forecasting model. UCLA Health Forecasting Team (May 2004) 16

17 Table 2: Data Summary Sources for Health Forecasting Model Type of Information Base Population Module 1. Historic and current population counts for each county, stratified by age, gender, and race/ethnicity 2. Historic and current mortality and fertility rates for each county, stratified by age, gender, and race/ethnicity 3. Historic and current immigration rates for each county, by country of origin 4. Historic levels of physical activity for Californians, stratified by age, gender, and race/ethnicity 5.Historic mortality rates for CHD, colon cancer, diabetes, and depression for each county, stratified by age, gender, and race/ethnicity 6. Historic levels of educational attainment for each county, stratified by age, gender, and race/ethnicity Physical Activity Module 1. Relation between physical activity levels and allcause mortality and CHD, colon cancer, diabetes, and depression mortality, stratified by age and gender 2. Persistence of individual physical activity levels over time, stratified by age, gender, and race/ethnicity 3. Relation between physical activity levels in childhood and adolescence and physical activity levels for adults, stratified by age, gender, and race/ethnicity Source(s) 1. Department of Finance (California) 2. Department of Finance (California) 3. Department of Finance (California) 4. Behavioral Risk Factor Surveillance System (BRFSS), National Health Interview Survey (NHIS), National Health and Nutrition Examination Survey (NHANES), California Health Interview Survey (CHIS) 5. Vital statistics data and assumptions. 6. Department of Education (California) 1. Previous studies in published literature; Longitudinal data set such as Alameda County Human Population Laboratory 2. Previous studies in published literature 3. Previous studies in published literature; NHISlinked file, Human Population Laboratory Forecasting Module 1. Projected population count for each county, stratified by age, gender, and race/ethnicity 2. Projected mortality and fertility rates for each county, stratified by age, gender, and race/ethnicity 3. Projected immigration rate for each county, by country of origin 4. Projected levels of physical activity for Californians, stratified by age, gender, and race/ethnicity 5. Projected mortality rates for CHD, colon cancer, diabetes, and depression for each county, stratified by age, gender, and race/ethnicity 6. Projected highest level of educational attainment for each county, stratified by age, gender, and race/ethnicity 1. Department of Finance (California) 2. Department of Finance (California) 3. Department of Finance (California) 4. Forecast based on historical trends from population surveys and assumptions. 5. Forecast based on historical levels from vital statistics data and assumptions. 6. Department of Education (California) or Forecast based on historical levels from population surveys and assumptions. UCLA Health Forecasting Team (May 2004) 17

18 Simulating an Individual Within the microsimulation framework, the birth and fixed characteristics (gender and race) of an individual are simulated with a probability determined by population tables the birth date could be anywhere in the simulated time period from 1870 through Historical population tables determine the probability of a birth in the past, and projected population tables determine the probability of a birth in the future. It must be noted that increasing the length of base data does not generally improve forecast accuracy (Ahlburg, 2001). Life Events Once this simulated individual has been created in the computer model, the life events (starting with the birth) can be generated, with the last life event being death. Life events are conditional on the status of the individual according to gender, age, race, education, and employment status. Some of these variables are static whereas others are dynamic and change as a result of an event. For example, marriage is an event causing the status to change from single to married. Events can also change the probabilities of other events (e.g., marriage will change the probability of having a child). Behaviors, such as exercise and smoking, and health outcomes can be generated in a similar fashion. Trends Generally, trends in the important variable such as educational attainment, mortality, and obesity are non-cyclical. For example, each year more people have more education, all-cause mortality rates are declining slightly, and over the past twenty years the percentage of California who are overweight has increased steadily. For some of the current trends, there is a theoretical maximum and minimum, but because of uncertainty there needs to alternative estimates for trend changes over time. Time-series and trending methods are used to extrapolate historical trends into the future. These techniques will be supplemented with expert judgment to predict future discontinuities (Webby, O Connor, Lawrence, 2001). Although the use of multiple forecasting methods has been shown to produce more accurate forecasts, it is a time-intensive process. Therefore, forecasting methods must be refined over time as the model contains better input data, time passes, and new data become available. Uncertainty and Validation Although it is done infrequently in practice, the results of forecasting models should be evaluated (Armstrong, 2001). Common validation methods can use either observed or projected data and include backcasting, making concurrent predictions using split samples, and calibrating models (Armstrong, 2001). For example, trends in historical data can be used to predict current mortality rates for the population. This is known as backcasting. Calibration is a valuable tool to understand feedback within the model. A natural feedback loop that is not preserved in the proposed microsimulation framework is multigenerational fertility rates; however, this impact can be reflected in model outcomes by recalibrating the model after introducing the impact on UCLA Health Forecasting Team (May 2004) 18

19 fertility rates. Because it often involves post-hoc adjustments to results, calibration needs to be used very carefully to ensure that real effects are not eliminated from the model or smoothed out. If used properly, calibration can provide additional insight into the underlying model. In assessing outputs from the model, evaluation criteria should be pre-specified and multiple error measures should be used (Armstrong, 2001). Once reasonable models have been identified, sensitivity analyses can be conducted. In order to indicate the level of precision of the forecast, prediction intervals will provided along with point estimates. The prediction interval formula for the forecast of a regression-dependent variable conditional upon known future values for the independent variables and normally distributed disturbances is commonly taught and used by forecasters (Chatfield, 2001). The standard formula, however, can fail when the disturbances are non-normal, with the degree of failure increasing rather than decreasing with sample size. Bootstrapped prediction intervals based on either the percentile principle or the percentile-t principle have been found to perform substantially better (Lam, 2002). Assumptions A series of assumptions are required in developing the forecast because: A limited number of databases exist that assess a wide range of health determinants acting at the individual and population levels; A limited amount of longitudinal data is available to assess disease incidence and health status changes and to capture the lag times between certain determinants and outcomes; Defining explicit rules and methods for relating information about different determinants and integrating different data sets into a single data set is challenging; and Setting the rules for when causal relations have not been adequately established and creating analytic frameworks that specify the intermediate and distal variables in the causal models is difficult. Running the microsimulation requires distributional assumptions for multiple variables for a number of years. Due to incomplete data, some estimates may initially be largely driven by underlying assumptions. The microsimulation suggested by this framework eliminates some of the feedback that normally is part of a forecasting framework, particularly for birth/fertility rate trends. This issue will be addressed through an iterative calibration process. Example: Modeling Physical Activity Rationale Due to the complexity of the task and resource limitations, the team decided to start with one risk factor and several diseases in the initial model. A number of risk factors, smoking, obesity, and physical activity, and one disease, diabetes, were considered. Physical activity was selected for several reasons: UCLA Health Forecasting Team (May 2004) 19

20 It represents a new and important contribution to the field. Physical activity has not been modeled as extensively as smoking and is easier to model than obesity, which is an intermediate health state; It has been shown to significantly impact health outcomes; Effort is increasing to develop interventions that may lead to increased physical activity levels in the population; and Variation in incidence and mortality from a wide range of conditions, such as coronary heart disease, diabetes, colon cancer and depression has been found to be associated with different levels of physical activity. Additional risk factor modules can be added once the initial model has been developed. Smoking, obesity, and diabetes are potential candidates for future inclusion. Challenges Choosing physical activity also poses certain methodological challenges. First, physical activity levels are difficult to measure accurately due to the choice and change over time of categories that are frequently used in classifying physical activity. Physical activity is typically classified according to its purpose: work, leisure time, or transportation. Although initial studies and surveys of physical activity focused on activity within the workplace, more recent work focuses almost exclusively on leisure time activities. As a result, our model initially includes leisure time physical activity. Second, physical activity is comprised of multiple interrelated components including frequency, duration, intensity and activity type, which can result in different effects on subsequent health outcomes. Ideally all of these components are included in the model, however these variables are highly correlated and the independent effects are not consistently measured in the literature. Consequently physical activity in the model is reduced to a single variable that relates energy expenditure in leisure time activities to an individual s metabolic rate in rest, Metabolic Equivalent Hours per week or METhrs/wk. E.g. if an individual walks at a moderate pace while carrying a light object, which has a MET value of 4.0, twice a week for 45 minutes, the weekly LTPA value is equal to 2 x 0.75 x 4 or 6.0 METhrs/wk. Lastly, large national data sets are often cross-sectional with a limited number of items asking respondents to self-report their physical activity levels during a short period, which typically ranges from one week to one month. Ideally, the model would include physical activity patterns over longer time periods, which would be subject to less bias from seasonal weather patterns and other short-term fluctuations in physical activity behavior such as New Year s resolutions. A major barrier to modeling the distribution of health outcomes by race and ethnicity according to levels of physical activity is the limited amount of published research on physical activity patterns among Latinos and relations between physical activity and morbidity and mortality for Latinos. Few data sets contain detailed physical activity information and Latino samples that are large enough to analyze them as a distinct subgroup. Based on the available published research, it seems reasonable to assume that the health impacts of similar physical activity patterns are similar for Latinos and non-latino whites. In other words, ethnicity for these two groups does not modify the effect of physical activity on health outcomes. Although the health impacts of UCLA Health Forecasting Team (May 2004) 20

21 physical activity are assumed to be constant, levels of physical activity will differ between the various ethnic groups. Part 2: Users, Uses, and Utility of Model Outputs Users The model allows decision-makers to address important issues and answer key questions related to maintaining the status quo, implementing interventions, and determining health status change among target populations. Thus, potential users of the California Health Forecasting Model are comprised of individuals from community organizations, policy makers, health plans, advocacy and local, county, and state public health departments. Before we discuss user groups in detail, it is important to classify them according to both level of control and degree of interest. Decision theorists distinguish between active and passive control over future outcomes. Does the decision maker have the ability to alter the direction or magnitude of the output or key input variables? Active control means that the entity has some ability to control while passive control is anticipatory of the likely consequence. In addition, we consider decision-makers interests, defined as the role forecasting results play in carrying out their primary work-related tasks and responsibilities. As seen in Figure 5, potential users of the Health Forecasting model are divided into four quadrants based on the following considerations: 1) the ability and opportunity of the potential user to directly impact health outcomes and 2) the importance and relevance of health forecasting to user s primary job responsibilities. High Opportunity, High Relevance Groups According to this classification, officials in county and state Departments of Health and elected officials constitute a very high priority audience for the forecast due to their considerable authority to implement interventions to improve population health outcomes and intermediate risk factors. These officials make daily decisions in keeping with their responsibilities to protect and improve the health of a specific population. Informed decisions are generally made using personal judgment, expert opinion, and published literature as well as analytic results from other sources. Results from the health forecast model will assist in making complex decisions where costs must be weighed against outcomes by estimating an appropriate outcome value that accounts for simultaneous changes and the way things move together. This in turn allows officials to make more knowledgeable tradeoffs in an effort to strike the most equitable, efficient balance, better evaluate the impact of interventions and identify the greatest opportunities for the future. In some instances strong community groups that are formed because of their high interest in improving community health can also be a high impact group because of their ability to affect change at the community level. High Opportunity, Low Relevance Groups As highlighted in a recent Health Impact Assessment (HIA), we recognize that professionals working in other sectors such as Boards of Education and Air Quality Management District have the ability to impact population health outcomes even though this might not be the primary focus of their work (Dannenberg et al., 2006, and Northridge et al., 2003). For example, policies in UCLA Health Forecasting Team (May 2004) 21

22 schools (e.g., school lunch foods and exercise regimens) can have a large impact on the health of children and adolescents while achieving a healthy level of air quality can greatly impact the health of an entire community or region. Other examples of decision-makers who currently are likely to have limited interest in forecasting but whose decisions strongly influence health are physicians, nurses, and other health care providers as they actively affect individual health outcomes through their daily work. These kinds of decision-makers have the ability to positively affect health outcomes by developing and carrying out interventions or policies. However, immediate relevance of health forecasting results is likely minimal given the demanding expectations of their primary job responsibilities. Low Opportunity, High Relevance Researchers, foundations and other policy organizations (e.g., State Department of Finance and Legislative Analyst s Office) have some ability to shape health outcomes but probably less than policy makers. However, they are likely very interested in the results produced by the model. Researchers consistently try to answer complex research questions and explain the state of the world. In particular, health researchers are interested in examining the effects of health policies and interventions on population subgroups and modeling the future impact of current decisions (e.g., adverse health effects of reduced access). Forecast results can be of high importance to researchers primary job responsibilities. Although some research organizations directly influence national and state policies, researchers are less likely to have an immediate ability to affect health outcomes. Similarly, the main activities of most foundations are to conduct research or grant funds to entities for specific projects. The health benefits derived from foundation grants are often byproducts rather than the primary goal of their programs. Foundations are likely very interested in assessing the implications of programs on marginalized subgroups and therefore health forecasting is highly relevant to support their decision-making. Low Opportunity, Low Relevance Most employers, excluding health care employers, do not have primary goals at hand that impact health outcomes. Through the provision of health benefits as well as worksite wellness, mental employee assistance programs, and work place safety programs, they can impact the health of their employers. For example, smoking cessation interventions might increase costs now, but curb health care costs in the future. Health forecasting results will likely be of low importance and relevance to the majority of employers, however, if results show that prevention reduces future health care costs employers, interest level may change. Average Opportunity, Average Relevance The health insurers as a group lie somewhere in the middle in terms of immediate job relevance and ability to shape health outcomes. They may not be as interested in forecasting results as health policymakers, but are interested promoting preventive health to reduce costs on the future. Thus, due to the nature of their business and their ability to affect health outcomes (e.g., providing coverage for prevention programs forecasted to have a large, positive effect on health in the future) is likely higher than employers. In addition, population health status is often UCLA Health Forecasting Team (May 2004) 22

23 relevant to health plans as it associated with lower costs to the health care system. Thus, health plans are an important target for forecast results. Uses As previously mentioned the population health forecasting model has a wide range of potential uses. The model will provide important estimates of current rates of disease prevalence and incidence by key population sub-group; evaluate the effects of maintaining the status quo; identify health-related trends and subsequent changes in disease burdens; assess the effects of programs, policies, and interventions on population health over time; compare the effects of alternative interventions; and increase awareness of disparities in health. Perhaps the best use of the forecasting model is its efficiency for projecting comprehensive results in a timely manner. For example, we can forecasts the direct costs of BMI as it is today compared to the cost if we can reduce BMI levels. These tools can assist in planning and prioritizing activities. In addition, the model s flexibility and ability to incorporate new information as it becomes available will be one of its strengths and ensure its continued use. Utility The true value of a health forecasting model comes from its "real world" applications. The health forecast can advocate for policy changes, plan public health programs or analyze health policy options. Our goal is to use the model as indispensable policy making tool. Throughout the development of the forecast, we plan to share prototypes and sample output with communitybased organizations, health departments, advocacy groups, health plans and policymakers. In order to maximum utility to each target user group, we will conduct a needs assessment among target users to understand how they currently making decisions and what information the health forecasting model can provide them to make more informed decisions. The three scenarios that follow depict the utility of the health forecasting model. UCLA Health Forecasting Team (May 2004) 23

24 Scenario #1: County Health Official The Health Services Resources Administration (HRSA), part of the Department of Health and Human Services, developed the Healthy Communities Access Program (HCAP) to allocate resources for community-based initiatives aimed at better serving uninsured and underinsured individuals. Several communities in California currently participate in the program. A County s Health Services Program consists of several program areas including medical care, behavioral care, and public health. In 2002, the Health Services Program received a grant from HRSA to coordinate efforts to improve the health of specific subpopulations by implementing disease management and health education programs. Unfortunately, the strategies have not been successful in some areas. For example, the rates of diabetes and hypertension are steadily increasing in the county, particularly among Latinos and African Americans. The Director of the County Public Health Department knows that it will soon be time to submit continuation grant applications for further HCAP funding. However, without a new strategy, he fears that the county will not receive continued funding. Knowing the evidence linking sedentary behavior to obesity which is further linked to these health conditions, he decides to include a physical activity component in the HCAP application. After filling his team in on the new approach, they examine the physical activity interventions and programs recommended by the U.S. Task Force on Community Preventive Services for effective options. The first option is a comprehensive community campaign, which includes physical activity promotion messages, support and self-help groups, community health fairs, counseling, worksite education and a toll-free information hotline. The alternative would be more of an environmental approach that would create additional spaces for physical activity such as bicycle and walking paths and playing fields throughout the county. In order to make a case for these programs, he decides to use a modeling approach that will incorporate key components of the population such as age and race/ethnicity composition. The population health forecasting model is used to project health outcomes over the next 10 years. He first runs the "base case" of the forecasting model to determine both the current levels of disease prevalence and what will happen in the future if no immediate actions are taken. Then he inputs data about the population and projected size, scope, effectiveness, and costs of these interventions into the forecasting model. Results indicate that the second program will have better health outcomes for the entire population over the next decade. Finally, the Director of Public Health presents model results to the Board of Supervisors for approval. After assessing the health outcomes, such as disease incidence, of the physical activity program in the whole population and Latino and African American subgroups during the next 10 years, the Board decides to support the application for continued HCAP funding including the new program goals and expenses. UCLA Health Forecasting Team (May 2004) 24

25 Scenario #2: State Senator The chair of the Committee on Education of the state senate is concerned about the dramatic increase in overweight children in California during the last two decades. Given that obesity and being unfit are known risk factors of high blood pressure and diabetes and that these health conditions are linked to increased medical expenditures by the state, she decides to increase the role of California schools in solving the problem. The first component of the solution will be a detailed report outlining that physical fitness plays a key role in keeping children fit and healthy and that many California schools do not comply with existing physical education (PE) requirements. The report will include national statistics including the following: The percentage of overweight children in the U.S. has doubled in the last three decades, At least 300,000 deaths are attributed to physical inactivity and poor nutrition each year, and Physically fit children consistently perform better on academic tasks. In addition, the senator needs to propose a comprehensive statewide program that will effectively increase the level of physical activity among school age children and adolescents by directly promoting physical activity and developing an education that instills the knowledge, behavior, and motivation required to sustain healthy activity levels throughout life. The program would include an explicit physical activity regimen that includes an increase both in the amount of time spent in PE class and the amount of time involving moderate to vigorous levels of physical activity during PE class. The program would also include a physical performance test each year from grades 1-9, the results of which would be provided to the parents for the first 5 years and subsequently reported orally to the student during PE class. Also, the program would embody a new curriculum designed to incorporate physical activity reading and instructional materials into each academic discipline. Qualified instructors such as physical activity specialists and appropriate training for other instructors would be required to prepare lesson plans and oversee annual reporting practices to the state. Finally, the senator needs to quantify the impact of the potential physical activity program on short-term health outcomes such as obesity and incidence of disease. In doing so, she needs to account for migration patterns and socioeconomic status of the population. Thus, she decides to use the population health forecasting model to simulate the effects the program will have on 10- year obesity as well as identify current baseline rates of disease prevalence among subgroups of the school-age population such as Latino, African American, and Asian children and adolescents. Additionally, she plans to assign relevant costs to the program and dollars to the health outcomes to generate a valid cost effectiveness estimate for the Legislature to consider before voting. Also, she plans to link the 10-year health outcomes to 40-year mortality to provide a general idea of long-term health effects attributable to the program. UCLA Health Forecasting Team (May 2004) 25

26 Scenario #3: Health Plan Executive Disease management is a cohesive, organized approach to managing a disease, its complications, and the prevention of comorbidities. Disease management involves a broad spectrum of health care services spanning several modes of delivery including the health care system, providers, and patients. Disease management begins with identifying populations with the disease and subpopulations with specific risk factors followed by the use of accepted clinical guidelines. Accurate tracking systems are then employed to assess health outcomes. Chronic, highly prevalent diseases such as diabetes are costly problems for health plans. In an effort to reduce costs, a health plan executive decides to focus on disease management programs for managed care beneficiaries with Type-II diabetes. The current program consists of system and provider interventions including electronic information systems and mailed reminders respectively. He is interested in other patient-centered interventions including disease selfmanagement education (DMSE) in the home or worksite. Implementing either program will incur substantial costs which must be offset by benefits. S(h)e knows that the analysis should consider the impact of several factors including age, socioeconomic status, and length of membership, therefore he decides to use the population health forecasting model to compare both the effectiveness and costs of the current intervention and the expanded one. After making the necessary assumptions and entering data on the distribution of each factor, the forecast generated results. The findings indicated that the expanded intervention will have a greater impact on health outcomes at a greater cost. To determine whether or not the additional cost is warranted, he assigns a dollar value to the additional health outcomes gained by the expanded intervention at that point. Based on these results, s(h)e determines that costs exceed benefits just 3 years after implementation. Assuming that this program is similar to others in terms of trajectory, s(h)e knows that the benefit-cost ratio would increase over time, after startup costs were incurred and only those costs required for intervention maintenance were considered. Thus s(h)e concludes that the health plan should invest in the program and expect to incur savings after 3 years. UCLA Health Forecasting Team (May 2004) 26

27 Communication and Dissemination Strategy As stated earlier, the true utility of a health forecasting model comes from its "real world" applications. For example, the health effects of policy interventions such as the introduction of more green space, multi-component interventions to reduce tobacco use, and requirements for mandatory physical education for all secondary school students could be modeled within this framework. During the needs assessment, we will determine which interventions and policies to incorporate into the model. In addition, we will obtain feedback from potential users on how to make this information accessible, usable and understandable. As we complete a final user interface, we will develop a comprehensive campaign to build awareness of the model and its benefits, inform and educate potential users, and train target users. The California Health Interview Survey (CHIS) researchers revealed there are many barriers for large and small organizations alike when using health data for advocacy, policy and intervention planning. These difficulties include limited availability of relevant data, inability to analyze findings, lack of knowledge to use the data effectively, and a shortage of resources to perform the analysis (Brown et al. 2005). Thus, there is a significant need for improvements in health data access and analysis. It is important that the Health Forecasting team develop a tool that is user-friendly and mitigates many of the barriers in health data application by maximizing usability for the average community-level user regardless of their technical experience as well as provide ample support for the tool. UCLA Health Forecasting Team (May 2004) 27

28 Figure 1. STAGES OF FORECASTING Formulate Problem Obtain Information Select Methods Implement Methods Evaluate Methods Use Forecasts What will health status of the population of California be in 2020? How will health outcomes differ for subgroups of individuals with different characteristics? What will the health status be if we improved physical activity by 20% for all adults? Demographic Data Existing Health Status Risk Factors Future Trends Microsimulation Extrapolation Techniques Judgmental Bootstrapping Reasonable Alternatives Testing Assumptions Testing Data and Methods Replicating Outputs Assessing Outputs Prediction Intervals Gaining Acceptance for Forecast Monitoring Forecasts UCLA Health Forecasting Team (May 2004) 28

29 Figure 2. MICROSIMULATION ALLOWS FOR SYNTHESIS OF THE DATA AT ALL LEVELS Aggregate Models Micro Simulation Models Demographic Models Time Factor Trending Model Covariates Discrete Time Model Heart Health Policy Model Naïve Model Covariates Cell Based Model Time Factor Combined Model PREVENT Tobacco Control Distributional Data on Individuals Continuous Time Model Facilitate Simultaneous Changes in Multiple Factors POHEM Global Burden of Disease Type of Model Based on Organization of the Data Data Extension UCLA Health Forecasting Team (May 2004) 29

30 Figure 3. APPROPRIATE FORECASTING TECHNIQUES CHARACTERISTICS OF FORECASTING METHODS Based on information available and type of analysis we choose a variety of forecasting techniques and compare results: Each of the components in the model requires extensive modeling and a large number of assumptions. In order to create projections for health outcomes the future values of many other variables, such as fertility, behavior and racial makup, need to be estimated first these estimates come from external sources, are provided by experts in the field or can be developed using statistical models. Knowledge Source judgmental statistical self others univariate multivariate role no role theory-based data-based Role Playing Intentions Expert Opinions Extrapolation Models Multivariate Models Analogies Conjoint Analysis Judgmental Bootstrapping Rule-Based Forecasting Expert Systems Econometric Models Source: J. Scott Armstrong, Principles of Forecasting, Kluwer Publishing 2001 Techniques currently used by project team Techniques to be incorporated in future UCLA Health Forecasting Team (May 2004) 30

31 Figure 4. Descriptive Population Framework Population model including socioeconomic and demographic information of the population of interest includes variables such as gender, age, race/ethnicity, education, income, etc BUILDING THE POPULATION HEALTH FORECASTING MODEL + Risk Factor/Disease Modules Smaller models that describe linkages between individual risk factors, environment effect, socio-economic and demographic characteristics and health outcomes + Forecasting Module Future trends of assumptions and underlying data of risk factor/disease modules and the population framework The model is built around a microsimulation setting, allowing for inclusion of joint distributions as well as analysis of complex interactions, and distributional information on outcomes UCLA Health Forecasting Team (May 2004) 31

32 Figure 5. STAKEHOLDERS COULD USE THE RESULTS DIRECTLY OR INDIRECTLY FOR POLICY, RESEARCH AND BUSINESS DECISIONS Relevance of Health Forecast for Primary Objectives High Researchers Legislative Analyst s Office Employers State Dept. of Finance Foundations Health Plans Elected Officials County Dept. of Health State Dept. of Health Low Boards of Education Health Care Providers Air Quality Mgmt District Low High Opportunity to Directly Impact Health Outcomes UCLA Health Forecasting Team (May 2004) 32

33 References Ahlburg, D. (2001). Population forecasting. In Principles of Forecasting: A Handbook for Researchers and Practitioners. Armstrong, J.S. (Ed), Norwell, MA: Kluwer Academic Publishers Ainsworth, B., Haskell WL, Whitt MC, Irwin ML, Swartz AM, Strath SJ, O Brien WL, Bassett DR Jr, Schmitz KH, Emplaincourt PO, Jacobs DR Jr, & Leon AS. (2000). Compendium of physical activities: an update of activity codes and MET intensities. Med Sci Sports Exerc 32(9S), S498-S516 Armstrong J. (Ed). (2001). Principles of Forecasting: A handbook for researchers and practitioners. Norwell, MA: Kluwer Academic Publishers Brown, ER, Holtby, S., Zahnd, E., & Abbott, G. (2005). Community-based Participatory Research in the California Health Interview Survey. Preventing Chronic Disease. Public Health Research, Practice, and Policy Chatfield C. (2001). Prediction Intervals for Time-Series Forecasting. In Principles of Forecasting: A Handbook for Researchers and Practitioners. Armstrong, J.S. (Ed), Norwell, MA: Kluwer Academic Publishers Citro C, & Hanushek E. (eds). (1991). The Uses of Microsimulation Modelling, vol. 1, Review and Recommendations, National Academy Press, Washington, D.C. Dannenberg AL, Bhatia, R, Cole BL, Dora C, Fielding FE, Kraft K, McClymont-Peace D, Mindell J, Onyekere C, Robert JA, Ross CL, Rutt CD, Scott-Samuel A, & Tilson HH. (2006). Growing the Field of Health Impact Assessment in the United States: An Agenda for Research and Practice. American Journal of Public Health 96(2),19-27 Goldman L, Phillips KA, Coxson P, Goldman PA, Williams L, Hunink MG, & Weinstein MC. (2001). The effect of risk factor reductions between 1981 and 1990 on coronary heart disease incidence, prevalence, mortality and cost. Journal of American College of Cardiology 38, Lam J, & Veall M. (2002). Bootstrap prediction intervals for single period regression forecasts. International Journal of Forecasting 18, Lee R, & Carter L. (1992). Modeling and forecasting U.S. mortality. Journal of the American Statistical Association 87, Koopman J. (2002). Modeling infection transmission the pursuit of complexities that matter. Epidemiology 13, Manton K, Stallard E, & Singer B. (1992). Projecting the future size and health status of the U.S. elderly population. International Journal of Forecasting 8, UCLA Health Forecasting Team (May 2004) 33

34 Myers M, Rogers D, Cox J, Flashault A, & Hay S. (2000). Forecasting disease risk for increased epidemic preparedness in public health. Advancement in Parasitology 47, Northridge M, Sclar E, & Biswas P. (2003). Sorting Out the Connections Between the Built Environment and Health: A Conceptual Framework for Navigating Pathways and Planning Healthy Citites. Journal of Urban Health: Bulletin of the New York Academy of Medicine 80(4), Sherden, W. (1998). The Fortune Sellers: The Big Business of Buying and Selling Predictions, New York: Wiley Webby R., O Connor M, & Lawrence M. (2001), Judgmental time-series forecasting using domain knowledge, in J. S. Armstrong (ed.), Principles of Forecasting. Norwell, MA: Kluwer Academic Press. World Health Organization. (2002). Methods summaries for risk factors assessed in chapter 4 (Quantifying selected major risks in health), World Health Organization UCLA Health Forecasting Team (May 2004) 34